The day that robots help children with autism to learn social skills is a step closer with the development of a system that allows a robot to monitor a child’s emotional state.
“A lot of research going on around the world today tries to use robots to treat children with autism spectrum disorders (ASD). That research shows that the children are attracted to robots, raising the promise that appropriately designed robots could play an important role in their treatment,” says Nilanjan Sarkar, associate professor of mechanical engineering at Vanderbilt. “Efforts so far have been quite limited because researchers haven’t had a way to monitor the emotional state of the children, which would allow the robot to respond automatically to their reactions.”
One baby in 150 born today in the United States is diagnosed with ASD. Autism currently costs the U.S. more than $90 billion per year—a cost projected to double by 2017 due to the growing population of those affected.
Sarkar has developed a method that uses physiological measurements, including heart rate, galvanic skin response, temperature and muscle response, to monitor emotional state. His original motivation was to improve human–robot interactions. When his nephew was diagnosed with autism, however, Sarkar got the idea of applying the technique to aid children with ASD. He sought out one of the leading authorities on the subject, Wendy Stone, professor of pediatrics and investigator at Vanderbilt’s Kennedy Center, and they formed a partnership to develop this new approach.
Last fall, Sarkar and Stone published two papers—one in the IEEE Transactions on Robotics and one in the International Journal of Human–Computer Studies—that describe the results of their first set of experiments, conducted with six children ranging in age from 13 to 16 years who had been diagnosed with ASD.
Physiological sensors were attached to the participants, and they were asked to play two games. One was the computer game Pong. The other was a variant of Nerf basketball, with the hoop and backboard attached to the end of a robotic arm that moves them back and forth or up and down. Graduate students Changchun Liu and Karla Conn participated in the studies.
The data they gathered can be used to develop individual models that can predict emotional state with an accuracy of better than 80 percent. This information can be used in real time to increase a child’s degree of engagement.
“That’s the part that really nailed me,” says Stone, “that the robot can read the physiological cues of the person playing the game, controlling the distance and angle of the hoop, and that the person playing reported a more positive mood when the computer was responsive to his needs.”
Ability to monitor emotional state is particularly important in treating ASD, Stone says. “Children with autism are not necessarily giving the kind of emotional cues that we know how to read. They are not necessarily good reporters of their inner feelings.”
As the children played Pong, the game was changed in several ways: Ball and paddle speeds were varied, and computer-based opponents of different skill levels were introduced. This allowed researchers to induce emotions of interest, boredom, anxiety and engagement. The model was then used to predict how each child would react. When they switched to robot basketball, the model’s predictions were equally accurate.
A robot’s ability to provide consistent and predictable responses should be particularly useful for treating ASD. Each child uses individual triggers such as direct eye contact or a loud voice. Once a trigger is identified, a robot could be programmed to increase the stimulus at a gradual rate the child doesn’t notice. The robot could back off when it senses that its responses are beginning to agitate the child. In this fashion, it could build up the child’s tolerance to problem stimuli. “Robots can be programmed to respond with a consistency that is difficult for humans to achieve,” Sarkar points out.
And something that robots lack also may be advantageous in this setting. “The children can be distracted by a lot of sensory stimuli coming at them,” says Stone. “Alternative methods of teaching that can remove the social component could be very helpful.”
The research was supported by a grant from the Marino Autism Research Institute.