Researchers determine how the brain controls robotic grasping tools

Could lead to advancements in assistive technologies benefiting the disabled
February 3, 2015

Planning and execution phase responses in a manual task. (A) Relative to resting baseline, both types of action planning were associated with significant increases in occipital cortex, extending dorsally into the medial superior parietal lobule, left premotor cortex, bilateral TPJ, and cMTG. (B) During movement execution, grasp-related increases in activity were found near the intersection of the IPS and postcentral sulcus contralateral to the hand involved. (Credit: Scott H. Frey et al./ Journal of Cognitive Neuroscience)

University of Missouriresearchers have found evidence that the cerebellum portion of the brain may play a critical role in the complex network of brain functions involved in grasping. Their findings could lead to advancements in assistive technologies benefiting the disabled.

“For those with disabilities, assistive technologies, such as robotic arms or sensors inserted in the brain, make it possible to accomplish actions like grasping with the press of a button or directly through brain activity; however, little is known about how the human brain adapts to these technologies,” said Scott Frey, professor of psychological sciences in the College of Arts and Science and director of the Brain Imaging Center at MU.

Frey’s team found that the brain is not natively able to control tools, such as robotic arms, but that the cerebellum, an ancient portion of our brain that has remained relatively unchanged, does plays a vital role in helping us reach and grasp with these tools — often with only minimal training.

In the study, participants completed a series of ordinary reaching and grasping tasks involving colored wooden blocks. Regions of the brain were monitored by functional magnetic resonance imaging (fMRI).

Then, in a training session, participants were introduced to a robotic arm that performed the same reaching and grasping tasks when they pressed specific buttons. Participants were told that the next day’s tasks would involve their controlling the robot remotely by video feed from within an MRI scanner.

“We found evidence that the brain is very flexible and can be rapidly conditioned to associate new consequences with a variety of movements,” Frey said.

“Pressing a button is a very simple act that does not naturally result in grasping. Nevertheless, after subjects learned that pressing one button would result in grasping objects with a robotic arm, this same movement resulted in a dramatically different pattern of brain activity than pressing an identical button known by them to have no effect on the robot’s behavior. Localized activity within the [cerebellum] increased dramatically. These findings suggests that we might look to the cerebellum when seeking potential targets for brain-controlled interfaces.”

For many years the cerebellum, an ancient structure that contains more neurons than the rest of the brain combined, was believed to control very basic motor and balance functions, Frey said. Results from this study provide further evidence of the cerebellum’s role in higher cognitive functions.

Research was funded in part by the Army Research Laboratory’s Army Research Office and the National Institutes of Health National Institute of Neurological Disease.


Abstract of Grasping with the press of a button: grasp-selective responses in the human anterior intraparietal sulcus depend on nonarbitrary causal relationships between hand movements and end-effector actions
Evidence implicates ventral parieto-premotor cortices in representing the goal of grasping independent of the movements or effectors involved [Umilta, M. A., Escola, L., Intskirveli, I., Grammont, F., Rochat, M., Caruana, F., et al. When pliers become fingers in the monkey motor system. Proceedings of the National Academy of Sciences, U.S.A., 105, 2209–2213, 2008; Tunik, E., Frey, S. H., & Grafton, S. T. Virtual lesions of the anterior intraparietal area disrupt goal-dependent on-line adjustments of grasp. Nature Neuroscience, 8, 505–511, 2005]. Modern technologies that enable arbitrary causal relationships between hand movements and tool actions provide a strong test of this hypothesis. We capitalized on this unique opportunity by recording activity with fMRI during tasks in which healthy adults performed goal-directed reach and grasp actions manually or by depressing buttons to initiate these same behaviors in a remotely located robotic arm (arbitrary causal relationship). As shown previously [Binkofski, F., Dohle, C., Posse, S., Stephan, K. M., Hefter, H., Seitz, R. J., et al. Human anterior intraparietal area subserves prehension: A combined lesion and functional MRI activation study. Neurology, 50, 1253–1259, 1998], we detected greater activity in the vicinity of the anterior intraparietal sulcus (aIPS) during manual grasp versus reach. In contrast to prior studies involving tools controlled by nonarbitrarily related hand movements [Gallivan, J. P., McLean, D. A., Valyear, K. F., & Culham, J. C. Decoding the neural mechanisms of human tool use.Elife, 2, e00425, 2013; Jacobs, S., Danielmeier, C., & Frey, S. H. Human anterior intraparietal and ventral premotor cortices support representations of grasping with the hand or a novel tool. Journal of Cognitive Neuroscience, 22, 2594–2608, 2010], however, responses within the aIPS and premotor cortex exhibited no evidence of selectivity for grasp when participants employed the robot. Instead, these regions showed comparable increases in activity during both the reach and grasp conditions. Despite equivalent sensorimotor demands, the right cerebellar hemisphere displayed greater activity when participants initiated the robot’s actions versus when they pressed a button known to be nonfunctional and watched the very same actions undertaken autonomously. This supports the hypothesis that the cerebellum predicts the forthcoming sensory consequences of volitional actions [Blakemore, S. J., Frith, C. D., & Wolpert, D. M. The cerebellum is involved in predicting the sensory consequences of action. NeuroReport, 12, 1879–1884, 2001]. We conclude that grasp-selective responses in the human aIPS and premotor cortex depend on the existence of nonarbitrary causal relationships between hand movements and end-effector actions.