Congenitally blind learn to see and read with soundscapes

November 9, 2012

Example of seeing an object with sound (credit: Striem-Amit et al./Neuron)

Congenitally blind people have learned to ”see” and describe objects, and even identify letters and words, by using a visual-to-auditory sensory-substitution algorithm and sensory substitution devices (SSDs), scientists at Hebrew University and in France have found.

SSDs are non-invasive sensory aids that provide visual information to the blind via their existing senses. For example, using a visual-to-auditory SSD in a clinical or everyday setting, users wear a miniature camera connected to a small computer (or smart phone) and stereo headphones.

The images are converted into “soundscapes,” using an algorithm, allowing the user to listen to and then interpret the visual information coming from the camera. The blind participants using this device reach a level of visual acuity technically surpassing the criterion of the World Health Organization (WHO) for blindness.

The study shows that following 70 hours of unique training developed in the Amedi lab, the blind people could easily use SSDs to characterize images into object categories, such as images of faces, houses, body shapes, everyday objects and textures. They could also identify even more complex everyday objects — locating people’s positions, identifying facial expressions, and even reading letters and words (for demos, movies and further information:

The Hebrew University study also tested what happens in the brain when the blind learn to see with sounds. They used functional magnetic resonance imaging (fMRI) to measure the neural activity of people blind from birth as they “saw” — using the SSD — high-resolution images of letters, faces, houses, everyday objects and body-shapes.

(Credit: Striem-Amit et al./Neuron)

Surprisingly, not only was their visual cortex activated by the sounds, their brain showed selectivity for visual categories that characterize the normally developing, sighted brain.

A specific part of the brain, known as the Visual Word Form Area, or VWFA — first discovered in sighted people by Profs. Laurent Cohen and Stanislas Dehaene of Pitie-Salpétriere Hospital-INSERM-CEA, of France, co-authors of the current article — is normally very selective.

In sighted people, it has a role in reading, and is activated by seeing and reading letters more than by any other visual object category. This was also found in this area in people deprived of sight. Their VWFA, after only tens of hours of training in SSD use, showed more activation for letters than for any of the other visual categories tested.

In fact, the VWFA was so plastic to change, that it showed increased activation for SSD letters after less than two hours of training by one of the study participants.

“The adult brain is more flexible that we thought,” says Prof. Amir Amedi of the Edmond and Lily Safra Center for Brain Sciences and the Institute for Medical Research Israel-Canada at Hebrew University. This and other recent research from various groups have also demonstrated that multiple brain areas are not specific to their input sense (vision, audition or touch), but rather to the task, or computation they perform.

All of this suggests that in the blind, brain areas might potentially be “awakened” to processing visual properties and tasks even after years or maybe even lifelong blindness, if the proper technologies and training approaches are used, says Amedi.

The findings also give hope that reintroduced input into the visual centers of the blind brain could potentially restore vision, and that SSDs might be useful for visual rehabilitation.

“SSDs might help blind or visually-impaired individuals learn to process complex images, as done in this study, or they might be used as sensory interpreters that provide high-resolution, supportive, synchronous input to a visual signal arriving from an external device such as bionic eyes,” says Prof. Amedi.