How to see with your ears

Device offers new alternative to blind people; subjects could “see” letters with 20/400 vision
July 9, 2013

A participant wearing camera glasses and listening to the soundscape (credit: Alastair Haigh/Frontiers in Psychology)

A device that trains the brain to turn sounds into images could be used as an alternative to invasive treatment for blind and partially-sighted people, researchers at the University of Bath have found.

“The vOICe” is a visual-to-auditory sensory substitution device that encodes images taken by a camera worn by the user into “soundscapes” from which experienced users can extract information about their surroundings.

It helps blind people use sounds to build an image in their minds of the things around them.

A research team, led by Dr Michael Proulx, from the University’s Department of Psychology, looked at how blindfolded sighted participants would do on an eye test using the device.

The vOICe program was run on “slow motion” setting. It  scanned images from left to right, producing soundscapes with a duration of 2 seconds, and in “negative video” mode, in which dark areas correspond to loud sounds and white areas produce no sound.

They were asked to perform a standard eye chart test called the Snellen Tumbling E test, which asked participants to view the letter E turned in four different directions and in various sizes. Normal, best-corrected visual acuity is considered 20/20, calculated in terms of the distance (in feet) and the size of the E on the eye chart.

Four orientations of Snellen E converted to“soundscape” stimuli (credit: Alastair Haigh/Frontiers in Psychology)

The participants, even without any training in the use of the device, were able to perform the best performance possible: nearly 20/400. This limit appears to be the highest resolution currently possible with the ever-improving technology.

Dr Michael Proulx said: “This level of visual performance exceeds that of the current invasive techniques for vision restoration, such as stem cell implants and retinal prostheses after extensive training.

“A recent study found successful vision at a level of 20/800 after the use of stem cells. Although this might improve with time and provide the literal sensation of sight, the affordable and non-invasive nature of The vOICe provides another option.

“Sensory substitution devices are not only an alternative, but might also be best employed in combination with such invasive techniques to train the brain to see again or for the first time.”