A new augmented reality unit developed by the European Space Agency (ESA) could provide just-in-time medical expertise to astronauts, using a head-mounted display for 3D guidance in diagnosing problems or even performing surgery.

The Computer Assisted Medical Diagnosis and Surgery System, CAMDASS, is a wearable augmented-reality prototype.

Augmented reality merges actual and virtual reality by precisely combining computer-generated graphics with the wearer’s view.

CAMDASS is focused for now on ultrasound examinations but in principle could guide other procedures. Ultrasound is a versatile and effective medical diagnostic tool, and already available on the International Space Station.

Future astronauts venturing further into space must be able to look after themselves. Depending on their distance from Earth, discussions with experts on the ground will involve many minutes of delay or even be blocked entirely.

“Although medical expertise will be available among the crew to some extent, astronauts cannot be trained and expected to maintain skills on all the medical procedures that might be needed,” said Arnaud Runge, a biomedical engineer overseeing the project for ESA.

How it works

CAMDASS uses a stereo head-mounted display and an ultrasound tool tracked via an infrared camera. The patient is tracked using markers placed at the site of interest. An ultrasound device is linked with CAMDASS and the system allows the patient’s body to be registered to the camera and the display calibrated to each wearer’s vision.

3D augmented reality cue cards are then displayed in the headset to guide the wearer. These are provided by matching points on a virtual human and the registered patient. This guides the wearer to position and move the ultrasound probe. Reference ultrasound images give users an indication of what they should be seeing, and speech recognition allows hands-free control.

“Once it reaches maturity, the system might also be used as part of a telemedicine system to provide remote medical assistance via satellite,” said Runge. It could be deployed as a self-sufficient tool for emergency responders as well. It would be interesting to perform more testing in remote locations, in the developing world and potentially in the Concordia Antarctic base. Eventually, it could be used in space.”

Funded by ESA’s Basic Technology Research Programme, the prototype was developed for the Agency by a consortium led by Space Applications Services NV in Belgium with support from the Technical University of Munich and the DKFZ German Cancer Research Centre.