digest | Breakthrough augmented reality medical tool

This system projects an image of the patient's internal body.
May 10, 2021

image | above

The ProjectDR augmented reality system projects a diagnostic medical image of a spinal injury directly onto the patient’s skin. The 3D overlay effect is visible using virtual reality eyewear goggles called a headset, or hand-held glass panels specialized for AR display.

— contents —

~ story
~ featurettes
~ webpages
~ selected reading

— story —

New tech — called Project DR — is bringing the power of augmented reality into clinical research + hospitals. The system projects medical images — like CT scans or MRI scans — directly onto a patient’s skin. This gives physicians a 3D visual of the patient from the outside + inside. With both views, a physician can virtually see inside the patient — since the images are mapped to the patient’s body from above.

The project was created by researchers at the Univ. of Alberta, in Canada.

How it functions.

ProjectDR uses a system called motion capture. Special markers are placed on the patient’s body. The space is flooded with infrared light that bounces off the markers. The computer system uses infrared cameras (that’s a type of thermal camera) that can see the markers — and keep-up with them as they move.

By tracking the body markers, ProjectDR knows just where to project the medical scan images onto the patient’s body. It maps the medical scans to the anatomical markers. This creates an effect — like you’re looking inside the human body. And it can follow the patient moving — so the scans can be projected continuously.

To understand a motion capture system — and for a primer on virtual reality, augmented reality, and mixed reality — see the good featurettes below. They show actual footage of virtual experiences + 3D demos — explaining this tech.

The project’s supervising researcher Pierre Boulanger PhD said:

Our ProjectDR system is an augmented reality (AR) software platform that projects medical diagnostic images onto the skin surface of a patient.

This system corrects the image if there’s distortion from skin contours — and automatically adjusts to fit the size + shape of the patient. Our software uses 2D or 3D diagnostic image formats. And the output can be viewed through see-through or projected displays.

Benefits for the future.

Some applications for this AR system include:

  • medical education
  • medical research
  • physio-therapy
  • laparoscopic surgery
  • surgical planning

ProjectDR can also project limited sections of body scans — for example: only lungs, skeleton, or blood vessels — depending on what a physician wants to examine. Researcher Pierre Boulanger PhD said:

We’ll demo ProjectDR in an operating room — in a surgical simulation lab — to test the pros + cons in surgeries. We’re also doing pilot studies to test this system for teaching physical therapy techniques. Next we’ll conduct real-life surgical pilot studies.

Team member Ian Watts explained:

We wanted to create a system that would show clinicians a patient’s internal anatomy — within the context of the body. The difficult part is having the image track properly on the patient’s body — even as they shift + move. I’m improving the system’s automatic calibration and adding depth sensors.


1. |

school: Univ. of Alberta • Canada
motto: Whatsoever things are true.
web: home • main channelscience channel

featurette title: ProjectDR demo
watch | featurette

— summary —

ProjectDR is an augmented reality system that allows medical images — such as CT scans + MRI data — to be displayed directly on a patient’s body, in a way that moves as the patient does.

2. |

school: Full Sail Univ.
motto: Explore degrees in entertainment, media, arts, and tech.
web: homechannel

featurette title: What is motion capture?
watch | featurette

— summary —

From films + video games — to the military, sports, and medical fields — motion capture is used to record the movement of objects or people. And create 3D animated models. Teacher Tyrone Jordan demos the basics of motion capture.

3. |

group: Simplilearn
motto: We transform lives by empowering people via digital skills.
web: homechannel

featurette title: The rise of tech: VR (virtual reality), AR (augmented reality), and MR (mixed reality)
watch | featurette

— summary —

These immersive reality systems: VR (virtual reality), AR (augmented reality), and MR (mixed reality) — are amongst the fastest-growing computer technologies today, involving both hardware + software. In a nutshell, immersive tech creates or extends our sensory experience.

It immerses the user in a digital environment — that’s visual, auditory, and even tactile. It can be a pure fantasy world, or a realistic model, or overlays that appear on top of the actual world around us. This tech is gaining momentum and has lots of applications. Watch this video to understand what immersive tech is — and how it’s useful in re-imagining our future.

— webpages —

school: Univ. of Alberta • Canada
motto: Whatsoever things are true.
web: homemain channelscience channel

projects: ProjectDR

— webpages —

name: Pierre Boulanger • PhD
web: home

selected reading

1. |

publication: New Atlas
tag line: Extraordinary ideas moving the world forward.
web: homechannel

story title: ProjectDR allows doctors to see into patients’ bodies
read | story

— summary —

Imagine if doctors could see through a patient’s skin. And their perspective of the underlying bones + organs changed accordingly — as the person moved around. That’s what scientists at the Univ. of Alberta in Canada have developed. It’s still in the experimental phase, and is called ProjectDR.

2. |

publication: Futurism
tag line: Wonder what’s next.
web: home • channel

story title: Doctors can now use augmented reality to peek under a patient’s skin
read | story

— summary —

A new tech takes some of the guesswork out of medicine.

presented by

group: Singularity Univ.
tag line: We prepare you to seize exponential opportunities.
web: homechannel

3. |

publication: the Richard van Hooijdonk blog
tag line: A keynote speaker, trend-watcher, and futurist.
web: home • channel • blog

story title: ProjectDR uses new augmented reality tech to let doctors see through your skin
read | story
watch | featurette

— summary —

A pilot program at the Univ. of Alberta brings digital magic to the operating room — so surgeons can see inside their patients before they make the first incision. The researchers have developed a system called ProjectDR, a revolutionary augmented reality tech for medical teaching + patient care.

— notes + abbreviations —

AR = augmented reality
VR = virtual reality

CT scan = computed tomography scan
MRI scan = magnetic resonance imaging scan

* physio-therapy is also called physical therapy

[ post file ]

post title: digest | Breakthrough augmented reality medical tool
deck: This system projects an image of the patient’s internal body.

collection: the Kurzweil library
tab: stories on progress

[ end of file ]