Kinect lets robots learn and navigate 3D environments

February 17, 2012

The researchers used a PR2 robot, developed by Willow Garage, with Microsoft's Kinect sensor to test their system (credit: Hordur Johannsson/MIT)

Robots could one day navigate through constantly changing surroundings with virtually no input from humans, thanks to a system that allows them to build and continuously update a three-dimensional map of their environment using a low-cost camera such as Microsoft’s Kinect.

The system, being developed by researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), could also allow blind people to make their way unaided through crowded buildings such as hospitals and shopping malls.

To explore unknown environments, robots need to be able to map them as they move around — estimating the distance between themselves and nearby walls, for example — and to plan a route around any obstacles.

But while a large amount of research has been devoted to developing one-off maps that robots can use to navigate around an area, these systems cannot adjust to changes in the surroundings over time, says Maurice Fallon, a research scientist at CSAIL  “If you see objects that were not there previously, it is difficult for a robot to incorporate that into its map.”

Kinect builds 3-D model

The new approach, based on a technique called Simultaneous Localization and Mapping (SLAM), will allow robots to constantly update a map as they learn new information over time, he says. The team shown how a robot can locate itself in such a map with just a low-cost Kinect-like camera.

As the robot travels through an unexplored area, the Kinect sensor’s visible-light video camera and infrared depth sensor scan the surroundings, building up a 3-D model of the walls of the room and the objects within it. Then, when the robot passes through the same area again, the system compares the features of the new image it has created — including details such as the edges of walls, for example — with all the previous images it has taken until it finds a match.

At the same time, the system constantly estimates the robot’s motion, using on-board sensors that measure the distance its wheels have rotated. By combining the visual information with this motion data, it can determine where within the building the robot is positioned. Combining the two sources of information allows the system to eliminate errors that might creep in if it relied on the robot’s on-board sensors alone, Fallon says.

Once the system is certain of its location, any new features that have appeared since the previous picture was taken can be incorporated into the map by combining the old and new images of the scene, Fallon says.

The team tested the system on a robotic wheelchair, a PR2 robot developed by Willow Garage in Menlo Park, Calif., and in a portable sensor suite worn by a human volunteer. They found it could locate itself within a 3-D map of its surroundings while traveling at up to 1.5 meters per second.

Uses: offices, hospitals, blind people, the military

Ultimately, the algorithm could allow robots to travel around office or hospital buildings, planning their own routes with little or no input from humans, Fallon says.

It could also be used as a wearable visual aid for blind people, allowing them to move around even large and crowded buildings independently, says Seth Teller, head of the Robotics, Vision and Sensor Networks group at CSAIL and principal investigator of the human-portable mapping project.

“There are also a lot of military applications, like mapping a bunker or cave network to enable a quick exit or re-entry when needed,” he says. “Or a HazMat team could enter a biological or chemical weapons site and quickly map it on foot, while marking any hazardous spots or objects for handling by a remediation team coming later. These teams wear so much equipment that time is of the essence, making efficient mapping and navigation critical.”

“This opens up exciting new possibilities in robot research and engineering, as the old-school ‘flatland’ assumption that the scientific community has been using for many years is fundamentally flawed,” says Radu Rusu, a research scientist at Willow Garage. “Robots that fly or navigate in environments with stairs, ramps and all sorts of other indoor architectural elements are getting one step closer to actually doing something useful. And it all starts with being able to navigate.”