Researchers Modify Kinect Gaming Device to Scan in 3-D

August 3, 2011

Making a freehand 3-D scan of Tiffany Fox in Calit2's StarCAVE using a modified Kinect. Though Fox's avatar appears to be projected onto the wall behind her, the scan is actually projected in the middle of the StarCAVE (credit: UCSD)

Researchers at the University of California, San Diego, preparing for a future archaeological dig to Jordan, will likely pack a Microsoft Kinect Xbox 360 in the field to take high-quality, low-cost 3-D scans of dig sites.

The researchers have figured out a way to extract data streaming from the Kinect’s onboard color camera and infrared sensor to make hand-held 3-D scans of small objects and people. The quickly-made avatars could conceivably be plugged right into virtual worlds such as Second Life.

The ultimate goal, however, is to extend the technology to scan entire buildings and even neighborhoods, the researchers said. For the initial field application of their modified Kinect — dubbed ArKinect (a mashup of archaeology and Kinect) —  the researchers plan to train engineering and archaeology students to use the device to collect data on a future expedition to Jordan.

The scans collected at sites in Jordan or elsewhere can later be made into 3-D models and projected in Calit2’s StarCAVE, a 360-degree, 16-panel immersive virtual reality environment that enables researchers to interact with virtual renderings of objects and environments.

Three-dimensional models of artifacts provide more information than 2-D photographs about the symmetry (and hence quality of craftsmanship, for example) of found artifacts, and 3D models of the dig sites can help archaeologists keep track of the exact locations where artifacts were located.

The steps for making a 3D reconstruction of a real-life stuffed bear (far left) include: 1) projecting a pattern of infrared dots onto the bear to construct a depth map (second from left); 2) connecting nearby dots with a triangular mesh grid (third from left); 3) filling in each triangle in the grid with color and texture information from the Kinect's color camera (far right) (credit: UCSD)

The ability to operate the Kinect freehand is a huge advantage over other scanning systems like LIDAR (light detecting and ranging), which creates a more accurate scan but has to be kept stationary in order to be precisely aimed.

The Kinect projects a pattern of infrared dots (invisible to the human eye) onto an object, which then reflect off the object and get captured by the device’s infrared sensor. The reflected dots create a 3D depth map. Nearby dots are linked together to create a triangular mesh grid of the object. The surface of each triangle in the grid is then filled in with texture and color information from the Kinect’s color camera. A scan is taken 10 times per second and data from thousands of scans are combined in real-time, yielding a 3D model of the original object or person.

The Kinect streams data at 40 megabytes per second. Keeping the amount of stored data to a minimum will allow a scan of a person to occupy only a few hundred kilobytes of storage, about the same as a picture taken with a digital camera.

Another advantage of the Kinect is cost: It retails for $150. This low price tag, coupled with the researcher’s efforts to make it a portable self-contained, battery-powered instrument with an onboard screen to monitor scan progress, makes it feasible to send an ArKinect to Jordan.