Body language-based gameplay

UK university developing technology with natural responses to human interaction
August 14, 2012

Researchers at Goldsmiths College led by Dr. Marco Gillies have enlisted theater performers to help design computer software capable of recognizing and responding to human body language.

Using the latest generation of motion detectors, including Microsoft’s Kinect camera, actors’ interactions with members of the public and their responses to specific body language are recorded by the software.

“Two people can take on the roles of the video game character and the player, showing how the character should respond by acting out the movements themselves,” explained Gillies.

“The software enables video game characters to move in a more natural way, responding to the player’s own body language rather than mathematical rules.”

Traditionally, the creators of interactive characters are programmers and animators, but with this project Gillies and his team are putting this task in the hands of people with artistic rather than technical knowledge.

“Our hypothesis is that the actors’ artistic understanding of human behavior will bring an individuality, subtlety and nuance to the character that it would be difficult to create in hand-authored models,” said Dr Gillies.

“These are the kinds of everyday movements, that we do unconsciously, which make them hard to program in the conventional way.”

The research intends to help interactive media represent more nuanced social interaction and broaden its range of applications.

The system is available to trial for those who would like a hands-on demonstration.