Using your WiFi for gesture recognition

June 5, 2013

A hand gesture changes the TV channel using WiSee technology (credit: University of Washington)

University of Washington computer scientists have developed gesture-recognition technology called “WiSee” that uses ambient Wi-Fi signals to detect specific movements (to turn off lights or flip through songs, for example )without needing sensors on the human body or cameras.

The team includes Shwetak Patel, an assistant professor of computer science and engineering and of electrical engineering and his lab.

By using an adapted Wi-Fi router and a few wireless devices in the living room, users could control their electronics and household appliances from any room in the home with a simple gesture.

“This is repurposing wireless signals that already exist in new ways,” said lead researcher Shyam Gollakota, a UW assistant professor of computer science and engineering.

The concept is similar to Kinect — which uses cameras to recognize gestures — but the UW technology is simpler, cheaper, and doesn’t require users to be in line of sight, or even in same room as the device they want to control. That’s because Wi-Fi signals can travel through walls and aren’t bound by line-of-sight or sound restrictions.

WiSee technology uses multiple antennas to focus on one user to detect the person’s gesture (credit: University of Washington)

As described in an open-access working-draft paper, the UW researchers built a “smart” receiver device that monitors all of the wireless transmissions coming from devices throughout a home, including smartphones, laptops and tablets. A standard Wi-Fi router could be adapted to function as a receiver.

When a person moves, there is a slight change in the frequency of the wireless signal. Moving a hand or foot causes the receiver to detect a pattern of changes known as a Doppler frequency shift (which happens when airplane sounds change pitch when approaching or going away).

The system requires one receiver with multiple antennas. Each antenna tunes into a specific user’s movements, so as many as five people can move simultaneously in the same residence without confusing the receiver. To use the WiSee, you would perform a specific repetition gesture sequence to get access to the receiver.

These frequency changes are very small — only several hertz — compared with Wi-Fi signals, which that have a 20 megahertz bandwidth and operate at 2.4 or 5 gigahertz. Researchers developed an algorithm to detect these slight shifts. The technology also accounts for gaps in wireless signals when devices aren’t transmitting.

WiSee can detect and classify these nine gestures with an average accuracy of 94% (credit: University of Washington)

The technology can identify nine different whole-body gestures, ranging from pushing, pulling and punching to full-body bowling. The researchers tested these gestures with five users in a two-bedroom apartment and an office environment. Out of the 900 gestures performed, WiSee accurately classified 94 percent of them.

“This is the first whole-home gesture recognition system that works without either requiring instrumentation of the user with sensors or deploying cameras in every room,” said Qifan Pu, a collaborator and visiting student at the UW.

This password concept would also keep the system secure and prevent a neighbor — or hacker — from controlling a device in your home. The receiver would be programmed to understand that a specific gesture corresponds to a specific device.

Collaborators Patel and Sidhant Gupta, a doctoral student in computer science and engineering, have worked with Microsoft Research on two similar technologies:SoundWave, which uses sound, and Humantenna, which uses radiation from electrical wires. Both also sense whole-body gestures. But WiSee doesn’t require the user to be in the same room as the receiver or the device.

In this way, a smart home could become a reality, allowing you to turn off the oven timer with a simple wave of the hand, or turn on the coffeemaker from your bed.

The researchers plan to look next at the ability to control multiple devices at once. The initial work was funded by the UW department of computer science and engineering.

No information on availability of a product on instructions for building a device (aside from the open-acess paper) is available at this time. This article will be updated as further information becomes available.