Pattern-recognition system for mobile devices blocks ‘shoulder surfers’

April 9, 2014

Georgia Tech’s LatentGesture system creates a “touch signature” on mobile devices to block “shoulder surfing” password-hacking (credit: iStock)

Cybersecurity researchers from the Georgia Institute of Technology have developed a new security-software system called LatentGesture that continuously monitors how a user taps and swipes a mobile device. If the movements don’t match the owner’s patterns, the system recognizes the differences and if programmed, can lock the device.

In a recent Georgia Tech lab study, the system was nearly 98 percent accurate on a smartphone and 97 percent on tablets. (The system can be combined with other security systems for higher accuracy.)

“The system learns a person’s ‘touch signature,’ then constantly compares it to how the current user is interacting with the device,” said Polo Chau, a Georgia Tech College of Computing assistant professor who led the study.

To test the system, Chau and his team set up an electronic form with a list of tasks for 20 participants. They were asked to tap buttons, check boxes and swipe slider bars on a phone and tablet to fill out the form. The system tracked their tendencies and created a profile for each person.


LatentGesture continuously monitors how a user taps and swipes a mobile device. If the movements don’t match the owner’s tendencies, the system recognizes the differences and can be programmed to lock the device. (Credit: Georgia Institute of Technology)

After profiles were stored, the researchers designated one person’s signature as the “owner” of the device and repeated the tests. LatentGesture successfully matched the owner and flagged everyone else as unauthorized users.

“Just like your fingerprint, everyone is unique when they use a touchscreen,” said Chau. “Some people slide the bar with one quick swipe. Others gradually move it across the screen. Everyone taps the screen with different pressures while checking boxes.”

The research team also programmed the system to store five touch signatures on the same device — one “owner” and four authorized users. When someone other than the owner used the tablet, the system identified each with 98 percent accuracy.

As security consultant Frédéric Fraces notes on Information Security Buzz, there is a danger of other people “shoulder surfing,” or looking at a mobile phone or tablet in an attempt to steal a password. The researchers also say the system is constantly running in the background, so the user doesn’t have to do anything different for added security and authentication.

The study will be presented in Toronto at ACM Chinese CHI 2014 from April 26 to 27.

This research was partially supported by the National Science Foundation.


  • Premkumar Saravana, Samuel Clarke, Duen Horng Chau, Hongyuan Zha, LatentGesture: Active User Authentication Through Background Touch Analysis, presented at The Second International Symposium of Chinese CHI (Chinese CHI 2014), Toronto, Canada, 2014