Overcoming the limits of touch screens, Carnegie Mellon University and Disney Research researchers have developed an inexpensive alternative: a hardware toolbox of physical knobs, sliders, and other mechanisms that can be readily added to any device.

The researchers drew inspiration from wind instruments in devising these mechanisms, which they call “Acoustruments.” The idea is to use simple 3-D-printable plug-in plastic tubes and other structures to connect the smartphone’s speaker with its microphone. The device can then be controlled by acoustically altering sounds as they pass through this system.

Just as a simple slide whistle or flute can produce expressive music, these Acoustruments can add a wide range of functionality to a smartphone, including proximity and pressure sensors. And because no electrical circuitry is involved, the plastic devices can be made rapidly and inexpensively.

“We’re providing people with tangible interactivity at basically no cost,” said Gierad Laput, a Ph.D. student in Carnegie Mellon’s Human-Computer Interaction Institute (HCII). Laput is the lead author of a study presented April 22 at CHI 2015, the Conference on Human Factors in Computing Systems, in Seoul, Korea, named a Best Paper by the conference organizers.

“Using smartphones as computers to control toys, appliances and robots already is a growing trend, particularly in the maker community,” Laput added. “Acoustruments can make the interactivity of these new ‘pluggable’ applications even richer.”

People who want to control a smartphone application while their eyes need to be focused elsewhere (on the road while driving, for instance) might be less distracted by a tangible knob or button made possible with Acoustruments than with trying to glance at a touchscreen. The new technology also enables users to make adjustments without disrupting a virtual reality experience.

The CMU and Disney researchers have used Acoustruments to build an interactive doll that responds when its tummy is poked, a smartphone case that can sense when it has been placed on a table or is being hand carried, and an alarm clock that provides physical on/off and snooze buttons.

Acoustruments can work like wind instruments, which take a sustained source of sound from a mouthpiece and then alter it by changing the cavity — with a series of holes, as in a flute, or by changing its size, as with a trombone. The new devices operate in a similar manner: the smartphone speaker produces continuous “sweeps” of ultrasonic frequencies; interactions that block, open holes or change the length or diameter of the plastic tubes connecting the speaker to the microphone alter this acoustic signal.

Experiments by the CMU and Disney researchers showed that Acoustruments can achieve 99 percent accuracy in controlling the device.

Acoustruments can be made with 3-D printers, with injection molds, or even by hand in some cases, Laput said. The ultrasonic frequencies are inaudible to people and the pluggable structures are designed to block out interference from external noise.

The research was sponsored by Disney Research.

DisneyResearchHub | Acoustruments: Passive, Acoustically-Driven, Interactive Controls for Handheld Devices

---

Abstract of Acoustruments: Passive, acoustically-driven, interactive controls for handheld devices

We introduce Acoustruments: low-cost, passive, and powerless mechanisms, made from plastic, that can bring rich, tangible functionality to handheld devices. Through a structured exploration, we identified an expansive vocabulary of design primitives, providing building blocks for the construction of tangible interfaces utilizing smartphones’ existing audio functionality. By combining design primitives, familiar physical mechanisms can all be constructed from passive elements. On top of these, we can create end-user applications with rich, tangible interactive functionalities. Our experiments show that Acoustruments can achieve 99% accuracy with minimal training, is robust to noise, and can be rapidly prototyped. Acoustruments adds a new method to the toolbox HCI practitioners and researchers can draw upon, while introducing a cheap and passive method for adding interactive controls to consumer products.