Stacks on the Surface

March 19, 2010

SURFACE STACKS R&D TEAM | Stacks on the Surface: Resolving Physical Order Using Fiducial Markers With Structured Transparency

Considerable work has explored tangible interactions on surface-based computing. Often, this entails the direct placement of physical objects – either abstract or iconic – onto interactive surfaces. Although static themselves, their physicality offers an accurate and intuitive means for instantiating, manipulating, or otherwise acting on dynamic elements within the interface.

Stacking, an action we perform regularly to organize physical objects in the real world, is notably absent from contemporary surface interactions. Stacking provides an intuitive way to group items and describe ordering. In this paper, we introduce a novel method that allows stacks of objects resting on camera-driven surfaces to be resolved both which objects are in the stack, and in what order they are placed.

This is achieved using conventional, passive fiducial markers, which in addition to reflective regions, also incorporate structured areas of transparency. This allows particular orderings to appear as unique marker patterns. We discuss how such markers are encoded and fabricated, and include relevant mathematics. To further motivate our approach, we comment on various scenarios where stacking could be especially useful. We conclude with details from our proof-of-concept implementation, built on Microsoft Surface.