Microsoft Research has developed a new system called HoloDesk that allows users to pick up, move, and even shoot virtual 3D objects.

An overhead screen projects a 2D image through a half-silvered beam splitter into a viewing area beneath. A Kinect camera keeps tabs on a user’s hand position, a webcam tracks the user’s face to help with placement accuracy, and custom algorithms bring everything together in real time.

Holographic objects can be picked up and stacked on top of real-world ones, and real hands can juggle virtual balls or shoot them at targets, or play with a non-existent smartphone. Users can remotely collaborate on shared multiuser virtual projects.