Improving the experience of the audience with digital instruments

June 2, 2015

Virtual content being displayed on stage and overlapping the instruments and the performers (credit: Florent Berthaut)

University of Bristol researchers have developed a new augmented-reality display that allows audiences to better appreciate digital musical performances

The research team from the University’s Bristol Interaction and Graphics (BIG) has been investigating how to improve the audiences experience during performances with digital musical instruments, which are played by manipulating buttons, mic, and various other controls.

Funded by a Marie Curie grant, the IXMI project, led by Florent Berthaut, aims to show the mechanisms of digital instruments, using 3D virtual content and mixed-reality displays.

Their first creation Reflets is a mixed-reality environment that allows for displaying virtual content anywhere on stage, even overlapping the instruments or the performers. It does not require the audience to wear glasses or to use their smartphones to see the augmentations, which remain consistent at all positions in the audience.

Reflets relies on combining the audience and stage spaces using reflective transparent surfaces and having the audience and performers reveal the virtual content by intersecting it with their bodies or physical props.

The research is being presented at the 15th International Conference on New Interfaces for Musical Expression (NIME) in the U.S. [May 31 -- June 3].

BristolIG | Ixmi: Improving the experience of the audience with digital instruments