The Laws of Mixed Reality — without the rose-colored glasses

June 3, 2016

(credit: Matrix Revolutions/Warner Bros.)

By John Rousseau

The future of human consciousness will be a hybrid affair. We will live and work in a ubiquitous computing environment, where physical reality and a pervasive digital layer mix seamlessly according to the logic of software and the richness of highly contextual data. This is mixed reality (MR) — and it will soon simply be reality: projected onto our mind’s eye, always on, always connected, and deeply personalized.

It will be delivered first through a head-mounted display, and ultimately embedded in our perception via subtler inputs. The resulting human network will be a massive, dynamic system capable of generating enormous value for humankind, as we embrace our cyborg future and the superpowers it enables.

We’re not there yet, though this vision is far from science fiction. Today’s MR/VR products are somewhat clunky in terms of hardware, awkward in terms of user experience, and constrained by practical performance issues and limited content. However, these are temporary limitations that will be solved in time — and probably more quickly than we expect.

“Light” — a conceptual virtual-reality device for 2020 that facilitates shared experiences. Future virtual-reality devices will be untethered, smaller, and lighter, and thus more mobile, further “tuning out” physical reality with more lifelike 3D video and audio, and stimulating other senses beyond sight and sound while allowing for “inclusion” of others. (credit: Artefact)

Future hardware will be capable of rendering high-resolution digital content that blends seamlessly with our environment, and devices will be small enough to wear all the time. The complex UX challenges will be resolved, and new interaction models will emerge along with a new computing paradigm. And eventually, the infrastructure, bandwidth, content, and connectivity will be in place to support a fully integrated experience.

Big data, high fidelity
What could possibly go wrong?

According to the World Economic Forum, these innovations and the exponential pace of change represent the emergence of a fourth industrial revolution, which will be characterized by massive disruptions in business and society, along with unprecedented global challenges like income inequality, the automation of labor, and permanent unemployment, to name a few.

Clearly, technology is not a neutral force, and if history is a guide, it will continue to shape both social and individual identity and outcomes for both better and worse. This time, the difference is the speed of change, the potential network effects of new technologies, and our capacity to adapt in time.

Mixed reality is a core component of this perfect storm — because it represents the high-fidelity interface between humans and the emerging network. Innovations in big data, artificial intelligence, and sensing will give rise to increasingly sophisticated digital products that will have the capacity to define a new digital reality based on extraordinary levels of context, insight and data. This network intelligence will include multiple signals pertaining to individuals, communities and populations, and will result in an unprecedented level of detailed knowledge about human behavior—down to the specific humans in question.

Given current business models — and the economy of attention in which they operate — mixed reality could be the gateway for a dystopian feedback loop of epic proportions, with the potential for large-scale social and personal manipulation. Imagine grocery shopping. Or watching television. Or social media.

Looking ahead, it’s not the robots we should be afraid of, but the capitalists who will use these systems to monetize our consciousness, attention, time, and behavior. And when that day arrives, will we even notice? Or will the transition be so natural that Facebook (or its successor) is simply the default interface of our lived experience?

Thus, with great data comes great responsibility. As future business models are determined, who gets to decide? Who will you trust?

The Laws of Mixed Reality

In 1942, Isaac Asimov coined his “Laws of Robotics” in an effort to understand and define the ethical boundaries of artificial intelligence. While fictional, these laws continue to inform our thinking about the social impact of new technologies. In a similar spirit, we are calling for “Laws of Mixed Reality” that help us shape the discourse and future development of MR with an emphasis on preferable outcomes. The Laws are aligned to three significant problem areas, covering the individual, society and economics.

The First Law: the Individual
Mixed reality must enhance our capacity for mindful attention.

There is a clear risk that the types of behaviors we notice now — being “in your phone” rather than in the world — will be exacerbated by mixed reality. Future digital experiences need to counter this tendency and improve our ability to be present for ourselves and others, and help users balance complex streams of information, notifications, and stimuli within a holistic experience.

There is an opportunity to counter the addictive aspects of technology (FOMO — fear of missing out) that impair our ability to focus, lead to increasingly short attention spans, and lessen human productivity and potential.

The Second Law: Society
Mixed reality must embody a shared human experience.

Empathy is not a given in this new world. Paradoxically, as MR/VR allow us to experience diverse worlds and contexts beyond our own, we may become isolated in terms of local experience and confined to an algorithmic filter bubble and feedback loop of our own making. To counter this, we will need to create new models and affordances designed to facilitate discovery, sharing and human connection — both virtually and in the real world.

The Third Law: Economics
Mixed reality must respect boundaries between commerce and data.

Opaque business models that provide services in exchange for personal data will not be sustainable in mixed reality. As the data stream becomes richer — in terms of understanding both context and human behavior — the data will be both more valuable and easily manipulated to serve other interests. New business models will be required that respect personal privacy, and data ethics will necessitate enhanced regulation and oversight.

Shaping a preferable future

The Laws provide a foundation for identifying preferable outcomes that should be considered in the creation of new products, services and business models. As we engage the complexity of the next industrial revolution, outcomes-focused thinking and design will be critical to optimize systems while minimizing negative externalities. The stakes are simply too high to think otherwise, or to pursue technological innovation at the sake of people and society. While not an exhaustive list, following are five critical outcomes that comprise a preferable future for mixed reality.

  1. Presence: New conventions that disrupt/replace the “feed” modality and help individuals optimize attention, connection, and effectiveness in daily life.
  1. Empathy: Increased understanding of others — both in my immediate environment and more broadly — and the emergence of a social etiquette for mixed-reality experiences.
  1. Diversity: Disrupting the filter bubble, contributing to a breadth of experience and a nuanced understanding of multiple points of view.
  1. Transparency: Transparent business models and transactions, based on the fair exchange of value and grounded in trust.
  1. Agency: The user can exercise control over critical aspects of the experience, including the level of immersion, integration with other systems, etc.

We are what we think

“The really important kind of freedom involves attention and awareness and discipline… That is real freedom. That is being educated, and understanding how to think. The alternative is unconsciousness, the default setting, the rat race, the constant gnawing sense of having had, and lost, some infinite thing.” — David Foster Wallace

In 2009, writer David Foster Wallace addressed the graduating class of Kenyon College and told them, in effect, that the most valuable outcome of their education was the freedom to think on their own terms. He was suggesting a fundamental kind of human agency — the freedom to define our own mental lives, which shape our lived experience, and in turn shape individual identity, and thus society.

I think Wallace would be worried about mixed reality. It represents an evolution of human consciousness that is possibly more profound and impactful than we acknowledge, particularly when viewed in the context of rapid technological innovation, exponential growth, and nature of the larger economic system. Because the technology is in its formative state, now is the time to discuss its ethical implications, adopt an outcomes-centric approach to innovation, and define the path forward.

If we do that, hopefully we can cultivate those infinite things that are most important to us, while creating a future that is as magical as it portends to be.

John Rousseau is Executive Director at Artefact, a design and innovation consultancy in Seattle. Twitter: @john_rousseau