Everything you need to know about mixed reality

Virtual reality (VR) and augmented reality (AR) are well known to the general public, even if the terms are not entirely understood. Credit hardware like Samsung GearVR and Microsoft’s Hololens and games like Resident Evil 7 and Pokémon Go for bringing what were once science fiction concepts to the masses. Now, with the emergence of mixed reality (MR) technology, a third class of “reality” is threatening to break into the mainstream.

What Is Mixed Reality?

What is mixed reality? In their paper, “A Taxonomy of Mixed Reality Visual Displays,” authors Paul Milgram and Fumio Kishino define MR as: “a particular subclass of VR-related technologies that involve the merging of real and virtual worlds, which we refer to generically as mixed reality. … The most straightforward way to view a mixed reality environment, therefore, is one in which real world and virtual world objects are presented together within a single display, i.e., anywhere between the extrema of the virtuality continuum.”

In other words, MR is the blending of the real-world environment with digital elements through a piece of technology like a head-mounted display (HMD). Though Mixed Reality is a fairly simple concept to wrap your brain around, there aren’t that many examples of MR applications to date. The HoloLens is still a developer-focused release, and though Microsoft has made some showcase apps available, there’s a lack of polished, consumer-ready software. So far, secretive startup Magic Leap has produced some of the most famous MR videos, though later examinations of the company’s output reveal they’re more concept art than product demos.

So how does mixed reality relate to and differ from virtual and augmented reality?

Mixed Reality vs. Augmented Reality

The term “augmented reality” was first coined in 1990 at Boeing by researchers Thomas Caudell and David Mizell, who were tasked with coming up with a device that could replace the expensive diagrams used to guide factory workers. According to Boeing, “They proposed replacing the large plywood boards, which contained individually designed wiring instructions for each plane, with a head-mounted apparatus that would display a plane’s specific schematics through high-tech eyewear and project those schematics onto multipurpose, reusable boards.”

Of course, the technology was not yet in place to realize Caudell’s and Mizell’s vision, but a device like today’s Hololens is in many ways the realization of their early-90s dreams.

With AR technology hitting the consumer space, it begs the question: What’s the difference between AR and MR anyway? Is there a real distinction, or are these two terms for essentially interchangeable? Interestingly, we can find the answer by asking the question, “What’s the difference between a Dalmatian and a dog?”

Obviously, there is no functional difference—a dog is a Dalmatian and a Dalmatian is a dog—but there are also many other differences. “Dog” is a broader term used to describe a species or the genotype of an organism. “Dalmatian” is used to describe a certain collection of characteristics within the species or the phenotype of a particular creature. This distinction parallels the one between mixed reality and augmented reality. MR describes the general technique of mixing sensory information from a virtual and real environment, and AR refers to a specific type of sensory mixing.

“Mixed Reality vs. Virtual Reality”

The differences between virtual reality and mixed reality are a little more apparent. First and foremost, VR is an immersive, closed experience in which the head-covering HMD fully envelops the operator. There isn’t much “mixing” going on in today’s VR content—you strap on a headset and disappear into a fully digital world. This is one of the big complaints people make about VR: It’s an isolating experience, with the person off by themselves, their face covered in technology.

Mixed reality, on the other hand, wants to seamlessly blend the real world with the virtual world, with the digital characters becoming indistinguishable from their living, breathing counterparts. An MR experience allows people in different locations to interact as though they were all in the same room. Each participant experiences the conversation in their actual surroundings, which are populated by a mix of real and digital people and things. We consider this communal aspect of the MR experience to be one of it’s big advantages over VR.

Reality Is What You Make of It

Perhaps the best way to understand the difference between VR, AR, and MR comes from Recode, which cited MR’s flexibility as its distinguishing feature. “[MR] tries to combine the best aspects of both VR and AR, while wrapped up in a marketable term that sounds marginally less geeky than its cousins. In theory, mixed reality lets the user see the real world (like AR) while also seeing believable, virtual objects (like VR). Then it anchors those virtual objects to a point in real space, making it possible to treat them as ‘real,’ at least from the perspective of the person who can see the MR experience.”

Looking ahead, the next wave of HMD hardware will most likely include multiple mixed reality glasses beyond just Microsoft’s Hololens. As every business from Hollywood movie studios to retail giants look for ways to leverage the technology, there will soon be all manner of new implementations hitting the market. It’s no wonder there’s buzz that MR will be the most important tech developed in 2017.

Glossary of Terms

Augmented Reality (AR): Viewed on a display, AR is an enhanced image or environment that’s produced by overlaying computer-generated images, sounds, or other data on a real-world environment.

Dual Reality (DR): DR is the concept of maintaining a virtual and a real world, that reflect, influence, and merge into each other by using deeply embedded sensor/actuator networks. The components of the real and virtual worlds are independent entities but become enriched through their mutual interaction.

Head-Mounted Display (HMD): An HMD is a headset (goggles or a full helmet) used with virtual and augmented reality systems. Many HMDs include a head tracker that enables the system to respond to head movements.

Hololens: Microsoft HoloLens is the first self-contained, holographic computer, enabling you to engage with your digital content and interact with holograms in the world around you.

Mixed Reality (MR): Mixed reality blends real and virtual worlds to produce new environments and visualizations in which physical and digital objects co-exist and interact in real time.

Pokémon Go: Launched in 2016 by The Pokémon Company, Niantic and Nintendo, Pokémon Go is smartphone game credited with bringing the idea of AR into the mainstream.

Virtual Reality (VR): VR is a computer-driven, artificial environment which you experience through sensory stimuli (as sights and sounds) and in which you partially determine what happens.

Explore the Future of Virtual Reality, Augmented Reality and Mixed Reality at AT&T SHAPE

SHAPE is an immersive event that explores the convergence of technology and entertainment. Experience interactive demos that give you a glimpse into the future and explore the magic of virtual reality, augmented reality, and mixed reality. See how entertainment tech will revolutionize content creation. You’ll be among the first to witness firsthand some of the latest technology trends poised to go mainstream. This is where you’ll find real inspiration.

SHAPE is happening July 14 and 15, 2017 in Los Angeles, California, at Warner Bros. Studios.