Mixed reality is the future of computing?

Alex Kipman knows what hardware is. By joining Microsoft 16 years ago, he was the chief inventor of over a hundred patents, including innovative motion detection technology XboxKinect, which paved the way for some of the functions in his latest creation – a holographic 3D headset called HoloLens.

But today, sitting in his office at Microsoft headquarters in Redmond, Washington, Kipman does not talk about hardware. He discusses the relationship between people and machines from a broader philosophical point of view. Regardless of whether we interact with machines through screens or things that sit on our heads, for him, all this is just a “point of time”.

Born in Brazil, Kipman, who is a technical specialist in Microsoft’s WindowsandDevicesGroup, enthusiastically explains that the key advantage of the technology is its ability to replace time and space. He cites the example of “mixed reality” (MR, mixedreality), the term Microsoft, which denotes a mixture of the real world with computer generated graphics. According to Kipman, one day it will invisibly unite the augmented and virtual reality. He says that among the most striking peculiarity of MR is the potential to unleash “super forces of substitution” in the real world.

People attach special value to the feeling that you experience when you physically share space with another person. That’s why Alice Bonacio of FastCompany decided to interview Kipman face to face. “But if you had the opportunity to carry out this kind of interaction without spending time to move,” says Kipman, “life would be much more interesting.” Further from the first person.

“My daughter can communicate with her brothers in Brazil every weekend, and my employees do not need to travel the world to do their job,” he continues. “With the advent of artificial intelligence, we could continue the conversation, but I already would not be here. Once you, and I will talk, you will be on Mars, and I will be a hundred years dead. Our work of technicians is to accelerate the future and constantly ask ourselves how to do it. ”

Microsoft is betting on a mixed reality to help us get into the future. And here again we return to the hardware. The availability of the desired device at the right price will be a factor that will determine whether consumers will accept a mixed reality (although some devices themselves are unlikely to start the MR revolution, as the VR example showed). Although HoloLens is the only stand-alone holographic computer on the market (unlike OculusRift or HTCVive, if you do not need to attach cables to an external device), smart points worth $ 3,000 serve more proof of the concept than a consumer product.

Now Microsoft wants to change this. This fall, the company launches the Windows Mixed Reality Headsets, which will be the first serious attempt to sell the concept to the general public. Although this device is still closer to the ideal hybrid of augmented and virtual reality, it already embodies the main HoloLens chips – like advanced tracking and mapping capabilities – and offers a more affordable price of $ 300-500. The headset will be available in various forms from various hardware partners, including Dell, HP and Samsung, and will allow users to create three-dimensional spaces that can be personalized through media, applications, browser windows and others.

According to Microsoft, the introduction of a platform that allows any person as a whole to create their own digital world is the first step in achieving the same leap into the world of tomorrow. “If you believe, like us, that mixed reality is the inevitable next trend in computing, you will have to connect productivity, creativity, education and a whole new range of entertainment, from casual games to hardcore games,” says Kipman.

Improving the mixed reality

Kipman is not the only one who is optimistic about mixed reality. California startup Avegant is working on a platform that presents detailed three-dimensional images, layering many focal planes that the company calls “light field technology.” “Applications are endless,” says Avegant CEO Jörg Tewis. “From designers and engineers who directly manipulate 3D models with their hands, to professors of medicine, illustrating various heart diseases by the example of an almost live model to their students. At home, users can surround virtual shelves with their favorite products. Mixed reality allows people to interact directly with their ideas instead of screens and keyboards. ”

To do this, mixed-reality devices must support virtual images that will appear indistinguishable from the real world and interact seamlessly with it. According to Professor Gregory Welch, a computer scientist at the University of Central Florida, most of the technologies developed to date have not yet reached such an equilibrium. “Mixed reality is especially difficult, because there is no concealing imperfection of the virtual, nor the amazing purity of the real.”

Together with colleagues, he found that in some cases the relatively wide field of view of the real world that HoloLens provides can harm an important sense of presence. While a healthy person sees 210 degrees, the HoloLens display increases the center of your field of view by 30 degrees or so. In the experiments conducted by Welch and his team, the gap between the real and extended landscapes reduced the feeling of immersion and presence.

“This means that if you look at a virtual person in front of you (as in our experiment), you will see only a part of it floating in the space in front of you,” says Welch. “You will need to move your head up and down to” draw “the perception of it, since you can not see the whole person right away, if you do not look at it from afar (it will seem less). The problem is that your brain constantly sees an “ordinary” world around itself, and it “overwrites” many types of perception that you might otherwise have. ”

Further, Welch explains that in the demonstrations that we see today with HoloLens or Apple ARKit, for example, virtual objects can be fixed on a flat surface, but in addition to the basic form and visual appearance, the software usually does not recognize many important physical characteristics of the object, such as weight, center of mass and behavior, or the surface on which it is located – not to mention any activity in the real world that occurs around the objects.

“If I accidentally drop a couple of cubes from a virtual table, they will not” fall “, reaching the edges, and do not bounce, as one would expect, based on their type and floor material,” he explains.

In a paper that Welch co-wrote with Professor Jeremy Balinson, director of the Virtual Human Interaction Lab (VHIL) at Stanford University, they outlined some of their research findings that show that virtual content has a much higher value when it demonstrates the behavior that we expect from physical objects in the real world.

“In our laboratory, we start using HoloLens to understand the relationship between the experience of augmented reality and the subsequent psychological attitude to the physical space itself,” says Baylinson. For example, he explains that his experiments show that virtual people who “go like ghosts” through real objects, rather than bypassing them or trying to avoid them, are perceived as less “real” than those who obey the laws of physics.

Advancements in mixed reality are likely to lead to the fact that headsets will become more accessible and easier, but it is also possible that at least some of our future interactions with this technology will not include wearable electronics at all. “Spatial Augmented Reality” (SAR), for example, developed by Welchom many years ago, allows you to use projectors to change the appearance of physical objects around you, such as the material of the table or the color of the couch – without glasses.

“Of course, the SAR will not work in all situations, but when it does, it will be convincing and easy,” says Welch. “If there’s something magical about when the world around you is changing, and you do not have anything to do with it – neither the headset, nor the phone, nothing. You simply exist in the physical world, which is changing almost around you. ”

A virtual tool for working together in the real world

Nonnie de la Peña, founder and CEO of Emblematic, helped put the use of virtual reality as a means of accountability and for storytelling. She is referred to as the “godmother of virtual reality”, and she believes that the immersion technologies are the closest to the audience – that is, put it in the place of the narrator. She believes that HoloLens has the potential to increase the quality and depth of our understanding of the world, in part thanks to the method of volumetric capture, which creates a three-dimensional model of objects due to the multitude of cameras and a green screen. “Microsoft began offering high-level realism, using a massive capture, and it immediately picked up the journalists,” says de la Pena. Own creation of Emblematic, After Solitary, is an award-winning documentary,

The most important change that the mixed reality promises is that the content will not be tied to any particular device. MR uses building blocks (real-world objects, or created by a computer) to create environments that include people for further interaction. In this context, devices become a window that allows you to look into and access to these worlds, rather than a repository that stores your personal content (like your smartphone).

Kipman notes that in these shared real / virtual environments, our relationship with computing varies from personal to collaborative – from devices that retain your own individual content to shared spaces for creativity, mediated by technology. Kipman thinks that this has profound implications for how we will develop applications in the future. If, for example, you create a virtual statue and place it as a hologram on top of a table in your living room, another person with a mixed reality device will see your statue when he enters the room and move it if he wants. Because the content is stored not in your device, but in the environment itself, defining objects (both real and virtual) that inhabit it.

“These concepts require a rethinking of the operating system in the context of a mixed reality,” says Kipman. “You have to build a foundation that goes from silicon to cloud architecture to realize your own transition from personal computing to collaborative. It will take time, “he smiles.

Leave a Reply

Subscribe To Our Newsletter

Join our mailing list to receive the latest news and updates from our team.

You have Successfully Subscribed!