Back in 2016, Microsoft released a video showcasing their research on Holoportation, a technology that allows people to talk to each other via the projection of virtual selves through the HoloLens augmented reality glasses. In it, the demonstration shows how two people can talk to each other from two separate rooms yet interact as though they are in the same space. The technology is capable of showing other objects inside the room and projecting those things inside the HoloLens as well. It also allows for the recording and playback of the 3D virtual space, which allows collaborators to sit back into an earlier discussion.
The demonstration is incredibly impressive, considering that this technology requires decades of improvements for every component involved. There is no way that real time projection of someone else’s body could be this well done without the power of modern processing, nor could these computers scan and create a 3D model interpretation of your body without the help of high quality cameras and tracking software.
As humans, we want the technology we use to interact with one another to feel as realist as it can. A lot of the disadvantages of email or instant messaging is that we are limited by the characters and the structure of the language that we use in text-based communication. Having hundreds of emojis on our smartphones is a wonderful addition to long distance communication because they allow us to share emotions that can’t be clearly expressed strictly with text. The next closest thing to face-to-face communication would be phone calls. They allow us to express language as well as laughter and vocal intonation which can express our emotions much clearer than just instant messaging. Videocalls add in a real time steam of a video of your face! They have the benefits of language, vocal intonation, and add in a person’s face. Plenty of people in long distance relationships with family, spouses, and/or significant others love to talk this way. However, this is currently as close to face-to-face interaction in the virtual world as we can get.
While there are several games and software that owners of virtual reality headsets use to interact with a virtual avatar of some other real person out there, it’s not as personal. In fact, having a virtual avatar to conceal your identity makes VRChat and other similar games a lot of fun. Having a real time representation of yourself in VRChat would probably ruin the fun in some way, by making it too personal. However, if you asked me if I preferred a real time projection of my family or some silly avatars, I would clearly prefer to “see” my real family in the virtual space.
In Holoportation, Microsoft offers another dimension of realism to long distance communication. However, this realism requires this technology to record, send, and project back accurate representations of body language in real time! It’s an incredible feat of technology, and the projections seem to work very well inside of the HoloLens.
However, there are a few downsides to consider with this kind of technology. My first immediate thought was that the original HoloLens doesn’t work particularly well. It does what it promised to do, but the quality of the projection wasn’t amazing. The hand tracking done by the HoloLens didn’t work as easily as I had hoped either. However, I have no doubt that Microsoft is working on much better iterations of their original HoloLens which will be capable of rendering higher quality AR projections. In this case, seeing a projection of your family or a significant other in the virtual space through the HoloLens would probably be incredible to experience.
My other concern is that of privacy. People already screenshot texts and save photos to send to friends or post on social media, so people recordings these interactions could mean that someone could record and save something you said indefinitely. You could have these recordings come back to haunt you, much like old phone calls and texts resurface years later when a politician is running for office or an executive at a company is being targeted. This just gives another layer of risk when you represent yourself in a virtual space. On top of that, the technology that scans your face and body to create a virtual representation of yourself just shows that governments everywhere could implement very accurate representations of your identity. Video demonstrations have shown that cameras can recognize you not just by your face, but by your heart rate through infrared (heat) cameras or by the way you walk.