Holoportation by Microsoft is an end to end system for telepresence using Augmented and Virtual Reality. Holoportation allows users to see, listen and interact with 3D objects like people, objects, furniture that are reconstructed using a 3D depth camera. Coupled with HoloLens which is a head mounted device for AR gives users the feeling that they are in the same physical space as the person or an object that they are interacting with.
Telepresence is the most advanced form of video conferencing and the idea behind holoportation is to bring this technology to the masses. According to this article, 1.3 trillion dollars was spent on business travel back in 2017 and this figure is expected to touch 1.7 trillion dollars in 2022. Using the latest video conferencing and telepresence technology we can bring this number down drastically.
Prior to Holoportation there have been extensive work on telepresence. One of the most popular among them was the Kinect again by Microsoft. While Kinect did an excellent job of capturing a person’s movement and translate it to a 3D avatar in a game, it did not have the capability to reconstruct the exact 3D model of the user. There have been other consumer driven telepresence products which focuses on reconstructing the human body. Holoportation takes it a step further by allowing objects like furniture, fruits etc to be reconstructed in the other end.
Here is a video demonstration of the product in action
Holoportation setup consists of 8 camera pods. Each pod has a combination of Near Infra red cameras, Diffractive optical element and a laser. The depth camera allows it to perform foreground segmentation. Spatial audio is recorder and the HMD provide users the position of the audio source directed towards him/her.
With such advances in audio and video streaming Holoportation has the upper hand when compared to traditional video conferencing products like Skype, Cicso Telepresence etc. As seen in the video the user has the ability to move around the space and benefit from the physical cues. The user at the other end also has a similar physical set up and experience the same physical cues. One of the major advantages is that the user is not only able to manage eye contact but also study the body language of the person in the other end. Interpreting body language is one of the essential skills that humans develop to understand what the other is feeling or thinking. This can be crucial in a business environment when the presenter wants to gauge the mood of the room. When a human is augmented in the physical space, we get a sense of personal space around them. This also helps us elevate the conversation by keeping the space in mind. Apart from business meetings, other application around communication and collaboration comes in the form of family gathering whose members are 1000 miles away celebrating an occasion together. Classroom experience where a remote teacher is trying to explain a complicated Mechanical module using a 3D object. Doctors performing initial diagnosis on certain ailments of patients
While Hololens provides a seamless AR experience it can also be extended for VR applications like social gathering, games etc. Overall Holoportation has set a new bench mark in the telepresence market and it is exciting to see what the future holds in this space.
Orts, Sergio & Rhemann, Christoph & Fanello, Sean & Kim, David & Kowdle, Adarsh & Chang, Wayne & Degtyarev, Yury & Davidson, Philip & Khamis, Sameh & Dou, Minsong & Tankovich, Vladimir & Loop, Charles & Cai, Qin & Chou, Philip & Mennicken, Sarah & Valentin, Julien & Kohli, Pushmeet & Pradeep, Vivek & Wang, Shenlong & Izadi, Shahram. (2016). Holoportation: Virtual 3D Teleportation in Real-time. 10.1145/2984511.2984517.