Debumping in 6DoF Point Cloud

Usually, the 6 DoF images contain a variety of bumps like the left images in the following video. MSP members have tried to remove those bump errors whereas other data such as edges and features are preserved. The following video shows that the proposed algorithm is effective in removing the bum errors.

Viewport Extraction from VR Image

Omnidirectional visual content has attracted the attention of customers in a variety of applications because it provides viewers with a realistic experience. Among the techniques that are used to construct omnidirectional media, a viewport rendering technique is one of the most important modules. That is because users can perceptually evaluate the quality of the omnidirectional service based on the quality of the viewport picture. To increase the perceptual quality of an omnidirectional service, we propose an efficient algorithm to render the viewport. In this study, we analyzed the distortions in the viewport picture to be the result of sampling. This is because the image data for the unit sphere in the omnidirectional visual system are non-uniformly sampled to provide the viewport picture in the conventional algorithms. We propose an advanced algorithm to construct the viewport picture, in which the viewport plane is constructed using a curved surface, whereas conventional methods use flat square surfaces. The curved surface makes the sampling interval more equally spaced than conventional techniques. 

The following pictures display the viewport images that are rendered by the various projection methods. The rightmost pictures are results produced by the proposed algorithm. The subjective qualities of the proposed method are much higher than those of the others. This is because the objects are deformed and the lines are bent in the pictures that are produced by the conventional method. 

Hole Concealment in VR Image

We applied the proposed algorithm for the test sets of Brick Road, Alleyway, Tree, Bench, and Street. In the following figure, VR images including the holes are compared with the concealed images, respectively. As shown in this figure, we can see that the proposed algorithm conceals the various holes effectively in the test images.

When we construct a 6 DoF data from multiple pictures, the quality of the depth image is one of the most important components to increase the quality of the constructed data. Usually, the depth image may contain a variety of holes. So, those holes degrade the constructed 6 DoF data. In order to solve this problem, we invented an algorithm to fill the holes in the depth image. The following video shows the effect of the proposed algorithm.

[left] 6DoF image is constructed from depth image including several holes  /  [right] 6DoF image is constructed from depth image after hole concealment