Solving Multi-camera interference in ToF 3D Cameras using Medium Access Protocol

Combatting MCI using Carrier Sensing

Divyanshu Saxena

Abishek Kumar

Department of Computer Sciences

University of Wisconsin - Madison

Mission of the project

3D imaging is being used by vision and robotic systems to recover 3D scene geometry, which then use these images to support technologies such as autonomous transportation, augmented reality, and robot navigation. Time-of-flight (ToF) 3D cameras are bringing about a revolution in this space, because of their low-cost, compact form factors, and low computational complexity. They are now becoming more and more ubiquitous - from vehicular LiDARs to consumer grade devices like iPad, premium phones, AR/VR headsets, etc. These cameras work on the principle of emitting modulated light, and then, inferring the reflected light from the scene to capture the depth of the objects and image them.

However, they encounter a major hurdle - when multiple ToF cameras try to picture the same scene, they shall receive the light from each other as well, and hence, they suffer from huge depth errors. In this project, we want to understand the interference of light signals and accurately recover the 3D scene information. More specifically, we draw parallels between the image capture process of cameras, and the communication of wireless RF systems. Interference is a well studied problem in the wireless space, and we claim that we can leverage techniques from the wireless space, for countering the multi-camera interference problem.

Motivation - Active 3D Cameras and Interference

Due to their low cost, compact form factor and low computational complexity; ToF based active 3D are now becoming more and more ubiquitous - even making their way into consumer grade devices like mobile phones, tablets, AR/VR headsets, etc. These cameras consist of a programmable light source that emits modulated (temporally coded) light, which travels to the scene of interest and gets reflected. This reflected light is received by the sensor (typically, co-located with the transmitter) and the scene depth is captured by comparing the reflected light with the emitted light. These cameras suffer from major interference, however, in the presence of multiple 3D ToF cameras. The following figure demonstrates the error in recovered depths when multiple ToF cameras image a particular scene. The red light is the received reflection intensity, while the true depth can be recovered when the camera recovers the depth from the true reflected light (in green).

This interference is even more pronounced for the more compact and recent solid state 3D ToF cameras, used in consumer grade devices; as these cameras use flood illumination for capturing a scene rather than a scanning beam - which causes strong interference and hence, large systematic errors in capturing the depths. We aim to address this problem by using a medium access protocol, based on dividing the time for image capture into multiple slots, and activating the different cameras in different time slots. More nuances regarding this approach are discussed in the following sections.

Parallels with the Wireless MAC Protocols

Interference is a widely studies problem for RF-based wireless communication systems, and these techniques exist in the form of Medium Access Control (MAC) layer protocols. Many of the successful interference miti- gation protocols in the communication systems are based on distributed medium access by all the actors in the system. The data packets are sent from the transmitters to receivers in a fashion such that collisions are mini- mized. One of the most basic approaches in this direction is that of ALOHA. However, there are numerous improvements available over it, in the form of CSMA (Carrier Sense Multiple Access) protocols. Particularly, we are interested in implementing and adapting the CSMA-CA (Carrier Sense Multiple Access with Collision Avoidance) protocol, which is also used in the IEEE 802.11 standard. Read more about the prior works here and the proposed technique here.