Abstract: We investigate the use of a graph processor for scene flow computation and propose that certain properties of this processor are highly suitable for this application. Specifically we use the GraphCore massively parallel IPU chip, which consists of 1216 independent cores called tiles, each with 256kB local memory. These tiles are connected to an ultra fast all-to-all communication fabric enabling efficient message-passing algorithms for sparse graphs.

We propose a distributed formulation for dense RGB-D scene flow based on Gaussian Belief Propagation which leverages the architecture and programming model of this processor to produce motion estimates suitable for rigid, non-rigid or piece-wise rigidly moving scenes. This is enabled by local as well as global regularisation of the flow field. Local regularisation is enforced for pairs of flow estimates whose corresponding pixels are neighbours, while global regularisation is defined for a sparse set of flow estimate pairs whose corresponding pixels are far from each other on the image plane. We demonstrate that using both types of regularisation results in flow fields with varying rigidity which makes this algorithm suitable for scenes with different amount of motion.