TASK-RELEVANT representation Learning for networked robotic perception

What is the right representation of sensory data for networked robotic perception?

Today, even the most compute-and-power constrained robots measure complex, high data-rate video and LIDAR sensory streams. Often, compute-limited robots need to transmit high-bitrate sensory data to a remote compute server if they are uncertain or cannot scalably run complex perception or mapping tasks locally. However, today's representations for sensory data are largely designed for human, not robotic, perception and thus often waste precious compute or wireless network resources in transmitting irrelevant parts of a scene that are unnecessary for a high-level robotic task. This paper presents an algorithm to automatically determine task-relevant representations of sensory data that are co-designed with the ultimate objective of a pre-trained, modular robotic perception model. Our algorithm aggressively compresses robotic sensory data by up to 11x more than competing methods and achieves high task accuracy on Mars terrain classification with low-power deep learning accelerators, neural motion planning, and environmental timeseries monitoring.