IEEE International Conference on Robotics and Automation (ICRA) 2021

Xi'an China

On- and Near-sensor Vision Processing,
from Photons to Applications (ONSVP)

A full day virtual workshop, June 4, 2021

Objectives

Almost all real-world AI and robotics applications are power-constrained and depend on real-time understanding of the robot's environment. Robotic systems that rely on simultaneous localization and mapping (SLAM) and other perception algorithms, would greatly benefit from novel emerging computing paradigms such as computation on or near the focal plane. In this workshop, we explore the emerging unconventional perception methods such as computation directly within the light sensor, performing early-stage recognition before converting signals to digital form, and implementing all or parts of the deep neural networks on the chip. In these methods, only where features of value are identified is data actually transferred to later stages of the vision pipeline. We aim to bring researchers from academia and industry from across multiple disciplines, including i) hardware design, ii) compiler, iii) software/algorithm design, and iv) holistic design to demonstrate the power of novel computational approaches using these unconventional sensors. These novel approaches enable a variety of recognition and tracking applications with exceptionally low latency, high frame rate and low power.

Several live demos using SCAMP5-d Focal-plane Sensor-processor Arrays (FPSP) will be presented. These demos are:

  • BIT-VO: running high-speed visual odometry on SCAMP5-d FPSP

  • Neural Sensor: HDR imaging using SCAMP5-d FPSP

  • CAIN: demonstrating a new pipeline for compiling CNN kernels to SCAMP5-d FPSP code.

  • AnalogNet: a robotic navigation demo using a SCAMP5-d FPSP, capable of navigation in indoor environments avoiding all obstacles, at high frame rate and low power

  • Focal Plane Inference: a demonstration of CNN inference performed on the focal plane of a SCAMP5-d for digit recognition.

BIT-VO

Neural Sensor

CAIN

AnalogNet

Topics of Interest


Topics of interest include, but not limited to:

  • Focal-plane Sensor-processor Arrays

  • Computational Sensors

  • Smart Vision Sensors

  • Event Cameras

  • Novel Depth Ranging Sensors

  • Compiler Methods for Emerging Sensors

  • Application in Robotics and AI

  • AI on the Focal Plane

  • SLAM on Chip/Focal Plane

  • Visual/inertial Odometry on Chip/Focal Plane

  • Metrics, Benchmarking, and Datasets for the Emerging Sensors

  • Performance Analysis of Algorithms and System using Unconventional Sensors

Invited Speakers

Andrew J. Davison

Imperial College London

Boris Murmann

Stanford University

Davide Scaramuzza

University of Zurich

Masatoshi Ishikawa

University of Tokyo

Piotr Dudek

University of Manchester

Raluca Scona

Imperial College London

Robert Mahony

Australian National University

Walterio W. Mayol-Cuevas

University of Bristol

Yu-Hsin Chen

Facebook

Program

ONSVP

Talks and paper presentations are live. Each talk is 20 minutes plus 10 minutes for Q&A, right after the talk. Paper presentations are 7 minutes plus 8 minutes for Q&A, right after the presentation.

You may add the event to your calendar via this link:

https://calendar.google.com/calendar/u/0?cid=aWNyYS53b3Jrc2hvcEBnbWFpbC5jb20

Papers and Posters

Event Camera

  • PDF, Video, "Event-based Asynchronous Sparse Convolutional Networks," Nico Messikommer, Daniel Gehrig, Antonio Loquercio and Davide Scaramuzza

  • PDF, PPT, "Classification of motion directions with spiking neurons: a continual learning case," Veïs Oudjail and Jean Martinet

  • PDF, Video, "DSEC: A Stereo Event Camera Dataset for Driving Scenarios," Mathias Gehrig, Willem Aarents, Daniel Gehrig and Davide Scaramuzza


Focal-plane Sensor-processor Arrays (FPSP), Pixel Processor Array (PPA)

  • PDF, Poster, Video1, Video2, "Direct Servo Control from In-Sensor CNN Inference with A Pixel Processor Array," Yanan Liu, Jianing Chen, Laurie Bose, Piotr Dudek and Walterio Mayol-Cuevas

  • PDF, Poster, "Super-pixel Compilation for Pixel Processor Arrays," Edward Stow and Paul Kelly

  • PDF, Poster, Video, "Bringing A Robot Simulator to the SCAMP Vision System," Yanan Liu, Jianing Chen, Laurie Bose, Piotr Dudek and Walterio Mayol-Cuevas


Hardware

  • PDF, Poster, "Towards Dynamic Fault Tolerance for Hardware-Implemented Artificial Neural Networks: A Deep Learning Approach," Daniel Gregorek, Nils Hülsmeier and Steffen Paul


Best Presentation and Best Poster/Video

Best Presentation:

  • PDF, Video, "Event-based Asynchronous Sparse Convolutional Networks," Nico Messikommer, Daniel Gehrig, Antonio Loquercio and Davide Scaramuzza


Best Poster/Video:

  • PDF, Poster, Video, "Bringing A Robot Simulator to the SCAMP Vision System," Yanan Liu, Jianing Chen, Laurie Bose, Piotr Dudek and Walterio Mayol-Cuevas

Call for Contributions

  • The workshop accepts contributions of research papers describing early research on emerging topics.

  • The workshop is intended for quick publication of work-in-progress, early results, etc. The workshop is not intended to prevent later publication of extended papers.

  • Prizes will be given to the best presentation and also to the best poster/video.

  • Submission Format: please use standard IEEE format (minimum two pages, and up to eight pages). For more information, see the IEEE website https://journals.ieeeauthorcenter.ieee.org

  • CFP Link: https://easychair.org/cfp/ONSVP-2021

  • Submission Link: https://easychair.org/conferences/?conf=onsvp2021

Important Dates

Full paper submission due date: April 15, 2021 April 30, 2021

Paper acceptance notification: May 18, 2021

Camera-ready version due date: May 28, 2021

Final Submission Link: https://easychair.org/conferences/?conf=onsvp2021

Workshop day: June 4th, 2021 (See the schedule above for the start time)

Youtube Recordings (talks and paper presentation):

https://www.youtube.com/playlist?list=PLLQ5-0tmXjiJRtVHNcZ9quwhUDwV2L9Jh

Gather Town Link: (For posters and demos): https://gather.town/i/PoM0B4Uz

Technical Program Committee

Paul Kelly (Imperial College London, UK)

Riku Murai (Imperial College London, UK)

Yu-Hsin Chen (Facebook, USA)

Raluca Scona (Imperial College London, UK)

Laurie Bose (University of Bristol, UK)

Thomas Debrunner (iniVation AG, Switzerland)

Edward Stow (Imperial College London, UK)


Program Committee Chairs:

Julien Martel (Stanford University, USA)

Sajad Saeedi (Ryerson University, Canada)



Organizers

Sajad Saeedi (Ryerson University, Canada)

Julien Martel (Stanford University, USA)

Riku Murai (Imperial College London, UK)

Paul Kelly (Imperial College London, UK)

Sponsors

The Workshop is supported by the following IEEE Robotics and Automation Society's Technical Committee on:


Computer & Robot Vision (TCVision)


Neuro-robotics Systems (NRS)

Contact us

If you have any questions, please contact organizers at icra.workshop@gmail.com