KEF Robotics Internship

Tasks

• Design an interactive visualization method that could be used for manual calibration of cameras and LiDaR

  • Designed methods of converting stored data to ROS2-compatible data and publishing it including camera info so that Rviz2 could be used to visualize the overlay

  • Designed a user interface that made it convenient to adjust transformation matrices for visual calibration

• Optimize static data collection using the sentinel interface

  • Changed the current code to use numpy rather than nested lists, which improved efficiency

  • Changed the current code to use a binary encoding rather than text-based

  • Went from 10-15 second runtime to almost instantaneous

• Design an automatic LiDaR-Camera calibration pipeline

  • Overcame several difficulties regarding calibrating LWIR and EO cameras

  • Used Intensity-based Lidar Camera Calibration (ILCC) to accurately calibrate EO cameras to LiDaR, then used Kalibr toolbox to calibrate EO cameras to LWIR cameras

  • Responsible for converting ILCC to a Python3 compatible version, and fixed several bugs in corner detection and data collection

• Run pretrained and conventional stereo methods on datasets to establish a baseline for performance

  • Ran SGM and pretrained RAFT-Stereo methods on several collected datasets

  • Developed utilities to convert outputted disparity images into human-readable formats and pointclouds

  • Developed utilities to pack output to rosbag and play back for analysis

• Develop methods for assisting training of a custom stereo method

  • Developed methods for converting pointclouds into disparity images for ground truth

    • Methods did not use extensive external library functions for easier debugging

  • Developed visualization methods for pointclouds and disparity images