Machine Learning Scientist December 2022 - Present
Text to speech for Customer Support Chatbot in multiple Indic languages.
Analyst Co-Pilot Tool (Text2SQL). Developed RAG approaches with Vector Embeddings of SQL Table Information using open-source models (StarCoder, LLaMa) and paid models (GPT4, GPT3.5).
AI based Avatar Generation for the Rush Gaming Universe (RGU). Developed models for prompt to mesh and image to mesh.
Research Associate (Computer Vision) July 2022 - December 2022
Advisor - Prof. Yogesh Singh Rawat
Video Geolocalization - Working on developing novel algorithms for real-time video geolocalization using deep learning.
Benchmarked current state-of-the art methods (conventional CNN models and Transformer based) for the same.
Machine Learning Engineer Jan 2022 - December 2022
Image Editing - Working on real-time user personalised image editing using deep learning.
Image Aesthetics Assessment - Developed a deep learning model for image aesthetics assessment using self-supervised learning. Self supervision enabled learning aesthetic attribute values with only image select or reject as labels.
Eye blink detection - Devised a real-time eye blink detection model with only a face image as input. The algorithm works at 20 frames per second on an i7-8th Generation CPU.
Image Blur Classification - Worked on image blur classification and devised a non-learning approach for the same. The method works for all blur types (motion blur, out of focus).
Research Assistant (SLAM) July 2021 - Jan 2022
Advisor - Prof. Saket Anand
Lidar Odometry - Developed a lidar odometry pipeline using VLP-16 Lidar for vehicle localization on a pre-built map. Added local bundle adjustment and loop closure to the pipeline for reducing drift. Used GPS for initializing the vehicle position on the map.
Lidar Camera Calibration - Worked on marker based calibration of sensor suite consisting of multiple lidars and cameras.
Research Intern (Perception and Motion Planning) Aug 2020 - March 2021
Advisor - Prof. K Madhava Krishna
Quadrotor Navigation Under Uncertainty - Worked on autonomous navigation of quad-rotors in outdoor environments under depth sensor uncertainty and working in a team of three, devised a novel algorithm (CCO-Voxel) for the same. The algorithm uses probabilistic collision checking and gradient free trajectory optimization.
Mapping and Path Planning for Drones - Developed navigation pipeline consisting of mapping (octomap with local euclidean distance map), kinodynamic A* planning, BSpline trajectory optimization and control using PX4 Firmware for drones.
Implemented the algorithm on Gazebo simulator with PX4-SITL and also on hardware.
Research Intern (Computer Vision and Machine Learning)
Advisor - Prof. Matias Valdenegro Toro Aug 2020 - July 2021
Marine debris segmentation using SONAR - Created dataset of 1868 images using a Forward Looking Sonar (FLS) for marine debris segmentation. Total of 11 classes were created for objects like can, bottle, hook, etc.
Benchmarked standard semantic segmentation models like DeepLabV3, UNet, PSPNet and LinkNet with 12 different encoders. The resulting paper was published in Ocean Vision Workshop, ICCV 2021 (paper).
BreatheDigital B.V., Netherlands
Research Intern (Motion Planning and Computer Vision) April 2019 - Aug 2020
Obstacle Avoidance for drones - Developed a real-time local occupancy mapping and path planning algorithm based on Vector Field Histogram for food delivery drones that worked at 15 Hz on a Jetson Nano.
Control - Controlled the drone using MAVSDK Python API with PX4 Firmware.
Multi-stage EKF State Estimation - Implemented a multi-stage Extended Kalman Filter (EKF) for fusing 3 GNSS sensors and 1 IMU (6 axis).
Presented our approach of event-enabled autonomy for food delivery drones at PX4 Dev Summit 2020.
Research Intern (SLAM and State Estimation) June 2018 - Aug 2018
Camera Calibration - Worked on monocular and stereo camera calibration. Developed a ROS package for camera intrinsic and extrinsic calibration
SLAM - Benchmarked open-source algorithms for Simultaneous Localization and Mapping (SLAM) like ORB SLAM, LSD-SLAM, Direct Sparse Odometry (DSO) etc. on our dataset and compared the performances based on root mean square error (rmse) and computation time.
Digital Signal Processing Lab, NSIT
Student Researcher (Computer Vision) Dec 2017 - May 2020
Advisor - Prof. Tarun Kumar Rawat
Road Segmentation-In a team of 4, developed a novel road segmentation algorithm by fusing the output of V-disparity ground segmentation with that of a semantic segmentation model. Fusion was done by using Conditional Random Fields (CRF) (paper, thesis).
Optimization based Image Thresholding - Worked in a team of 3 students on multi-level image thresholding. Used Vortex Search Optimization technique for optimizing Otsu inter-class variance cost function (paper).
Image Enhancement - Worked in a team of 6 students to develop an image enhancement algorithm for underwater and sandstorm images. Enhancement was done in 3 steps - Color correction, histogram stretching and unsharp masking (paper).
ARES Robotics, NSIT
Software Team Lead Dec 2017 - April 2020
Autonomy Stack Layout - Designed the autonomy layout of the ARES Rover for University Rover Challenge (2019, 2020) consisting of rover localization, Mapping and Path Planning.
Rover Localization - Developed an Extended Kalman Filter (EKF) based rover localization module for position estimation by fusing RGBD odometry output, IMU and GPS.
Local Mapping - Worked on developing local occupancy grid and euclidean distance map generation for motion planning.
Motors and Sensors - Worked on interfacing motor drivers and other sensors with microcontrollers for rover control.
NSIT Solar Car Concept
Electrical Team Member Dec 2016 - April 2017
Vehicle Circuit Design - Designed the overall circuit for Phoenix -1 solar car on Eagle CAD. Also designed and fabricated circuits for various on-board electronic modules like telemetry system.
Battery Management System - Worked on 18650 Li-Ion cells and worked with the team to assemble battery for the vehicle. Also worked on interfacing battery management system.
