As part of the demo prepared for the Consumer Electronics Show (CES), 2023, we aimed to utilize high-resolution, vision-based GelSight tactile sensors for the classification of three rubber samples with varying hardness (Shore 00-40, Shore 00-65, Shore 00-85). My responsibility included molding the rubber samples, hardware setup, including the robot arm and tactile sensors, as well as establishing the data collection pipeline. Additionally, I developed and trained a Recurrent Nerual Network (RNN) to successfully perform the classification task.

This project focused on tracking the 6D pose of an object in contact with GelSight tactile sensors mounted on a robot gripper (end-effector/hand). By tracking the movement of the 3D geometry of the object in contact with the sensor, we could determine the position and orientation (pose) of the object relative to the sensor surface, which was crucial for in-hand manipulation, industrial insertion, and assembly tasks. The 3D geometry of the object in contact with the tactile sensor was registered using a standard point cloud registration algorithm, utilizing a known object model point cloud. This allowed us to accurately determine the 6D pose of the object with respect to the world frame. The above method was successfully implemented at a client site for an industrial insertion task, where we provided the code and technical support. This practical application demonstrated the effectiveness and robustness of the pose estimation technique in a real-world scenario, showcasing its potential for various industrial automation tasks.

The task of identifying objects buried in granular media using tactile sensors poses significant challenges. Firstly, particle jamming within the granular media hinders downward movement. Additionally, the presence of granular media particles between the sensing surface and the object of interest often distorts the object's actual shape. To address these challenges, we introduce the Digger Finger, an early prototype designed to overcome these limitations. The Digger Finger incorporates mechanical vibrations to fluidize the granular media during penetration, enabling smoother movement. Furthermore, it is equipped with high-resolution vision-based tactile sensing capabilities, allowing for the identification of buried objects within the granular media. The utilization of such robotic fingers holds great potential for applications such as explosive ordnance disposal and Improvised Explosive Device (IED) detection, offering a finer resolution compared to conventional techniques like Ground Penetration Radars (GPRs).

PDF | VIDEO | WEBPAGE

Designed and implemented a tele-operating system for precise manipulation of a four-fingered Allegro robot hand. The system incorporates a glove equipped with flex sensors on the finger joints, which exhibit changes in resistance when bent. Kinematic re-targeting i.e. mapping the flex sensor readings to control the corresponding movements of the robot hand was achieved by using the Sequential Least-Squares Quadratic Programming (SLSQP) algorithm,  inspired from the work titled DexPilot. To enhance the system's performance, magnetic sensors were later utilized, offering improved accuracy and response compared to the bend sensors previously employed.

In this project, we undertook modifications to the initial design of the PCF sensor to seamlessly integrate it into an upper limb prosthesis device, specifically the Bebionic Hand. Our enhanced sensor design incorporates an infrared proximity sensor chip and a barometric pressure sensor chip, both carefully embedded within an elastomer layer. My involvement in this endeavor encompassed various responsibilities, including sensor prototyping, electronic testing and debugging, mechanical testing, and fitting. Additionally, I implemented a signal fusion algorithm to combine data from both sensor chips, enabling measurements of proximity within the range of 0-10mm, contact force at 0N, and force localization at five spatial locations and three angles of incidence (0-50N). Notably, our work has garnered attention in the media, with news articles featuring our advancements published in prestigious outlets such as CU News and CBS4.

PDF

Our study demonstrates the significant enhancement in the reliability of basic manipulation tasks through the integration of low-cost in-hand proximity and dynamic tactile sensing techniques. Inspired by the mechanoreceptors in human skin, we employed an array of infrared proximity sensors embedded in a transparent elastic polymer, along with an accelerometer located in the robot's wrist. By breaking down the manipulation task into eight distinct phases, we showcased the efficacy of our approach. Firstly, we illustrated how proximity information derived from our sensors can greatly improve the reliability of picking and placing objects. Additionally, we highlighted the utilization of dynamic tactile information to discern different phases of grasping. Our experimental findings, conducted with a Baxter robot engaged in a tower construction task, validate the effectiveness of our proposed methodology. The results reinforce the potential of incorporating proximity and dynamic tactile sensing for achieving more robust and accurate manipulation capabilities.

PDF

In this project, we outline our endeavors to leverage machine learning methods for the classification of hand gestures. To capture these gestures, we utilize a wristband equipped with an array of 16 embedded capacitive sensors. Throughout our study, we explore diverse classifiers and employ different data processing approaches. Our data collection involves gathering samples from 14 subjects performing five distinct hand gestures. Subsequently, we conduct experiments using various classifiers to evaluate the classification performance. Our findings indicate that it is indeed feasible to classify hand gestures using data obtained from our capacitive sensing wristband. However, we acknowledge that further refinement is necessary to optimize this method for real-time gesture recognition applications. This work sheds light on the potential of machine learning techniques and capacitive sensing in accurately discerning hand gestures. Nonetheless, additional research and development efforts are essential to enhance the effectiveness and practicality of this approach in real-world scenarios.

PDF

This project assesses various algorithms for data set registration on the SQUID database, which was developed by the University of Surrey. The evaluation is performed using MATLAB and focuses on three algorithms: the Iterative Closest Point (ICP) algorithm, the Levenberg-Marquardt ICP (LMICP) algorithm, and the Coherent Point Drift (CPD) algorithm. The comparison between these algorithms primarily revolves around the distribution of resulting registration configurations. Given the limited research in this field, the project explores and evaluates the methodology required for conducting such a comparison. To facilitate dense sampling, an experiment suite is designed. Upon analyzing the experimental results, it becomes evident that the CPD algorithm surpasses the other two algorithms in terms of cluster size and the percentage of successful registrations. However, due to computational and time limitations, the 3D experiments were not conducted in depth, highlighting the need for further exploration in this area. In conclusion, this project sheds light on the performance of different data set registration algorithms using the SQUID database. The findings highlight the superiority of the CPD algorithm in certain aspects, while also acknowledging the necessity for additional investigations to delve deeper into 3D experiments.

PDF

As an undergraduate student, I had the opportunity to represent my college, the National Institute of Technology (NIT) Surat, in the Robocon Asia Pacific Robotics Competition in 2012 and 2013. As part of the team, I played a key role in designing and fabricating a group of robots that were tasked with completing a complex set of objectives. My contributions primarily focused on mechatronic and electronic design, as well as prototyping and fabrication.