Summer for me is a time for challenge and growth. I've always sought out opportunities to expand my knowledge and skills during this time. Whether it's conducting research, interning at a tech company, or starting my own tutoring program, I find value in spending my Summers working on projects that enrich me intellectually and strengthen my skill set

SUMMER OF 2023: MIT Beaver Works Summer Institute AUV Challenge

Project Overview

Project Name: Predator vs. Prey

My Team: The Argonauts

ROVs' Names: Nemo and Dory

Objective: Use a Raspberry Pi to automate control of an ROV (Remotely Operated Vehicle) using Computer Vision and Control Theory

In the MIT Beaver Works Summer Institute (BWSI) course, my team and I were challenged to build an Autonomous Underwater Vehicle (AUV) that could autonomously navigate and interact with its environment using only Computer Vision—a unique challenge given the typical reliance on SONAR for underwater navigation. The final project, titled Predator vs. Prey, involved creating two AUVs with distinct objectives. The “Predator” was programmed to detect the “Prey” and plot a path to intercept it, while the “Prey” was designed to detect the Predator and plot a path to evade it. This task required complex programming, control theory, and strategies.

Technical Skills and Key Concepts

Week 1: Python & Physics Foundations

We began by learning the fundamentals of Python and Object-Oriented Programming (OOP), which were essential for structuring the complex control algorithms of the AUV. Using Jupyter notebooks and Matplotlib, we simulated physics equations to model the movement of the AUV. Through Python libraries like NumPy, we calculated matrix operations for mapping the forces and torques acting on our AUV.

Physics of the ROV: We analyzed Newtonian mechanics to calculate the net force and torque generated by the six thrusters on our ROV. By treating each thruster’s force as a vector and applying rotational matrices, we were able to determine the exact positioning and movement of the ROV in simulations. These foundational calculations were later integrated into the control systems to map the AUV’s paths in the real-world pool environment.

AUV_LOGO.mp4

Real-World Testing and Control Theory

Week 2:

Our first test in a real-world environment took place at the Sailing Pavilion on the Charles River, where we explored closed-loop control systems and observed how our AUV responded to commands in a natural setting.

To further refine the ROV’s movement, we designed a Dance Template where the AUV performed a series of maneuvers—Forward/Backward, Crab Left/Right, Donut, Reverse Donut, Up/Down, and Clockwise/Counterclockwise. These movements allowed us to test each thruster’s calibration and observe the effects of our control parameters. The Dance Template simulation also helped us understand the response time and drift tendencies of our AUV in a more controlled, artistic scenario.

Networking and Advanced Control Theory

Week 3:

Networking and Chatbot Development: We learned about TCP/UDP protocols and how to create host/client connections, enabling us to interface with various devices on a network. Using Python’s socket library, I built a chat server that allowed users to join, send messages, and interact—similar to a simplified version of Discord. This helped us establish communication protocols that we later used to coordinate between multiple ROVs.

Control Theory: We implemented PID (Proportional-Integral-Derivative) control to automate the AUV’s heading, depth, and lateral movements. With input from the 9 DOF IMU (Inertial Measurement Unit), we derived the robot's heading and depth. Here’s how it worked:

Depth Control: The PID system processed the error in depth (measured in meters) to calculate motor power needed to adjust the AUV’s position underwater.
Heading Control: The PID adjusted the heading of the AUV by calculating the error in radians. With this system, our AUV was able to detect its heading, adjust accordingly, and move in the intended direction. Fine-tuning the PID parameters allowed us to optimize stability and control in water.

Advanced Computer Vision

Week 4:

Lane Detection: To enable the AUV to navigate autonomously, we developed a lane detection algorithm:

We used an edge detector to highlight features in the image, then applied the Hough Transform to draw lines for these detected edges.
By analyzing gaps between lines and extrapolating lane lines, we calculated a centerline for the ROV to follow.
This centerline served as a path for the AUV to autonomously navigate down the pool, avoiding veering off-course.

April Tags for Localization: April Tags acted as fiducial markers, allowing for precise localization of the AUV relative to another ROV. By detecting the tag attached to one ROV, we were able to use pose estimation to calculate the translational position of one ROV relative to the other. This localization capability was critical in our Predator vs. Prey challenge, as it allowed one ROV to "track" the other, simulating a realistic predator-prey interaction.

Award

The Pioneer Team Award: An award presented annually to a highly distinguished individual or team, who by their achievements in the engineering and programming of the Autonomous Underwater Vehicle, has contributed uniquely to problem solving with an innovative approach and critical thinking.

SUMMER OF 2024: MIT Beaver Works Summer Institute UAV Challenge

Project Overview

Project Name: Autonomous UAV

Team Name: Nano

Objective: Build a UAV (Unmanned Aerial Vehicle) from individual components and program it to complete complex autonomous tasks using Computer Vision, Estimation and Control Theory, and Kinematics.

This second summer was a step up in both complexity and technical depth. As a software developer for Team Nano, we designed and assembled a UAV from scratch and configured its software. Our final goal was to program the UAV to execute a series of tasks and navigate obstacles, earning points for each successful maneuver.

Below a breakdown of the process and techniques we used in building this fully autonomous drone.

Hardware and Software Setup

Hardware Components:

Motors: 2216 KV920 Motors
Flight Controller: Pixhawk 6C
Sensors: Optical Flow Sensor, Stereo Camera, Teraranger, Downward Camera

These components formed the basis of our UAV, which we carefully assembled piece by piece. The Pixhawk 6C served as the primary flight controller that handles real-time data from various sensors to stabilize and control the drone.

Software Setup:

Ubuntu Linux: Our primary operating system for software development.
PX4 Development Software: Used for programming the flight controller and simulating drone behavior.
QGroundControl: A ground control station software that allowed us to monitor telemetry and set flight parameters.
ROS2 (Robot Operating System 2): Used for communication between the onboard sensors and processing modules.
Gazebo Simulator: Enabled us to test the drone’s behavior in a simulated environment, optimizing control strategies and sensor settings before real-world trials.

Basics of Kinematics and Linear Algebra

Week 1:

Understanding kinematics was essential for programming our UAV to move and respond accurately in 3D space. We used linear algebra to:

Define the drone's movement vectors.
Calculate rotations and translations for precise navigation.

By applying linear algebra to solve for spatial transformations, we ensured that the UAV could orient itself effectively and navigate autonomously within the environment.

Computer Vision and Obstacle Detection

Week 2:

We relied heavily on Computer Vision to enable object tracking, obstacle detection, and even autonomous landing. Here are some of the core techniques implemented:

OpenCV and IMU Integration: Used OpenCV with data from the Inertial Measurement Unit (IMU) to stabilize images and create a more accurate perception of the drone’s surroundings.
AR Detection: Applied augmented reality (AR) markers to help the drone recognize specific targets or landing zones and improve its precision.
ROI (Region of Interest) Filtering: By programming Regions of Interest, we could focus processing power on areas likely to contain obstacles, which optimized detection speed and accuracy.

Object Tracking and Avoidance

We programmed algorithms for object tracking to identify moving obstacles and their respective distances to the drone's camera for it to respond accordingly. By calculating predicted object paths and distance relative to the drone, the UAV could avoid potential collisions.

Trajectory Planning

After identifying obstacles, we used trajectory planning to map out alternative routes, allowing the UAV to bypass obstacles more smoothly while staying on course.

Advanced Filtering and Estimation

Week 3:

Kalman Filtering

We implemented Kalman Filtering to refine sensor data by combining noisy inputs into a stable, accurate estimate of the drone's position. This was critical for a stable flight.

Color Segmentation

We used color segmentation to identify and isolate cues in scenarios where visual cues (like colored markers) were needed. This enhanced the accuracy of the drone's object detection.

Linear Regression (Machine Learning)

To refine obstacle detection and trajectory adjustments, we used linear regression models, enabling the UAV to "learn" from previous patterns and improve its responses in real-time.

Estimation and Control Theory

Week 4:

Revisiting PID Control (Proportional-Integral-Derivative) was essential for precision in the drone’s movements. Building on what I learned from the AUV project, I applied PID feedback loops to stabilize the UAV and control aspects including altitude and heading.

Flight Modes

Manual: Controlled directly by the pilot for testing and calibration.
Takeoff/Landing: Autonomous takeoff and landing using sensor inputs.
Altitude Hold: Maintained a set altitude, allowing for stabilized hovering.
Hover: Enabled stable flight with minimal drift, important for gathering sensor data.
Offboard Mode: Allowed external control of the UAV’s path through real-time commands, enabling complex actions based on sensor feedback.

Awards

MIT Beaver Works Spirit Team Award: Awarded to our team, Nano, for fostering a positive and collaborative atmosphere throughout the program

MIT Beaver Works Overall Challenge Award: Recognized as the most technologically advanced team in the program, excelling in the real-world application of drone technology

Conclusion & Moments

Over two consecutive summers at MIT Beaver Works Summer Institute, I've delved into the intricate world of autonomous systems. Through the research and engineering of both Autonomous Underwater Vehicles and Unmanned Aerial Vehicles, I explored complex fields like control theory and computer vision. However, beyond the exhilaration and joy of navigating nerve-wracking technical challenges, I learned that the true value of engineering lies in collaboration and sharing knowledge. Working alongside like-minded scholars from across the world, I saw how diverse perspectives spark innovation. This experience challenged me to approach problems by stepping out of my comfort zone and taught me to pursue STEM with empathy and an open heart, committed not just to advancing technology but also sharing it openly.

Cogo Labs Summer of 2024

This past summer, I interned at Cogo Labs, a startup incubator in the field of data science and machine learning. I focused on developing AI-powered fitness model named "Mr.Peak" using OpenAI's API. Mr.Peak was designed to assess user-uploaded physique photos and assign them a heathly score from 1-100, and provide workout plans and meal suggestions based on the analysis. This project taught me the skill for calling and securing API keys while offering insights into the importance of training data for a reliable AI model.

Page updated

Google Sites

Report abuse