Anhur Car Object Tracker

2 . PIXY2 Cam GUI: The main objective of our project was to build an object tracking car that would identify an object and follow it. Therefore, a camera was one of the main sensors in our project to acquire data from the tracked object. The camera is used to get data of the center-coordinate of the tracked object and to send it to micropython where we coded our motor-controller and ultrasonic sensor. The code for the motor control in micropython then uses this data to control the motors to adjust the angle of the car’s velocity, dynamically controlling the car’s heading. This data is then be displayed as read serial in the front panel. Moreover, command A prints this data just like read serial but just for one sample and VISA resource name tells us which COM port is being used. used.                                                                  

The camera uses pixels to get data of the image it is capturing, so we decided to divide the total pixels in the x coordinate (horizontal) of the camera into half so we can know the center-coordinate data. This helps us find if the tracked object is right or left of center of the camera. Since the camera will be in front of our car and in the middle, the center x axis pixel of the camera will define the center of the car as well. In micropython, we receive the data of the tracked object, including magnitude and direction of the object center point, and use that to control two of the motors that are moving the car. The ultrasonic sensor in our car also play a part in controlling the motor. For any distance less than 30, the ultrasonic sensor stop both the motors regardless of the direction of the object. This way the car will be maintaining a safe distance from any object on its way. 

Motor Control:

The overall design for our motor control was based on input from other sensors. The original design and our future design took input from the PIXY2 camera in the form of a value. This value was calculated as the difference between the object's center, and the center of the video frame. This delta signified the amount, in pixels, that the object was either to the left or the right of the current trajectory. After running tests to correlate this value with an increase/decrease in motor power, we then plot the power needed to change direction with respect to the change in pixels. This calibration could be avoided with the implementation of PID controller.

The code as shown below ran during our demonstration that used sensor data from one of the ultrasonic sensors mounted to the front of the vehicle. This sensor measured the distance between itself and the object in front and used this value to determine whether the motors should be on or off. If the vehicle came within 30 cm of the front object, the vehicle stops until this was no longer the case. This scheme would ensure that the vehicle would keep a distance of 30 cm at all times. This is another perfect candidate for PID control implementation implementation.                                                                     

Team Reflections & Future Changes:

Our team faced many challenges in this project, especially when trying to integrate the three components (camera, ultrasonic sensor, and motor control) into one working system. One of the biggest challenges we faced in our project was making our camera communicate with our microcontroller. The main objective of the project was to use a camera as one of the sensors. This is for data acquisition of an object so it can send data to our microcontroller. We tried researching different kinds of cameras that would work best for our microcontroller. Our plan was to use the camera attached to the car to locate an object and calculate the center coordinate of the object. We would then use this data to actuate the motors proportional to the amount of change needed to lock onto the center of the object again. We initially started with an ESP32-CAM as the camera for object detection and tracking which we thought would work well with the ESP32 microcontroller. However, the ESP32-CAM turned out to be very slow for our purposes as it was sending the coordinate data very slowly with lots of lag. The lagging data would fail the whole objective of real time object tracking because our car needs to acquire data as fast as possible to follow the object. Due to these attributes of the ESP32-CAM, we decided at the last minute to switch to a different camera for object tracking. After more research, we decided to use the PIXY2 camera which assured a much faster rate of image processing (61fps) for tracking an object. We used an Arduino Nano for data acquisition of the camera during testing and wanted to send that data to micropython which was the main language for our project 

Overall, given more time and upon our corrected understanding of the ping pong buffer implementation, we would have liked to improve our real time implementation in data collection via the ultrasonic sensor. We would have the “toggle_led” function take nine measurements, each at a certain time interval, and concatenate them to an array to be sent to the LabView. This way the distance measurements would be more accurate since it would be one distance measurement taken at every interval (created by the timer interrupt) instead of nine rapidly measured data points in succession and then a gap before another set of measurements are taken.