Chronological Entries - 2020

YEAR SUMMARY

I would call this year my "First Steps". I didn't know much about much and I was just trying to learn the very basics. I could only learn off tutorials online and on YouTube. My CAD designs where a little chaotic and my software only worked good enough for what I needed and was hard to debug. One thing I wished I would have tried is prototyping. I seemed to change my design just because I felt it was better and not really making any prototypes of it.

By the end of the year I feel that I took my knowledge from only things I learned In school, to a much intricate understanding of CAD and computer vision systems. I actually started to feel a little bewildered because there was just so much to learn after opening the doors to these things. One thing that did change was my passion, I started to envelop myself in this project and other STEM hobbies like it after teaching myself what's out there.

2/9/20 - First Designs!

This original design used sheets of aluminum, high powered servos and a rail system where two wheels pinched some sort of rail. This design was the first sketch of what the final arm will look like (this design changes quickly after a little more thinking).

2/15/20 - Some more designs on paper!

This is the more revised arm. Using high speed, high torque NEMA 23 stepper motors. The motors are placed on the opposite side of the arm to be used as a counter balance so the arm can maintain high speed movements. Using steppers can allows us to have much more torque will still keeping a somewhat small form factor. I will have to learn how to control steppers in the code to make sure I'm using the full potential of the steppers. The rail system will need more designing as of today, but for now I think I might change it to a strong lead screw system because of the high weight of the arm.

2/30/20 - Started some CAD to model one of my designs

After sketching out the idea of the arm, I went to CAD to get a more accurate representation of the arm. I used the CAD models of parts that I would be able to buy online. I decided that the there will be a system of pulleys and be belt driven. The bottom arm will use two stepper to double the output torque to move the mass much quicker. This arm has a max length reach of about 2 feet and max height of almost 3 feet! I also need to design how the paddle will mount and how the fifth or maybe sixth axis will be mounted as well. The bottom half of the robot still needs to be designed, and more thinking needs to go into what system I will be using to linearly move the arm on the track.

4/1/20 - Started working on some Software

So after days of smacking my head on my keyboard, I finally got a working tracker that tracks object based on color. Now this tracker may be changed in the future for a more precise tracker that might track based on parse through the frames and seeing what has moved based on looking at the previous frame. But for now, this tracker works surprising well when the ping-pong ball is a different color then the object and background around it. The program spits out coordinates of where it thinks the ball is (top left is origin). All-in-all, the tracker works really well for the mean time and spits out an accurate reading of where the ball is located. But, I can see the tracking of background objects that have a similar color to be trouble some if I don't precisely mount the camera.

4/3/20 - More Software testing

In this simple project I used a linear slider, stepper, and a belt and pulley system to move a slider black to hit the ping pong ball. I designed and 3D printed a simple sled to follow the position of the ball. No predictions based are being made, I just used the current X coordinate of the ball and mapped that to the stepper to move the slider block. It was a test to see if the tracking system was working properly. I'd say it worked partially as intended (more on that farther down).

4/4/20 - Arduino Stepper Mapper w/ Homing Sequence

This arduino code is an interrupter to parse through the coordinates map the coordinates to the slider, then tell the stepper how far to move the block on the slider block. It also has a homing sequence to determine where zero is.

IMG_0041.MOV

4/6/20 - Homing and Tracking!

In this test, the slider is homed, then moves to the middle of the slider shaft until I adjust the threshold to track the white in the image. Notes:

1) The slider jitters around quite violently which needs to be adjusted. Maybe after I create a prediction algorithm it will reduce or even eliminate that and just move to the predicted position.

2) Belts seem to work well, but maybe design some deceleration curve for starting and stopping.

3) The tracker to pick up just the ball and not the objects around it. Maybe this problem will be eliminated when I create the mount on the ping pong table, or maybe I need to find a new motion capture software or system entirely.

ezgif.com-gif-maker.mov

4/7/20 - More Tracking Testing

1) I changed position of the camera and also changed the lighting in the room to further enhance the color of the ball against color of the table. When this gets moved to the real ping-pong table, I will have a solid mount for the camera to reduce problems with line the camera's coordinates with the movements of the slider. I will also have to make sure the lighting is close to ideal to get optimal readings of the ball. I will also want to have a correctly colored ping pong table to so it doesn't interfere with the ball color.

2) I also changed the color of the ball and adjusted the threshold to accompany it so the ball doesn't interfere with outside environment. Changing the ball to a red ball allowed me to get a more precise reading and also eliminate excess noise.

4/15/20 - Started Building Large Slider

Received the slider
80-20 for camera mount
The ball screw and support

4/16/20 - Realized problem with Lead Screw

I ordered the lead screw to play around with after seeing slack in the belt-driven system. However, the lead screw had nowhere near the speed that I wanted to be able to reach with the budget available . While it has very little backlash and provides structure, I will have to swap back to belts.

4/17/20 - Changed the linear motion and built a temporary rig

I switched the linear motion method from ball screw to belt. With belt drive, we can achieve much greater speed at a lower cost and while still retaining accuracy, so long as the belts are tight. I will have to by thicker belts and pulleys for the final assembly to increase rigidity of the belt. I will also need to find a easy a way to mount the belt to the carriage. At first I used tape which obviously did not work well at all, but then drilled and tapped some hole for a bolt to hold down the belt which seems to work perfectly.

Also Installed the limit switch for homing.

IMG_0132.MOV

4/18/20 - Homing

Here's a clip of the belt system with acceleration and homing at 75% speed. The rattling is due to the cheap linear rails.

final_5e9e58a36415980015802c7f_712157 (1).mov

4/6/20 - Camera Mount

Here I made the camera mount and attached the camera with the slider and mapped the slider to the table size and added a dead band stop the violent jittering of the carriage.

The camera has trouble picking up the ball's color, on the table, probably due to the lighting or the tracking system entirely. I will experiment with different lighting sources and different ball colors or maybe change the tracking system.

IMG_0152.MOV

4/23/20 - Sequential Imaging

After playing around with color tracking, changing the camera, and lighting sources, I just couldn't get it to work reliably.

I decided to play around with a new tracking system, Sequential Imaging. This tracking system works by comparing the previous frame to the current frame, so directly tracking motion.

1) This type of tracking does not use color to track the object. This is great because we can use what ever ball we want and the tracker will also work accurately in most lighting conditions we want.

2) It also records and tracks at high frame-rates other then the low fps steam to be able to accurately pinpoint the position moving objects

3) This tracking system works extremely well even in high speed tracking. Which is why this is a great contestant for the final project.

4/23/20 - Camera Equipment

The camera I will be using is a high-speed USB camera that is capable of of shooting 640p @ 240fps. The trade from quality to frame rate is beneficial so we can update the frame images quicker. The quality is not a requirement since we will just be doing basic blob detection and taking the average between the frames. But, if needed this camera can also shoot at HD @ 120fps.

4/25/20 - Arm Concept Redesign!

Today I decided make things cooler, while using the slider I made as a testing rig. I thought that I would change the design so I have a larger stationary arm mounted behind the table rather than have it slider on a rail. I decided to go with this mainly because I wanted to work with a large arm, and not have to worry about keep the entire assembly light so it could be moved on a slider. In my eyes at the moment, this is a simpler solution in concept. I know that I probably could have got it working with the slider, I just felt that I would have more fun building a large arm. I will use the current slider for a testing rig for my code that I will make in the future (coming soon).

4/28/20 - Cad Tweaking

I lengthened the size of the arm reach to about 3 feet in radius around the base. I will have to do some research if the NEMA 23 steppers should have enough torque to hold the payload weight of paddle while maintaining quick speeds. This model is unfinished as of now, I still need to complete the the hip axis, and the twist axis and mount for the paddle on the end of the arm. I plan to have the arm would be mounted behind the table, flush with the top with the table with a stand that reaches the floor. I must keep in mind weight while designing this arm.

4/29/20 - Cad Tweaking

Here is the "almost" completed arm with two high-powered servos on the end for the additional axis. The mount for the paddle can be improved, it is just bolted on to the end servo as of now. The stand has some work to be done on it but the size is correct to hold the arm flush with the table. The wiring paths will also have to decided on in CAD before construction of the arm as well.

5/1/20 - Possible New Motor

After learning what Torque is online, I reached out to someone who could help me decide if my stepper would supply enough torque. Since Torque is Force * Radius, and I have a relatively large radius, the angular toque needed exceeds the max torque of the steppers I have selected. With my current setup the motors would be liable to stall or break. So I decided to look into Brushless Driven Actuators! I choose this for the large torque output and also it just seems super cool.

Looking around, I found many brushless motors with encoders and gearboxes. This allows for quick actuation with lots of torque because of the gear boxes. The RMD-X8 Pro actuator is has a brushless motor inside with a large gear reduction, a driver built in, and encoder built into a "pancake" type casing. I wont be using this because of the $450 price tag. However, I will use it and many other ones like it to model my new motors for the arm. I plan on designing and 3D printing my own for affordability. I can also use something called the ODrive control the position of the arm extremely accurately using encoders.

5/8/20 - Updating Arm Design

After pulling the CAD models for a variety of different actuators, I concluded that the torque output, along with the simplification of the system as a whole makes the system much more likely to succeed. I decided that if I was to use these actuators I can simplify the design immensely saving on weight, time to build, and cost, hopefully increasing the agility and precision of the arm. I will not be using these exact motors, so I will be evolving the CAD model as I do more research about what motor/gearbox combo will be best for this application.

video0.mov

5/15/20 - Simple Prediction Algorithm

I decided to to a break with CAD as I do more research about what I will be using for motors.

After spending with some time playing with the new high fps camera. I began to looking into simple prediction methods to predict where the ball might end up so the robot has time to move the paddle.

The method I used is a weighted averaging method. The algorithm averages most of the previous points together while giving the most recent points the highest weight in the overall average. This is a simple method for predicting the end points of the ball which seems to be working really well. If need be, we can use a more sophisticated algorithm in the future.

final_5ec1bd482d4b460015b69340_390619.mp4

5/17/20 - Height Tracking

This was a test of how the code works while streaming the side view of the ball. One thing notice is how little of the table I can fit in the frame without moving the camera like 20 feet away.

This is a problem since I want to keep the system as simple and compact as possible. I can use a fisheye lens, however the distortion it creates is a large problem for my targeting software. I know there is a way to undistort the video feed in OpenCV (the computer vision library) but I will have to learn how it works first. I will be looking into different methods to capture the entire table in 3D space.

I also noticed a lot of false triggers in the tracking software. I will have to learn how to add certain criteria to which "things" we can track so we can filter out the other non-important moving objects in the frame at a later date.

5/20/20 - Software Testing Rig

As I decided to change up my arm design, I wanted to use the slider I have as a testing rig for me to fid issue in my code. This is what I will be adding in the future on top of my current slider. This will be mainly using to test my targeting software so I don't have to worry about the kinematics of the arm.

However with this setup, there will be no serve back, this is just for positioning the paddle and getting a sense for how long we actually have to move the paddle in time.

5/22/20 - Camera Calibration and Distortion Correction

I am extremally interested on how we can alter a video feed, so I decided to research how the OpenCV library does this.

After spending a lot of time learning how OpenCV can manipulate a camera matrix I was able to implement it's calibration and distortion correction functions in to my software.

Simply put, OpenCV uses a checkerboard pattern to calibrate a camera by seeing how bent the lines in relation to each other are.

I first take a bunch of pictures of the checkerboard in the frame. Then use OpenCV to calculate the intrinsic matrix and the distortion coefficients matrices. Finally use the "undistort" function to revert the distortion caused by the fisheye lens. The end result is either good, or not so good. Depending on how many / how good the pictures you gave the program are.

6/3/20 - Issues with Current Vision System

After spending some time getting the calibration working best I could, here some issues I am dealing with using the current system as a whole.

1) For an accurate calibration it need upwards of 30-50 pictures of the checkerboard in different positions which takes almost 15 minutes.

2) Added complexity to the code, which made the program run slower since it needs to undistort each frame that the camera captures.

3) The whole tracking system is very sensitive to light conditions, I noticed as I tested outside, even if an hour went by and the sun was at a different angle, the software would behave differently.

4) Each camera needs to mounted quite far from the table, requiring a large and unreliable setup.

I think I have some ideas on how to improve the current design however! More to come!

6/5/20 - Changing up Vision System!

Breaking this problem down, I basically just need to track the ball in 3D space. One 2D camera is not sufficient since it will not give us depth which is the 3rd Dimension.

After a little research, Humans see using 3D stereoscopic imaging. Two eyes produce two slightly different / offset images and then showing each eye only one of those images. Our brain then "combines" these images into one 3D image.

We can recreate this using cameras spaced out a known distance and finding the angle that is between them and the tracked object. This will allow us to ditch the large setup and use two cameras placed near the arm angled at the table!

6/15/20 - Upgraded and Installed Vertical Axis

Today I finished the the upgraded horizontal axis with installing a thicker belt and larger stepper for more torque. I also mounted the vertical axis to the base plate. I'm waiting on the other driver to come to fully test the rig out to see how the sliders move together. I will work on positioning the paddle on the 2D plane when the other supplies come in the mail.

PinPongTracker.mp4

6/26/20 - Working on Prediction Algorithm

I am still using the camera on the large over head mount while I work on the prediction, but I plan on switching to the new vision system soon!

Here I have the sequential imaging, simple prediction algorithm, and image video at work. It seems to work really well when it tracks the right object. This problem of tracking other moving things I think can be solved by combining different tracking methods into one.

7/06/20 - Second Camera

Today I finished adding software for the second camera to capture the second angle. There are two frames for the two cameras with the exact same movement based tracking system on them.

In the future, I would like to combine these two predicted points into an, "x,y,z" coordinate plane with the origin being one of the corners of the table.

7/15/20 - Rendering Table

The table scales to the correct size of the trimmed table and makes visualizing what the computer is seeing super easy.

Ideally though, I really want to get a 3D render, but I don't know how to do this, ill figure it out eventually. But for now this taught me a lot!

7/24/20 - Python!!

I've made a decision. For all further programing I'm going to be using python. Python is a powerful and high-level programming language that lets you develop a wide variety of applications. And, most of all, for me, it is SUPER easy to understand when compared to C++. I loved C++ but I soon realized as I complicated the project C++ quickly became outside my comfort zone and I felt I was better off swapping.

Luckily, I've learned so much when it comes to programming in general that I feel I can easily Phython-ify my code from C++. After some digging, there are also tons of OpenCV computer vision projects with great documentation as well. After a day installing and playing around with python I instantly fell in love with it. Using Pycharm as my IDE, libraries and super easy to install and overall a super beginner friendly environment when compared to C++.

I'm very happy with the switch since it allows me to skip over the super tough learning curve with C++ and power through with the project. While I AM swapping main languages, the skill of C that I have learned are super cool and I definitely continue using it as I grow as a programmer. (Also If I wanted to game develop, Unity uses C#!)

8/29/20 - Learning Python!

Over the past while, I've been having a blast learning and making small projects with Python. At the end of the day, My goal is to learn as much possible so having fun making small projects to learn, so this is a good use of my time! Here a couple projects that I've made myself or recreated:

Simple Tennis Ball Tracker
Snake!
Face Detection Software

11/02/20 - With the starting up with school I've been super stock pilled with work, I haven't been able to catch up with the project. While I've hated that I had to take a break and focus on school, I got to do what I got to. I plan to continue with the project once hybrid school starts to stabilize. Updates to come!!

Page updated

Report abuse