SANSKRITI BHATNAGAR

Background information of the projects / tasks which I was involved in

Our project mainly focused on using the robot to do a daily task. We came up with several ideas before we came up with the final one. Our idea is to use the robotic arm to sort through trash. Singapore has an aging population and Singaporeans are not willing to do such jobs. So the number of people to help us at recycling centres will decrease. This means that we have to rely more on robots and machines to do such tasks for us. This is where the robot would come in handy and can help recycle our trash. Also as the world becomes more conscious of the environment, the robot can also be used in homes. Japan asks its residents to sort through all their trash and put into different bags. This can be very problematic but with the help of the robot the residents no longer have to do it on their own. The robot can do it for them.

Elaboration of Process

For the first week, our mentors introduced us to robot and told us that we needed to learn how to code using python. They started us off with some links to websites like udemy so we knew where to learn but other that that it was mostly independent learning. We had already learned how to code using python IP2 but we still needed to refresh our knowledge so we referred to online tutorials on YouTube and other websites. The mentors also asked us to start brainstorming ideas for our project. We had to use the robotic arm for daily tasks. We came up with several ideas and presented them to our mentors by the end of the week. The first one we came up with was to use the robot arm to paint, our mentor had shown us videos where they has used the robotic arm to draw a picture and we thought that we could do something similar. The second idea was to use the robotic arm to make cake batter. We wanted the robotic arm to pick up ingredients from the table and put it in a bowl and mix it together to make cake batter. The mentor had shared with us that she had also come up with similar idea where she wanted the robotic arm to make a salad. She said that robotic arm could slice up the vegetables and then mix it together in a bowl. Our third idea was that we could the robotic arm to massage people. Our mentors found this idea hilarious, but they said that it would not be safe as the robotic arm is an industrial machinery. Lastly, our final idea was to use the robotic arm to sort and throw our trash. Our mentors liked our last idea the most and we chose to use our last idea for our final project. Since our idea required us to use computer vison that suggested that we learn OpenCV and sent us a few links to online tutorials.

For the second week our mentors asked us to start writing our codes for our project. We initially started with a very simply pick and place code for the robotic arm. We just hardcoded the positions we wanted the robotic arm to go to and placed a object there accordingly. After this we started to research more on object detection so that our robotic arm would know here the trash is located. We found several online tutorials that used TensorFlow to detect objects. However to train your own neuro network is not easy and a large dataset is required which was not easy to collect. Fortunately, we were able find an opensource YOLOv3 that could detect the object for us. After which, we started to write the code for our final project. We mounted the camera on top of the robotic arm using masking tape so that a picture of the area that needs to cleaned can be taken. So, we used OpenCV to create a 4x4 grid and the position of arm for each grid was hardcoded. The object was detected using YOLO and then it was located using OpenCV after which the arm moved to go pick up the piece of trash and disposes it into the trash can. The problem we faced with using opensource is that it could only detect plastic bottles. We decided that we needed to solve this problem since we still had enough time.

So on the third week after all the groups had presented we now had a better idea on what each group is doing this led to us collaborating with Gabriel and Chu Qi’s group. Their group had presented on creating object detection programs and facial detection programs. Since we needed a trash detection program their group volunteered to help us make one which we can use for project. The first step was still to find a dataset since we did not have the resources to create our own dataset. We were lucky enough to find a dataset online that the trash divided into three classes – plastics, glass, paper, metal and trash(this was everything that could not put into the other four categories). We had a total of five people in our group so each person just labelled the images from each category. We used labelImg to label the images. After all the images were labelled we sent the files to Gabriel and Chu Qi who started working on the training the neuro network for the trash detection. For the time they worked on it, we decided to finish up our deliverables, the poster and the video. By the end of the week, they finally were able to create a trash detection program but their program required us to use Tensorflow and none of our laptops were able to download Tensorflow so we had to use the computers in the office which we could not move hence we could not bring them to the lab so that we could test it with the robotic arm.

Elaboration of results

Our project aims for us to write a code that get our robot to sort through trash, and the deliverables required are a poster and a video. The poster has also the info on the steps the robot takes, our inspiration behind this project and how relevant it is to the real world. We also worked on a video which shows the robot in action and also explain what is happening in the subtitles. We cant post the video of our project here but we did come up with a final code with help from Chu Qi and Gabriel’s group.

3 Content knowledge and skills learnt

1. We all learnt how to code although we did learn in IP2 many of us had forgotten how to code but we were able to refresh our knowledge and were able to understand it better. We learnt python, OpenCV and URscript. Python coding has become more popular over the easier because it is very easy to understand, and the commands are English-like and many courses that we apply for outside of school even ask for programming as prerequisite knowledge. This was really good exposure to start us off and if we are interested, we can go learn how to code in other programming languages.
2. Something else our WOW! attachment had taught us is to be resourceful. Our mentors did not teach us coding like how our teachers taught it to us during the Skills and Knowledge lessons we had to find tutorials online ourselves so that we knew how to use OpenCV and URscript. We were using websites with lessons or YouTube videos.
3. Last thing we had to learn was to be independent. The mentors were there to guide us but we had to do most of the work ourselves. We had to set our own goals and deadlines and inform the mentors of these goals so they had a rough idea what we were doing and also give us some feedback example if they are feasible or not. This also felt very similar to an actual work environment where employees are assigned a big project and they continue to give regular updates to their supervisors so they could plan accordingly. I feel this would also be required of us in JC as we well especially during project work where we would have to complete our projects according to a specific timeline.

2 interesting aspects of MY learning

1. One thing that was interesting was the hands-on aspect of the project. We were given the opportunity to use the robot and use different codes to see how it works. We have not given any opportunity to do anything similar in schools since most of our projects only require research and do not require us to do any hands on so this was something was still rather new for us.
2. I usually do not prefer projects related to technology and coding so the reason I chose this project was for exposure since it was something new. It was exciting experience because I got to learn something new like coding since it was not something that knew how to do well. Although I am still not very good at coding I have gotten better over the past months as I knew the commands and how to use them.

1 takeaway for life

One of my takeaways is to always try something new and not to be afraid. I was not someone who had tried programming before so this was really new for me and it gave me an opportunity to learn something new. I feel like we should have this attitude about everything and always try to step outside of my comfort zone because it teaches new things about yourself. Although I learnt that programming is not for me it was still something new and it gave me an opportunity to learn something new.