TOH EE SEN, IZEN

Overview of our Project

We were assigned two different tasks; to work on the software of two different floor cleaning robots - Panthera and hTetro, by working on the object classification of both of their systems.

To complete both tasks, we took photos of trash and non-trash items and used the labelImg.py software to label them. For the Panthera, objects were split into the classes "Pick Up" and "No Pick Up". For the hTetro, objects were split into the classes "Can Clean" and Cannot Clean". Afterwards, we passed the labelled images throught the network to teach it. In addition to that, we co-authored the scientific reports regarding our experiments and research done. In total, we worked on 2 reports with journals under the mentorship of our mentors Mr Bala and Dr Mohan.

Methodology

Our process for the training our Artificial Intelligence system went as follows

1. Planning of training photos needed
2. Obtaining of training photos
3. Labelling of training photos
4. Training of AI
5. Review of results
6. (repeat if needed)

To obtain trash to get photos of, we gathered trash that was found in the SUTD office, and from our homes. We were deliberate in taking our photos, getting many different angles as possible of the trash, on as many different kinds of pavements as we could find. Each time we snapped a few shots, we also rearranged items in the photos or distorted the shapes of the objects (by crushing/bending them), to achieve more variation. The labelling of photos is done to tell the AI what each object should be classified under, so it can learn the right things. We tried to standardise our labelling methods, such as what objects we should classify under which class, and how big the "bounding boxes" are for each object. ("bounding box" referring to the box that encases the object on the photo, see below for examples). We used the SSDMobilenet framework for our AI. To do this, we simply placed all our labelled images in a folder and ran the program that our mentor shared with us. The program would train the AI, then attempt to detect and classify objects in photos given to the program. From this, we can find out if there's any issues with the detection and classification and deduce whether there is anything we can do to improve the training results, whether it's taking better photos, labelling the photos better, or using a different AI framework.

Main Takeaways from this Experience

3 Content Knowledge/Skills learnt

1. We were required to utilise our problem solving skills in completing some of the tasks that were given to us, as it was not always as straightforward as following a given set of instructions. We conducted several rounds of training our artificial intelligence system before our mentor was satisfied with its accuracy. We realised that it was falsely classifying many objects, and after discussing this we came to the conclusion that it was because our object labelling methods were not standardised enough. We took this into account and used more structured methods to obtain and label training data. Our final training results came out satisfactory.
2. During the attachment, we also got the opportunity for a first hand experience at learning more about artificial intelligence. We had collected over almost 2000 images for training (1800, to be exact) for both robots that we worked on, afterwards labelling then feeding these images to the AI training program. Finally we got to see the AI that we had trained start recognising objects in images just as we had trained it (for the most part at least).
3. Learning to write scientific reports was also one of the main components of our attachment. We learnt how the researchers are able to produce professional looking scientific reports. I had always thought that it was required that one be good at using all the features of Microsoft Word (or similiar software) if one was to write a professional looking paper. Using https://overleaf.com, we were able to type out content in text form, and use LaTeX (something similiar to a programming language except it's used for text formatting) to format our paper. Other than the visuals, it was also important that we maintained a certain formality in the language we used when typing the scientific report.

2 Interesting Aspects of the Experience

1. The accuracy of the artificial intelligence system is not expected to be as high as I had thought. I was not satisfied with our final training results, and we were not able to conduct training many more times as the end of the attachment was approaching. Good systems can approach 95% accuracy, but our framework was only able to achieve (approximately) 70% accuracy. But our mentor said that the results were very good already. It's worth considering that we were using a highly portable AI framework, which is able to be used on smaller systems such as smartphones, but the accuracy of the framework was reduced to achieve this. In fact, the lower accuracy of a floor cleaning robot can be accounted for by letting the robot scan over the same area of floor several times, to reduce the chances that it misses out on a piece of trash.
2. The layout of the office that the SUTD staff worked in was quite unique. We got to see this place several times, and we'd always see random prototypes of robots lying around the room. There were several Xbox/Playstation controllers which were probably used for controlling the robots manually, there were several 3d printers, 2 open concept meeting areas and low separation between workers' desks.

1 Takeaway for Life

1. It’s difficult to be productive when deadlines are changing without notice, instructions are unclear, effective communication between both ends is lacking. There’s going to be many opportunities to work with many different kinds of people, and in all of that, the only constant is going to be yourself. It’s hard to change other people, so the best option is to be the person that the team needs. You could be the best team leader, or the best team player. That’s the best way one could contribute to a team, I think.