Embedded Files
Background information of A*STAR and our task:
The project aims to investigate the use of robots for entertainment in interactive settings, such as board games, physical games and interactive drawing. We need to learn how to work with an actual robot arm, and will be responsible for planning and implementing a simple interactive game using the robot arm.
Development in the robotics sector of science has not only simplified human beings work over time, but also provided a medium of entertainment. It is common to see entertainment robots that are mass-produced and relatively cheap nowadays. The entertainment robots are more than affordable toys that can be enjoyed by the common people.Through playing against itself for multiple hours with it only given the rules of chess and no other instructions, it remains unbeatable by any human players until today. The ability of computers to play games such as chess and go, which have clear sets of rules and an almost infinite level of variation of play, have been recognized as a test for the development of weak AI. What the common people perceive as entertainment are actually important milestones of how much we have progressed in the field of robotics.
We have three groups in total of SCEI, our group which is SCEI002 have shared two mentors with the group SCEI002. A*STAR AI is a warm, lovely and inviting place with many different people and it the office layout is very modern and it increases the level of comfort that we feel. We quickly felt acclimatized to the environment here as our mentors and the other people in the office helped us integrate well and be familiar with the place. Each of us were allocated to our own working desk.
We had done plenty of research on the definition of the robots, the history of entertainment robot and the brief history of the games to prepare for our project. We have also came up with a lot of ideas for the games, such as connect4, snakes and ladders, the claw machine. After the presentation, we decided to do the claw machine first because it is the easiest to code for. We also considered to do the connect 4 if we have extra time.
We also learnt about basic python and OpenCV to prepare us for eventually programming the robotic arm for our project. We found it hard but interesting and we helped each other to better understand the concepts.
this is the user manual of our robot arm (UR5)
This is the URScript programming language which helps us better use the coding
Elaborations of activities done and results :
1. The progress and the knowledge about Connect 4 :
By Applying the basic python knowledge we learnt from last week and the help of online tutorials, we are able to code a playable connect four game. We used numpy to generate a “board”, which is basically a 6 by 7 matrix made of ‘0’s, and represented the pieces using ‘1’s and ‘2’s. Using pygame, we have also drawn out the board and pieces to provide a smoother game play for the player. This procedure was optional as the final project will be showcased on an actual connect 4 board. For robotic arm control, we coded a simple joystick code by giving ‘w’,’a’,’s’,’d’ corresponding values to rotate the joints in different axis. We made an “AI” for our connect 4 game by using an algorithm called minimax. What a minimax algorithm does is that it takes all the possible moves for each player, maximising and minimising player, and uses a scoring system to determine the best possible moves to make. The further into the future the algorithm looks, the harder it will become to play against the AI as it figures out the most optimal solution.
By experimenting with blue colour detection code importing opencv, we roughly had a clue on board detection. Game pieces in Connect 4 have distinct colour of red and yellow. Given that it is not too complex to discover the range value of these colours on opencv (on the rgb scale), and distinct blue colour of the board will not be easily mixed up with the rest of the colour, board detection is doable in a few days. The only issue would be to link the detection of colour with the actual gamerule. This will also become a hindrance to the coding of robotic arm movement. Nevertheless, we decided to start the actual coding on Tuesday. Firstly, we rewrote the gamerule code base on our own knowledge of the game and python. The display section of the code is removed so that it won’t cause further complications with the infusion of opencv codes.
2. The progress and the knowledge about Claw machine :
After looking through basic functions of URX, a python library that control UR robots, we experimented with the movement of robotic arm. We first attempted to run a raw code of joy-stick copied from Github, but realise that it did not work. As an alternative solution, we decided to observe the movement of robotic arms through manually control, only to realise that to move the robotic arm horizontally or vertically along one line is very difficult, as a simple movement of the robotic arm may require the movement of more than one joint. By then, we were only able to move the robotic arm between designated position based on the angle of each joint. Position (0,0,0,0,0,0) of the robotic arm was discovered on that day so that we can make more accurate calculations in terms of robotic arm’s movement in the future.
We worked with our code in the lab, and we found that if we pressed the keyboard which moving to the left and the keyboard which moving to the right at the same time, the robot arm could move horizontally to x-axis. So this solves the motion problem that we had before, and it is now free to move back and forth, left and right, like a normal baby grabber. And we're going to speed it up to make it a little bit smoother. We have done the code to grab the claw machine, and we've brought the plush toy to the lab to simulate our final product. We felt very proud of ourself because we have done what we aimed to do. We have already finished the coding for the claw machine, we have experienced the claw machine in the lab.
How the coding for the claw machine works:
Our code consists of 2 major components: msvcrt and urx. Msvcrt is a built-in python library that allows us to use 2 functions, kbhit() and getch(), which can be used together to detect whether a key has been pressed on the keyboard, and automatically detect what key is being pressed without needing to click ‘Enter’. urx is a python library to control an UR robot, which enable us to move the robotic arm and control the opening of its gripper. In urx, there is a function known as movel([x, y, z, rx, ry, rz], acc, vel, wait, threshold), which will cause the robotic arm to move to a specific area (nvm i’m too lazy to explain the whole xyz thing here’s an image)
In our project, we assigned the keys “w, a, s, d” with values that when pressed, will move a small distance forward, leftward, o backwards and rightward (from our perspective) respectively. Combining this code with the msvcrt functions mentioned above, by pressing and holding down any of the “w, a, s, d” keys, the robotic arm will move in the direction the user intended it to until the key is released.
Our final product :
We have done a claw machine, we used the coding to control the robot ram as the gripper to move and catch the dolls.
3 content knowledge and skills that I have learnt:
1. I have learnt the coding in python, I think it is a very useful skill. I do not have programming basics, but at last of the program, I have finished the complex coding for our claw machine with the help of the online tutorials with my group mates. We have also learnt the use of the coding in the actual robot arm through opencv. OpenCV is a library of programming functions mainly aimed at real-time computer vision. With the use of opencv, we can control the robot arm by our coding.
Nowadays, programming is an important skill, it is because the advanced technology are everywhere in our lives. Python is great for building micro-project to macro enterprise web services as well as on supporting other types of programming languages.With better programming knowledge, I may solve problems more logical.
2. I have also learnt the use of the coding in the actual robot arm through opencv. OpenCV is a library of programming functions mainly aimed at real-time computer vision. With the use of opencv, we can control the robot arm by our coding. It is hard because we also need to learn how the robot arm works with its particular positions.
3. I also learnt problem solving skills during the attachment.One of professors from SUTD, Mr Jianxi Luo, came to A* and shared with us about his experience on solving engineering problems.When I'm solving a problem, I can't focus too much on the knowledge or content of a particular topic, maybe I can take what I've learned from other topics and apply it to the problem. In this case, we may have diverse ways to solve one problem.
2 interesting aspects:
I think figuring out how the robot arm works is really interesting. We learnt how to use the coding to control the robot arm and we did it for an interactive game, it is enjoyable and interesting.
Besides, the working environment is really different with school, we always have meetings for improving our final products. It is really interesting to experience in working environment.
Takeaway of life:
I think it is teamwork. I realised the importance of group cooperation and accumulated some work experience. With teamwork, we can have diverse methods to do our product. We can also help each other, to give play of everyones’ advantages. We also encourage and supervise each other for getting a better product.
Page updated
Google Sites
Report abuse