Driving Question: How can computer vision be used to interact with websites online?

Purpose: To assess what is possible with computer vision and create a preliminary project utilizing it. We will be using computer vision techniques with the OpenCV library to create a interactive dinosaur game. The goal of the game is to help a stranded dinosaur get back home by solving various challenges and puzzles. Along the way, you will learn about different STEM (science, technology, engineering, and math) concepts and how to apply them in a fun and engaging way.

Background Research:

Before making this project, I had to research what coding software would work best, how best to go about coding it, and how to get the computer vision to work properly. The exact method I would use to control it also had to be planned carefully and implemented in the best way possible.

1. Computer vision: This field involves the use of computers to analyze, understand, and process visual data from the real world. OpenCV is a popular open-source library that provides tools and functions for performing various computer vision tasks, such as image and video processing, feature detection, and object tracking. You can learn more about the basics of computer vision and how to use OpenCV by reading online tutorials and documentation.

2. Python programming: Python is a popular programming language that is widely used in various fields, including scientific computing, data analysis, and web development. It is known for its simplicity, readability, and flexibility, making it a great language for beginners. You can learn more about Python programming by reading online tutorials and taking online courses.

3. Game development: Creating a game involves a combination of programming, design, and creativity. You will need to think about the gameplay mechanics, the user interface, the graphics and sound, and how to make the game fun and engaging. You can learn more about game development by reading online tutorials and looking at examples of games created with Python and OpenCV.

4. STEM concepts: The STEM SOS PBL Dinosaur Game project is designed to help you learn about various STEM concepts and how to apply them in a real-world context. Some possible topics to explore include problem-solving, logic, spatial reasoning, and physics. You can learn more about these concepts by reading online resources and engaging in hands-on activities.

Materials:

We will be using the below libraries for the project:

OpenCV is a huge open-source library for computer vision, machine learning, and image processing. OpenCV supports a wide variety of programming languages like Python, C++, Java, etc. It can process images and videos to identify objects, faces, or even the handwriting of a human.

MediaPipe Library offers open-source cross-platform, customizable ML solutions for live and streaming media.

PyAutoGUI library lets your Python scripts control the mouse and keyboard to automate interactions with other applications. The API is designed to be as simple. PyAutoGUI works on Windows, macOS, and Linux.

Procedure:

• Do background research and find methods for proper coding

• Launch website and code in PyCharm

• Install MediaPipe, OpenCV, and PyAutoGUI Packages

• Connect code to Dinosaur Game in Google Chrome

• Test and debug

Requirements for this project: Make sure you install the following programs on your computer.

• Install python – Download the latest version for your computer. www.python.org/downloads

• Install Pycharm – Download the latest version for your computer www.jetbrains.com/pycharm/download

• Install MediaPipe, OpenCVand PyAutoGUI Packages by using the pip install library.

• Run these pip commands in your terminal:

pip install selenium

pip install PyAutoGUI

pip install MediaPipe

Pip install cv2

Essential Understanding:

I learned computer vision and image processing concepts with this project. I learned about concepts such as edge detection, color space, and feature extraction, and how they can be applied to detect and classify objects in images. I gained practical experience with programming concepts such as loops, conditionals, and functions, as well as develop my problem-solving skills as mydesign and debug my project. SI also learned about the basics of game development, including how to design game mechanics, create game assets, and implement user interactions. I also learned about the principles of game design, such as player engagement, challenge, and feedback This project helped me to understand about learning about the role of computer science in creating interactive experiences such as games, and how my dinosaur game project illustrates this. I learned about the different types of careers in the gaming industry and how computer science skills are relevant to them.

Real Life Connection:

1. Education: Games can be a powerful tool for teaching and learning, particularly in subjects such as computer science and mathematics. A dinosaur game could be used to teach students about concepts such as programming, geometry, or logic in a fun and engaging way.

2. Rehabilitation: Games can be used to help people recover from injuries or conditions that affect their physical or cognitive abilities. A dinosaur game could be used to help people with conditions such as stroke, traumatic brain injury, or developmental disorders to improve their motor skills, coordination, or attention.

3. Research: Games can be used as a research tool to study a wide range of phenomena, such as human behavior, decision-making, or social interaction. A dinosaur game could be used to study how people learn, adapt, or cooperate, and to test hypotheses about these processes.

4. Entertainment: Games are a popular form of entertainment, enjoyed by people of all ages around the world. A dinosaur game could be used to provide entertainment and enjoyment to players, and to create a sense of community and connection with others.

Investigation Questions:

1. What are the requirements for creating this project?

This project required a well-rounded knowledge of both Python and OpenCV, as well as a good structure in mind. To avoid any failures in the execution of our code, it needed to be structured well and have a clear method of both receiving the input and sending the proper output. On a more physical level, we also needed a computer with a webcam and the proper software downloaded.

2. What field would benefit the most from the application of computer vision?

This is a tough question due to the applications of computer vision being incredibly wide, but we think mapping and security could possibly benefit the most from it. Self-driving cars are an emerging technology and they wouldnt be possible without the mapping technologies that computer vison provides. Computer vision allows the car to map its surroundings and deduce any possible threats/obstacles. With the constant development and fine-tuning of computer-vision, we could see its applications everywhere within the next few years/decades.

3. How can we improve the accuracy and reliability of our hand gesture recognition system?

Increase the amount of training data: The more diverse the training data, the better the model will be able to generalize to new gestures. Use better features: The features used to represent the gestures should be carefully designed to capture the relevant information about the gestures. This could include using depth data from a depth camera or using a combination of hand and joint positions. Use a more powerful model: A more powerful model, such as a deep neural network, may be able to better learn the relationships between the features and the gestures. Use transfer learning: If you have a small amount of data, you can use a pre-trained model and fine-tune it on your hand gesture dataset. This can allow you to leverage the knowledge learned by the pre-trained model on a larger dataset.

4. How can we test and evaluate the performance of our hand gesture control system?

There are several ways to test and evaluate the performance of a hand gesture control system. Some potential approaches include: Functional testing: This involves testing the system to ensure that it performs the tasks it is designed to do, such as recognizing and interpreting specific hand gestures. Performance testing: This involves measuring the system's speed, accuracy, and reliability. For example, you could measure the time it takes for the system to recognize and respond to a hand gesture, or the percentage of gestures that are correctly recognized. User testing: This involves having actual users test the system to see how well it performs in a real-world setting. This can help identify any usability issues and gather feedback on the user experience. Stress testing: This involves testing the system under high load or adverse conditions to see how it performs and identify any potential weaknesses.

5. What additional features can we add to the dinosaur game using the hand tracking system, such as power-ups or special moves?

Special moves: You could allow players to perform special moves by making specific hand gestures. For example, you could allow players to perform a "super jump" by making a fist with their hand. Power-ups: You could allow players to activate power-ups by making specific hand gestures. For example, you could allow players to activate a shield or a speed boost by making a specific hand gesture. Health or energy meter: You could use the hand tracking system to create a health or energy meter that decreases as the player takes damage or performs actions in the game. Players could restore their health or energy by making specific hand gestures, such as a "healing" gesture. Multiplayer mode: You could use the hand tracking system to allow multiple players to play the game at the same time, each controlling their own dinosaur.

Code:

Dinosaur Game Using OpenCV PBL Project Video