Big Picture

To incorporate more physical movement in computer games using Python and OpenCV, libraries and frameworks provided by these technologies can be used to develop interactive, motion-based gameplay mechanics. This can be done by utilizing OpenCV's image processing capabilities, integrating motion-sensing input devices and machine learning and computer vision techniques such as object detection and gesture recognition. 

Principle Concepts

Having a basic understanding of Python is important before attempting to take on an OpenCV or Mediapipe project because Python is a powerful and versatile language that can be used for a wide range of programming tasks, including image and video processing. Additionally, many popular libraries and frameworks for computer vision and machine learning, such as OpenCV and TensorFlow, have Python interfaces that make it easy to interact with these tools. Furthermore, Python is a widely used language in the field of computer vision and machine learning, providing a wealth of resources and documentation, which makes it a great choice for developing computer vision and machine learning projects. 

Keywords

1. Function: A block of code that performs a specific task and can be called by other parts of the program. Functions are used to organize and modularize code, making it more readable and reusable.

2. Variable: A container that holds a value, which can be of various types such as numbers, strings, and lists. Variables are used to store and manipulate data in a program.

3. Loop: A control structure that allows a block of code to be executed multiple times. The most common loops in Python are the for and while loops.

4. Class: A blueprint for creating objects, which can have properties and methods. Classes are used to model real-world objects and encapsulate data and behavior.

5. Module: A file that contains Python definitions and statements. Modules can be imported and used in other parts of a program to provide additional functionality.

6. Exception: An abnormal event or error that occurs during the execution of a program. Exceptions are used to handle and recover from errors in a program.

7. List: A collection of items that can be of different types. Lists are used to store and manipulate collections of data in a program.

Big Picture

To incorporate more physical movement in computer games using Python and OpenCV, libraries and frameworks provided by these technologies can be used to develop interactive, motion-based gameplay mechanics. This can be done by utilizing OpenCV's image processing capabilities, integrating motion-sensing input devices and machine learning and computer vision techniques such as object detection and gesture recognition. 

Principle Concepts

Having a basic understanding of Python is important before attempting to take on an OpenCV or Mediapipe project because Python is a powerful and versatile language that can be used for a wide range of programming tasks, including image and video processing. Additionally, many popular libraries and frameworks for computer vision and machine learning, such as OpenCV and TensorFlow, have Python interfaces that make it easy to interact with these tools. Furthermore, Python is a widely used language in the field of computer vision and machine learning, providing a wealth of resources and documentation, which makes it a great choice for developing computer vision and machine learning projects. 

Keywords

1. Function: A block of code that performs a specific task and can be called by other parts of the program. Functions are used to organize and modularize code, making it more readable and reusable.

2. Variable: A container that holds a value, which can be of various types such as numbers, strings, and lists. Variables are used to store and manipulate data in a program.

3. Loop: A control structure that allows a block of code to be executed multiple times. The most common loops in Python are the for and while loops.

4. Class: A blueprint for creating objects, which can have properties and methods. Classes are used to model real-world objects and encapsulate data and behavior.

5. Module: A file that contains Python definitions and statements. Modules can be imported and used in other parts of a program to provide additional functionality.

6. Exception: An abnormal event or error that occurs during the execution of a program. Exceptions are used to handle and recover from errors in a program.

7. List: A collection of items that can be of different types. Lists are used to store and manipulate collections of data in a program.

Step 0: Setup PyCharm and Supporting Libraries

Step 1: Pose Detection

Step 2: Starting Mechanism

Step 3: Horizontal Movements

Step 4: Vertical Movements

Step 5: Keyboard and Mouse Input

Step 6: Compile

Download PyCharm(Community) using this link

Once set up with everything, create a new .py file. This creates a python-based file

In order to use the libraries, we need to download them. To download them, we can run the following commands inside of the terminal:

pip install pyautogui 

pip install time

pip install matplotlib

pip install mediapipe

pip install cv2

Next, we import these into our program by using: 

import cv2 #pip install cv2

import pyautogui #pip install pyautogui

from time import time #pip install time

from math import hypot #pip install math

import mediapipe as mp #pip install mediapipe

import matplotlib.pyplot as plt #pip install matplotlib

These lines call the libraries into our program, making them and their functions usable to us. 

Now we initialize the media pipe pose class, which sets up the pose function and allows us to see the body lines on the screen

# Initialize mediapipe pose class.

mp_pose = mp.solutions.pose

# Setup the Pose function for images.

pose_image = mp_pose.Pose(static_image_mode=True, min_detection_confidence=0.5, model_complexity=1)

# Setup the Pose function for videos.

pose_video = mp_pose.Pose(static_image_mode=False, model_complexity=1, min_detection_confidence=0.7,

                         min_tracking_confidence=0.7)

# Initialize mediapipe drawing class.

mp_drawing = mp.solutions.drawing_utils

We are going to create a detectPose() function that can be called later in the code. The purpose is so it can plot the 32 points on a person, and track that person only, eliminating pickup of people walking in the background.

def detectPose(image, pose, draw=False, display=False):

    '''

    This function performs the pose detection on the most prominent person in an image.

    Args:

        image:   The input image with a prominent person whose pose landmarks needs to be detected.

        pose:    The pose function required to perform the pose detection.

        draw:    A boolean value that is if set to true the function draw pose landmarks on the output image.

        display: A boolean value that is if set to true the function displays the original input image, and the

                 resultant image and returns nothing.

    Returns:

        output_image: The input image with the detected pose landmarks drawn if it was specified.

        results:      The output of the pose landmarks detection on the input image.

    '''

    # Create a copy of the input image.

    output_image = image.copy()

    # Convert the image from BGR into RGB format.

    imageRGB = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    # Perform the Pose Detection.

    results = pose.process(imageRGB)

    # Check if any landmarks are detected and are specified to be drawn.

    if results.pose_landmarks and draw:

        # Draw Pose Landmarks on the output image.

        mp_drawing.draw_landmarks(image=output_image, landmark_list=results.pose_landmarks,

                                  connections=mp_pose.POSE_CONNECTIONS,

                                  landmark_drawing_spec=mp_drawing.DrawingSpec(color=(255,255,255),

                                                                               thickness=3, circle_radius=3),

                                  connection_drawing_spec=mp_drawing.DrawingSpec(color=(49,125,237),

                                                                               thickness=2, circle_radius=2))

    # Check if the original input image and the resultant image are specified to be displayed.

    if display:

        # Display the original input image and the resultant image.

        plt.figure(figsize=[22,22])

        plt.subplot(121);plt.imshow(image[:,:,::-1]);plt.title("Original Image");plt.axis('off');

        plt.subplot(122);plt.imshow(output_image[:,:,::-1]);plt.title("Output Image");plt.axis('off');

    # Otherwise

    else:

        # Return the output image and the results of pose landmarks detection.

        return output_image, results

Instead of having to press play all the time, we developed a function checkHandsJoined() to check if the hands are joined. When they are joined it will press play (space)

def checkHandsJoined(image, results, draw=False, display=False):

    '''

    This function checks whether the hands of the person are joined or not in an image.

    Args:

        image:   The input image with a prominent person whose hands status (joined or not) needs to be classified.

        results: The output of the pose landmarks detection on the input image.

        draw:    A boolean value that is if set to true the function writes the hands status & distance on the output image.

        display: A boolean value that is if set to true the function displays the resultant image and returns nothing.

    Returns:

        output_image: The same input image but with the classified hands status written, if it was specified.

        hand_status:  The classified status of the hands whether they are joined or not.

    '''

    # Get the height and width of the input image.

    height, width, _ = image.shape

    # Create a copy of the input image to write the hands status label on.

    output_image = image.copy()

    # Get the left wrist landmark x and y coordinates.

    left_wrist_landmark = (results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_WRIST].x * width,

                          results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_WRIST].y * height)

    # Get the right wrist landmark x and y coordinates.

    right_wrist_landmark = (results.pose_landmarks.landmark[mp_pose.PoseLandmark.RIGHT_WRIST].x * width,

                           results.pose_landmarks.landmark[mp_pose.PoseLandmark.RIGHT_WRIST].y * height)

    # Calculate the euclidean distance between the left and right wrist.

    euclidean_distance = int(hypot(left_wrist_landmark[0] - right_wrist_landmark[0],

                                   left_wrist_landmark[1] - right_wrist_landmark[1]))

    # Compare the distance between the wrists with a appropriate threshold to check if both hands are joined.

    if euclidean_distance < 130:

        # Set the hands status to joined.

        hand_status = 'Hands Joined'

        # Set the color value to green.

        color = (0, 255, 0)

    # Otherwise.    

    else:

        # Set the hands status to not joined.

        hand_status = 'Hands Not Joined'

        # Set the color value to red.

        color = (0, 0, 255)

    # Check if the Hands Joined status and hands distance are specified to be written on the output image.

    if draw:

        # Write the classified hands status on the image.

        cv2.putText(output_image, hand_status, (10, 30), cv2.FONT_HERSHEY_PLAIN, 2, color, 3)

        # Write the the distance between the wrists on the image.

        cv2.putText(output_image, f'Distance: {euclidean_distance}', (10, 70),

                    cv2.FONT_HERSHEY_PLAIN, 2, color, 3)

    # Check if the output image is specified to be displayed.

    if display:

        # Display the output image.

        plt.figure(figsize=[10,10])

        plt.imshow(output_image[:,:,::-1]);plt.title("Output Image");plt.axis('off');

    # Otherwise

    else:

        # Return the output image and the classified hands status indicating whether the hands are joined or not.

        return output_image, hand_status

In order to make the character go left or right, we need a function to check if the person is going in either direction. The function checkLeftRight() is implemented to do this. 

def checkLeftRight(image, results, draw=False, display=False):

    '''

    This function finds the horizontal position (left, center, right) of the person in an image.

    Args:

        image:   The input image with a prominent person whose the horizontal position needs to be found.

        results: The output of the pose landmarks detection on the input image.

        draw:    A boolean value that is if set to true the function writes the horizontal position on the output image.

        display: A boolean value that is if set to true the function displays the resultant image and returns nothing.

    Returns:

        output_image:         The same input image but with the horizontal position written, if it was specified.

        horizontal_position:  The horizontal position (left, center, right) of the person in the input image.

    '''

    # Declare a variable to store the horizontal position (left, center, right) of the person.

    horizontal_position = None

    # Get the height and width of the image.

    height, width, _ = image.shape

    # Create a copy of the input image to write the horizontal position on.

    output_image = image.copy()

    # Retreive the x-coordinate of the left shoulder landmark.

    left_x = int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.RIGHT_SHOULDER].x * width)

    # Retreive the x-corrdinate of the right shoulder landmark.

    right_x = int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_SHOULDER].x * width)

    # Check if the person is at left that is when both shoulder landmarks x-corrdinates

    # are less than or equal to the x-corrdinate of the center of the image.

    if (right_x <= width//2 and left_x <= width//2):

        # Set the person's position to left.

        horizontal_position = 'Left'

    # Check if the person is at right that is when both shoulder landmarks x-corrdinates

    # are greater than or equal to the x-corrdinate of the center of the image.

    elif (right_x >= width//2 and left_x >= width//2):

        # Set the person's position to right.

        horizontal_position = 'Right'

    # Check if the person is at center that is when right shoulder landmark x-corrdinate is greater than or equal to

    # and left shoulder landmark x-corrdinate is less than or equal to the x-corrdinate of the center of the image.

    elif (right_x >= width//2 and left_x <= width//2):

        # Set the person's position to center.

        horizontal_position = 'Center'

    # Check if the person's horizontal position and a line at the center of the image is specified to be drawn.

    if draw:

        # Write the horizontal position of the person on the image.

        cv2.putText(output_image, horizontal_position, (5, height - 10), cv2.FONT_HERSHEY_PLAIN, 2, (255, 255, 255), 3)

        # Draw a line at the center of the image.

        cv2.line(output_image, (width//2, 0), (width//2, height), (255, 255, 255), 2)

    # Check if the output image is specified to be displayed.

    if display:

        # Display the output image.

        plt.figure(figsize=[10,10])

        plt.imshow(output_image[:,:,::-1]);plt.title("Output Image");plt.axis('off');

    # Otherwise

    else:

        # Return the output image and the person's horizontal position.

        return output_image, horizontal_position

Similar to checkLeftRight(), our new function checkJumpCrouch() checks for movement of the player. Instead of left and right, this one detects if a user jumps or crouches judging by a center line. It is implemented as follows:

def checkJumpCrouch(image, results, MID_Y=250, draw=False, display=False):

    '''

    This function checks the posture (Jumping, Crouching or Standing) of the person in an image.

    Args:

        image:   The input image with a prominent person whose the posture needs to be checked.

        results: The output of the pose landmarks detection on the input image.

        MID_Y:   The intial center y-coordinate of both shoulders landmarks of the person recorded during starting

                 the game. This will give the idea of the person's height when he is standing straight.

        draw:    A boolean value that is if set to true the function writes the posture on the output image.

        display: A boolean value that is if set to true the function displays the resultant image and returns nothing.

    Returns:

        output_image: The input image with the person's posture written, if it was specified.

        posture:      The posture (Jumping, Crouching or Standing) of the person in an image.

    '''

    # Get the height and width of the image.

    height, width, _ = image.shape

    # Create a copy of the input image to write the posture label on.

    output_image = image.copy()

    # Retreive the y-coordinate of the left shoulder landmark.

    left_y = int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.RIGHT_SHOULDER].y * height)

    # Retreive the y-coordinate of the right shoulder landmark.

    right_y = int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_SHOULDER].y * height)

    # Calculate the y-coordinate of the mid-point of both shoulders.

    actual_mid_y = abs(right_y + left_y) // 2

    # Calculate the upper and lower bounds of the threshold.

    lower_bound = MID_Y-15

    upper_bound = MID_Y+100

    # Check if the person has jumped that is when the y-coordinate of the mid-point

    # of both shoulders is less than the lower bound.

    if (actual_mid_y < lower_bound):

        # Set the posture to jumping.

        posture = 'Jumping'

    # Check if the person has crouched that is when the y-coordinate of the mid-point

    # of both shoulders is greater than the upper bound.

    elif (actual_mid_y > upper_bound):

        # Set the posture to crouching.

        posture = 'Crouching'

    # Otherwise the person is standing and the y-coordinate of the mid-point

    # of both shoulders is between the upper and lower bounds.    

    else:

        # Set the posture to Standing straight.

        posture = 'Standing'

    # Check if the posture and a horizontal line at the threshold is specified to be drawn.

    if draw:

        # Write the posture of the person on the image.

        cv2.putText(output_image, posture, (5, height - 50), cv2.FONT_HERSHEY_PLAIN, 2, (255, 255, 255), 3)

        # Draw a line at the intial center y-coordinate of the person (threshold).

        cv2.line(output_image, (0, MID_Y),(width, MID_Y),(255, 255, 255), 2)

    # Check if the output image is specified to be displayed.

    if display:

        # Display the output image.

        plt.figure(figsize=[10,10])

        plt.imshow(output_image[:,:,::-1]);plt.title("Output Image");plt.axis('off');

    # Otherwise

    else:

        # Return the output image and posture indicating whether the person is standing straight or has jumped, or crouched.

        return output_image, posture

We can't play the game without output to the keyboard. This is where PyAutoGui comes in. It can help us convert OpenCV camera input into an output that we can play the game with.

We can see this snippet from our code: **this is a snippet and will cause error if ran individually**

  # Check if the person has jumped.

            if posture == 'Jumping' and y_pos_index == 1:

                # Press the up arrow key

                pyautogui.press('up')

                # Update the veritcal position index of  the character.

                y_pos_index += 1

            # Check if the person has crouched.

            elif posture == 'Crouching' and y_pos_index == 1:

                # Press the down arrow key

                pyautogui.press('down')

Now we build the code that utilizes our functions defined above. It takes input from OpenCV camera, and gathers data from all the functions we call then produces output with PyAutoGui. This is not the final code!!! this cannot be run alone !

# Initialize the VideoCapture object to read from the webcam.

camera_video = cv2.VideoCapture(0)

camera_video.set(3,1280)

camera_video.set(4,960)

# Create named window for resizing purposes.

cv2.namedWindow('Subway Surfers with Pose Detection', cv2.WINDOW_NORMAL)

# Initialize a variable to store the time of the previous frame.

time1 = 0

# Initialize a variable to store the state of the game (started or not).

game_started = False  

# Initialize a variable to store the index of the current horizontal position of the person.

# At Start the character is at center so the index is 1 and it can move left (value 0) and right (value 2).

x_pos_index = 1

# Initialize a variable to store the index of the current vertical posture of the person.

# At Start the person is standing so the index is 1 and he can crouch (value 0) and jump (value 2).

y_pos_index = 1

# Declate a variable to store the intial y-coordinate of the mid-point of both shoulders of the person.

MID_Y = None

# Initialize a counter to store count of the number of consecutive frames with person's hands joined.

counter = 0

# Initialize the number of consecutive frames on which we want to check if person hands joined before starting the game.

num_of_frames = 10

# Iterate until the webcam is accessed successfully.

while camera_video.isOpened():

    # Read a frame.

    ok, frame = camera_video.read()

    # Check if frame is not read properly then continue to the next iteration to read the next frame.

    if not ok:

        continue

    # Flip the frame horizontally for natural (selfie-view) visualization.

    frame = cv2.flip(frame, 1)

    # Get the height and width of the frame of the webcam video.

    frame_height, frame_width, _ = frame.shape

    # Perform the pose detection on the frame.

    frame, results = detectPose(frame, pose_video, draw=game_started)

    # Check if the pose landmarks in the frame are detected.

    if results.pose_landmarks:

        # Check if the game has started

        if game_started:

            # Commands to control the horizontal movements of the character.

            #--------------------------------------------------------------------------------------------------------------

            # Get horizontal position of the person in the frame.

            frame, horizontal_position = checkLeftRight(frame, results, draw=True)

            # Check if the person has moved to left from center or to center from right.

            if (horizontal_position=='Left' and x_pos_index!=0) or (horizontal_position=='Center' and x_pos_index==2):

                # Press the left arrow key.

                pyautogui.press('left')

                # Update the horizontal position index of the character.

                x_pos_index -= 1              

            # Check if the person has moved to Right from center or to center from left.

            elif (horizontal_position=='Right' and x_pos_index!=2) or (horizontal_position=='Center' and x_pos_index==0):

                # Press the right arrow key.

                pyautogui.press('right')

                # Update the horizontal position index of the character.

                x_pos_index += 1

            #--------------------------------------------------------------------------------------------------------------

        # Otherwise if the game has not started    

        else:

            # Write the text representing the way to start the game on the frame.

            cv2.putText(frame, 'JOIN BOTH HANDS TO START THE GAME.', (5, frame_height - 10), cv2.FONT_HERSHEY_PLAIN,

                        2, (0, 255, 0), 3)

        # Command to Start or resume the game.

        #------------------------------------------------------------------------------------------------------------------

        # Check if the left and right hands are joined.

        if checkHandsJoined(frame, results)[1] == 'Hands Joined':

            # Increment the count of consecutive frames with +ve condition.

            counter += 1

            # Check if the counter is equal to the required number of consecutive frames.  

            if counter == num_of_frames:

                # Command to Start the game first time.

                #----------------------------------------------------------------------------------------------------------

                # Check if the game has not started yet.

                if not(game_started):

                    # Update the value of the variable that stores the game state.

                    game_started = True

                    # Retreive the y-coordinate of the left shoulder landmark.

                    left_y = int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.RIGHT_SHOULDER].y * frame_height)

                    # Retreive the y-coordinate of the right shoulder landmark.

                    right_y = int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.LEFT_SHOULDER].y * frame_height)

                    # Calculate the intial y-coordinate of the mid-point of both shoulders of the person.

                    MID_Y = abs(right_y + left_y) // 2

                    # Move to 1300, 800, then click the left mouse button to start the game.

                    pyautogui.click(x=1300, y=800, button='left')

                #----------------------------------------------------------------------------------------------------------

                # Command to resume the game after death of the character.

                #----------------------------------------------------------------------------------------------------------

                # Otherwise if the game has started.

                else:

                    # Press the space key.

                    pyautogui.press('space')

                #----------------------------------------------------------------------------------------------------------

                # Update the counter value to zero.

                counter = 0

        # Otherwise if the left and right hands are not joined.        

        else:

            # Update the counter value to zero.

            counter = 0

        #------------------------------------------------------------------------------------------------------------------

        # Commands to control the vertical movements of the character.

        #------------------------------------------------------------------------------------------------------------------

        # Check if the intial y-coordinate of the mid-point of both shoulders of the person has a value.

        if MID_Y:

            # Get posture (jumping, crouching or standing) of the person in the frame.

            frame, posture = checkJumpCrouch(frame, results, MID_Y, draw=True)

            # Check if the person has jumped.

            if posture == 'Jumping' and y_pos_index == 1:

                # Press the up arrow key

                pyautogui.press('up')

                # Update the veritcal position index of  the character.

                y_pos_index += 1

            # Check if the person has crouched.

            elif posture == 'Crouching' and y_pos_index == 1:

                # Press the down arrow key

                pyautogui.press('down')

                # Update the veritcal position index of the character.

                y_pos_index -= 1

            # Check if the person has stood.

            elif posture == 'Standing' and y_pos_index   != 1:

                # Update the veritcal position index of the character.

                y_pos_index = 1

        #------------------------------------------------------------------------------------------------------------------

    # Otherwise if the pose landmarks in the frame are not detected.      

    else:

        # Update the counter value to zero.

        counter = 0

    # Calculate the frames updates in one second

    #----------------------------------------------------------------------------------------------------------------------

    # Set the time for this frame to the current time.

    time2 = time()

    # Check if the difference between the previous and this frame time > 0 to avoid division by zero.

    if (time2 - time1) > 0:

        # Calculate the number of frames per second.

        frames_per_second = 1.0 / (time2 - time1)

        # Write the calculated number of frames per second on the frame.

        cv2.putText(frame, 'FPS: {}'.format(int(frames_per_second)), (10, 30),cv2.FONT_HERSHEY_PLAIN, 2, (0, 255, 0), 3)

    # Update the previous frame time to this frame time.

    # As this frame will become previous frame in next iteration.

    time1 = time2

    #----------------------------------------------------------------------------------------------------------------------

    # Display the frame.            

    cv2.imshow('Subway Surfers with Pose Detection', frame)

    # Wait for 1ms. If a a key is pressed, retreive the ASCII code of the key.

    k = cv2.waitKey(1) & 0xFF    

    # Check if 'ESC' is pressed and break the loop.

    if(k == 27):

        break

# Release the VideoCapture Object and close the windows.                  

camera_video.release()

cv2.destroyAllWindows()

We complie the entire thing with all functions defined, and we get the code to be working. Now we need to open Subway Surfers, the only safe website to do so is on Poki.com which you can do here

The Final code is all of this put together. You can find it here

The project aims to incorporate more physical movement into computer games by utilizing Python and OpenCV. The approach is to leverage the libraries and frameworks provided by these technologies to develop interactive, motion-based gameplay mechanics. This can include using OpenCV's image-processing capabilities to track the movement of the player's body, or utilizing Python's scripting capabilities to integrate motion-sensing input devices such as accelerometers or gyroscopes into the game's control scheme. Additionally, machine learning and computer vision techniques like object detection and gesture recognition can be used to enable more intuitive and immersive physical interactions within the game. Ultimately, the goal is to enhance the physicality of computer games and create more engaging and immersive experiences for players by creatively leveraging the capabilities of Python and OpenCV. 

The real-life applications of this project could be in the field of virtual reality, fitness, and rehabilitation. Incorporating more physical movement into computer games utilizing Python and OpenCV can provide a more engaging and immersive experience for players in virtual reality. This could be used for fitness training and rehabilitation, where patients can engage in physical therapy in a virtual environment. This approach could also be used in the gaming industry to create more engaging and immersive games that require physical movement. Additionally, using machine learning and computer vision techniques, such as object detection and gesture recognition, could be used to enable more intuitive and immersive physical interactions within the game. This could be used in other industries such as manufacturing and construction where gesture recognition could be used to control machines or robots. Overall, the application of this project can be used in many different fields to provide a more engaging and physical experience for users. 

Personally, I would like to take this project and put it into a 3D game that isn't just one thing. I think this will be big for VR, as these cameras are fairly accurate. I will try to implement this code to work with the popular game ROBLOX or something.

OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision. In our application, it helps us grab input from our camera and when paired with other libraries like mediapipe and pyautogui, we can turn camera input into output.

We chose Python because it is by far one of the easiest languages to code in. Some of our libraries, like PyAutoGui, are (Py)thon based, so it would be easier since the other libraries also supported Python. We could write this code using alternative libraries in other languages, however, it would bring unnecessary complications. 

Our project has one main limitation:

• Delay: The camera input (when the person performs the movement) is delayed by a fraction of a second. This means when you move right in real life, it takes around .5 seconds to move the character in the game. This can be solved with better computer GPU and CPU specs, or we can write it in another language that does not take as long to compile. After playing for a few rounds, the user begins to get the hang of the game, making the delay not too intrusive of the gameplay. 

One of the main considerations when making this code, was to build it in a sense to be 'future proof' meaning we can edit the code easily and make it work for other applications that use similar arrow keys or WASD systems. After some further testing, with minimal edits, we actually got this program to work for 3D games like ROBLOX and vehicle driving simulators.