Deliverable 1: Data Collection and Curation

• Record and upload videos of people rock climbing

• Curate and crop the videos of people rock climbing to make a dataset.

Deliverable 2: Percentage completion (climb report aspect)

• Output lowest and highest holds in the image of the wall (in terms of the center coordinates)

• Output the final position of the hands of the climber in the video (center coordinates)

• Calculate and report the percentage of the climb completed

Deliverable 3: Hold identification and grouping (by color)

• Evaluate and document results of noise reduction techniques for image preprocessing

• Evaluate and document results of Neural Network for hold detection approach

• Evaluate and document results of classical Computer Vision for hold detection approach

• Evaluate and document results of Neural Network for hold detection approach in combination with image preprocessing

• Evaluate and document results of Computer Vision for hold detection approach in

combination with image preprocessing

• Evaluate and document results of method of color grouping (1)

• Evaluate and document results of method of color grouping (2)

Deliverable 4: Hold/Position integration function - Sequence of holds used (climb report aspect)

• Define and document input/outputs with team 2

• Write and document function

Deliverable 5: Check if move is valid (climb report aspect)

• Create and document function with logic for checking move validity

Deliverable 6: Integration and testing

• Integrate and document with team 2

• Generate climb report output

• Test and report results of evaluation

Deliverable 7: Collect data for testing

• Collect images and videos of people rock climbing and of rock climbing walls

• Curate and crop the images and videos to make a testing dataset

Deliverable 8: Evaluate and Improve Robustness Hold detection

• Determine methods for evaluating hold detection and color grouping, and document results

• Determine method for evaluating percentage completed and document results

• Determine method for evaluating move validity and document results

• Try methods to improve accuracy of hold detection and color grouping, and document

results

• Try method to improve accuracy of percentage completed and document results

Deliverable 1: Data Collection and Curation

• Record and upload videos of people rock climbing

• Curate and crop the videos of people rock climbing to make a dataset.

Deliverable 2: Define and output time instances to get coordinates of hands, legs, and hips positions

• Research and formulate the logic for computing the time instances

• Define and output the time instances in a format to be used by subsequent deliverables

Deliverable 3: Get the coordinates of hands, legs, and hips positions at specific instances of time

• Preprocessing of the input data to make its format compatible with pose estimation

algorithm(s)

• Using the predefined time instances, output the coordinates for hands/legs/hips - O/P format: {time1: (hand_x, hand_y, leg_x, leg_y, hips_x, hips_y)}

Deliverable 4: Distance center of climber (hips) moved overall (climb report aspect)

• Sum of each consecutive jump (difference between hips position from one point to another)

• Compute the total hips movement

Deliverable 5: Time elapsed for entire climb (climb report aspect)

• Determine the start and end position of the climber based on initial and final hold position(s) and hand position(s)

• Define the threshold for detecting whether arms and legs are at a hold

Deliverable 6: Number of moves taken (climb report aspect)

• Determine the change in coordinates of hands, legs and hips at each interval and increment the count

• Return total count as the number of moves taken during entire climb

Deliverable 7: Integration and testing

• Integrate and document with team 2

• Generate climb report output

• Test and report results of evaluation

Deliverable 8: Evaluate and Improve Robustness of Pose Estimation

• Determine and try methods for evaluating pose landmarks other than MediaPipe.

• Determine and try other algorithms for computing time instances more accurately.