To extract human hand motion from the camera, I utilized the MediaPipe AI model, which provides the coordinates of human hand landmarks as pixel coordinates. I then used these pixel coordinates to calculate the relative X, Y, and Z coordinates in the real world. This machine vision and AI aspect of the project was entirely executed within the PyCharm IDE using Python. Unfortunately, due to limitations with my computer, I couldn't run the entire algorithm in Matlab, even though Matlab offers some image processing functions. 

The most challenging part was calculating depth values from the X and Y plane, as MediaPipe only provides X and Y coordinates since it processes a 2D image of our hand. I had to devise a method to derive depth values from these coordinates, enabling me to calculate the necessary rotations for each finger. This was accomplished through the following calculations, which I developed 

was able to create a 3D model in SolidWorks, thanks to the strong foundation provided by my university. I then imported this model into Matlab using the Simscape library, made necessary adjustments to the robot hand's joints, and added constraints and angles to the design within Matlab. This allowed me to input the joint angles and observe the actual hand's movement. 

I ran my machine vision program in PyCharm IDE using Python, while I used Matlab to simulate the robot hand. This approach allowed me to run the program on my computer without encountering performance issues. To establish communication between PyCharm and Matlab, I utilized a virtual serial port. Through this serial port, I could send and receive data between both IDEs, enabling me to simulate the robot within the Matlab environment.