Build the testing hardware (rudimentary screen and mount)
Make code to load custom objects and move around viewing area
Create new or improve existing hand tracking software
Integrate the hand tracking software with rendering program to move objects (this is the main function of the project)
Add more functionality to improve speed, viewing angle, comfort, and style
Market and produce in a repeatable and efficient manner
There are two primary primary physical technologies involved in this project.
Custom hardware to hold Samsung Galaxy S9 to reflect images to mirror and then on to tinted glass.
Hand tracking glove to help with speed and accuracy of the positioning to move virtual models.
There are two major software components to this project.
Software hand tracking from camera to also help with speed and accuracy and provide visuals.
Object rendering in virtual space with high accuracy and render speeds
Import custom files from SolidWorks or other 3d rendering and modeling software
Although it is still subject to be improved and possibly redesigned, the final concept is as follows:
For the headset, we will be using a housing which can reflect light from an android smartphone on to a translucent screen. We will not be focusing on the wide angle field of view, so we will make the housing eliminate as much outside light as possible so that the reflection from the phone is able to be seen easier. The headset will have a wireless connection to a desktop or laptop to help with computing power, and it will make use of the smartphone's onboard sensors for positioning and rotation.
We will make use of both a camera and motion capture gloves already on the market to make sure we have the most accurate hand tracking possible. As needed, we will modify or add on to the hardware to make it more efficient.
We will be importing models from Solidworks to render, and have all options be software oriented for the time being. As we progress it may change to use a physical keyboard for some selection, but we will for now use software to select and move visuals, with a very basic user interface.
3d printed chasis PLA
Samsung Galaxy S9 monitor and processor
Mirror for reflected image
Tinted glass lens for viewing
Head strap to secure
Accelerometer and gyroscope modules
Accelerometer and gyroscope modules
Motion capture glove
Import pre loaded objects
Move according to hand tracking
Track hands from starting position based on accelerometer data
Tracks palm up or palm down based on gyroscope
Senses the shape of hand for selection and object manipulation
Load in a 3d file to android app and have it move rotate and scale
Collect data from hand tracking and display it as text
Connect movements from test 1 to correspond with data from test 2