Milestone 2

Project Plan

Phase 1

  1. Build the testing hardware (rudimentary screen and mount)

  2. Make code to load custom objects and move around viewing area

  3. Create new or improve existing hand tracking software

Phase 2

  1. Integrate the hand tracking software with rendering program to move objects (this is the main function of the project)

Phase 3

  1. Add more functionality to improve speed, viewing angle, comfort, and style

Phase 4

  1. Market and produce in a repeatable and efficient manner

Design Concept

Hardware

There are two primary primary physical technologies involved in this project.

Headset

Custom hardware to hold Samsung Galaxy S9 to reflect images to mirror and then on to tinted glass.

Hand Tracking

Hand tracking glove to help with speed and accuracy of the positioning to move virtual models.

Software

There are two major software components to this project.

Hand Tracking

Software hand tracking from camera to also help with speed and accuracy and provide visuals.

Object Projection

Object rendering in virtual space with high accuracy and render speeds

3d Model Export

Import custom files from SolidWorks or other 3d rendering and modeling software

Design Selection

Although it is still subject to be improved and possibly redesigned, the final concept is as follows:

Headset

For the headset, we will be using a housing which can reflect light from an android smartphone on to a translucent screen. We will not be focusing on the wide angle field of view, so we will make the housing eliminate as much outside light as possible so that the reflection from the phone is able to be seen easier. The headset will have a wireless connection to a desktop or laptop to help with computing power, and it will make use of the smartphone's onboard sensors for positioning and rotation.


Hand Tracking

We will make use of both a camera and motion capture gloves already on the market to make sure we have the most accurate hand tracking possible. As needed, we will modify or add on to the hardware to make it more efficient.


Viewing

We will be importing models from Solidworks to render, and have all options be software oriented for the time being. As we progress it may change to use a physical keyboard for some selection, but we will for now use software to select and move visuals, with a very basic user interface.

System Model

Process Model

Analysis

Hardware

Headset

  • 3d printed chasis PLA

  • Samsung Galaxy S9 monitor and processor

  • Mirror for reflected image

  • Tinted glass lens for viewing

  • Head strap to secure

  • Accelerometer and gyroscope modules

Gloves

  • Accelerometer and gyroscope modules

  • Motion capture glove

Software

Image Manipulation

  • Import pre loaded objects

  • Move according to hand tracking

Tracking System

  • Track hands from starting position based on accelerometer data

  • Tracks palm up or palm down based on gyroscope

  • Senses the shape of hand for selection and object manipulation

Testing Plan

Test 1

Load in a 3d file to android app and have it move rotate and scale

Test 2

Collect data from hand tracking and display it as text

Test 3

Connect movements from test 1 to correspond with data from test 2