DIY 3D Virtual Reality - HMD using BBC Microbits to track head and object orientation

Page updated 30/12/2018

Featured on Hackaday!

Link to Hackaday article

Summary

This project outlines my desire to build a relatively comprehensive 3D Virtual Reality system which gives a similar immersive experience to commercial offerings available 20-30 years ago. This system I built consists of a head mounted display (HMD) that is able to track 3D head orientations such as heading, pitch and bank,and a separate 2D VR hand controller (pitch and bank) which is used to independently control another object in the virtual scene (e.g. a weapon).

Any hardware bought to aid the project (e.g. microcontrollers) had to be very low cost and accessible to buy from any major online source. The whole project cost was around 60 GPB (UK).

Introduction

For many years I've had an interest in 3D Virtual Reality and always wanted to build my own homebrew head tracking system. I wanted to be involved with building all parts of the VR system including hardware needs, electronic requirements, microcontrollers, data flow (serial) and the programming of the 3D visual system itself. I wanted to understand everything necessary to create a similar immersive experience to the first arcade machine Dactyl Nightmare (by Virtuality Group) I was fascinated by back in the 90s,

Recently I found the time to make a this project a reality and finally managed to create a reasonably good head mounted display (HMD) with the separate VR controller The HMD enables me to look around a 3D scene and the secondary VR controller allows me to swing, move or aim a weapon too! I can move independently using the keyboard cursor keys and use the A-Z keys to fly up or down! With all this capability the VR experience is quite pleasing and with the enhancement of true stereoscopic depth vision the scene its quite a good immersive experience!

Parts

2x BBC Microbits. One is v1.3 and the other v1.5 (updated version with a single chip combining the accelerometer and compass). Cost 30 GBP each

1x AA battery pack for wireless hand tracking using the BBC radio feature.No extra cost

1x 5" LCD HDMI display - the screen for the HMD. Cost 20 GBP

(5-inch Resistive Touch Screen LCD Display HDMI for Raspberry Pi XPT2046 IL)

1x Google cardboard - used as a housing for the lenses and LCD display

(5.0" Google Cardboard Valencia Quality VR 3D Virtual Reality Glasses + NFC A6G8) Cost 2.99 GBP

1x head strapping - to support HMD housing, LCD and head orientation microcontroller on the head.

(Head strap mount For Go Pro SJ5000 Sport Camera). Cost 2.50 GBP

1x USB cable. No extra cost

1x HDMI cable. No extra cost

Software

Mu (Python)

Used to program the microcontrollers. Heading, pitch and bank orientations are captured from each microcontroller and sent to the PC via USB as realtime serial data.

Download Mu

Visual Basic 6 (VB6)

Made a utility to capture the serial data from both microcontrollers. VB6 converts the captured serial data and turns it into a format that Blitz3D can use to help orientate the cameras, view and objects in the virtual scene.

Blitz3D (fully free game engine)

Programmed the 3D engine to create the 3D virtual stereoscopic scene. Viewer orientation and movements within the 3D virtual scene are derived from the serial data provided by the Microbits through the visual basic utility.

Download Blitz3D

Microbits

The project uses 2 Microbits for 3D head and hand orientation tracking. One Microbit is connected to my laptop with a USB cable. This has two purposes; 1) to power the Microbit 2) send the combined tracking data of both Microbits to the laptop for processing via USB (via the Mbed virtual serial driver).

Mbed virtual serial port driver

The second Microbit is self powered using batteries. It sends its compass, pitch and bank data to Microbit 1 using the Radio feature. This data is appended to the data captured on Microbit 1.

The data sent from each Microbit is a string of numbers relating to each compass heading, pitch and bank. An example string of Microbit 1 data sent from Visual Basic to Blitz3D

Output:

X120Y34.6Z-24A43B-32C13X121Y34.6Z-27A41B-37C14

XYZ is the compass, pitch and bank for Microbit 1

ABC is the compass, pitch and bank for Microbit 2 (compass not used)

The Visual Basic utility read this stream of data and separates the XYZABC lines into individual chunks.

A text file is output to Blitz3D similar to the following:

Output:

X120

Y34.6

Z-24

A43

B-32

C13

Blitz3D reads each line and interprets each value as orientation to the 3D cameras or weapon object in the scene. As the stream is constant the screen update is in realtime.

*A quick difference between version 1.3 and newer version 1.5 Microbits

For this project I have two Microbits. One is a version 1.3 device and the other is a 1.5 version.

The later version of the Microbit has a combined compass and accelerometer chip. I found the same code for the v1.3 device didn't work for reading the sensors on the 1.5 version. A fix I found was to simply download the latest updated version of Mu (1.0.1). The fix is documented below

micro:bit motion sensor hardware changes for editor developers

Read about the micro:bit motion sensor revision

HMD setup

I'm using a low cost Google Cardboard as a housing for the LCD display. I need this to hold the LCD display and for the lenses.A couple of notches were cut out for the cable connectors and a header on the back of the display.

A 3D scene has been created in Blitz 3D and made into a stereo pair. The laptop screen is being duplicated and projected across to the 5" display. The size of the display is similar to a mobile phone that fits into the cardboard housing.

The LDC display fits perfectly into the Google Cardboard housing.

Testing for comfort and fit before attaching cables.

With power attached to the LCD its ready for stereo viewing

The head strapping is wrapped around the housing and ready to wear!

Viewing the 3D scene through one of the lenses!

Constant refinement

Testing positioning of Microbits for head tracking. Best position was on top of head strap. Used tie-wraps to hold the HMD in place during initial prototyping. Ouch!

Testing weapon code and hand tracking in Blitz3D. Using a long cuboid as the weapon.