SM64 Real HUD

You don't have to stop just at the HUD; other in-game events can trigger other things in real life. For e.g., what if you had to unlock the doors in SM64's castle to unlock doors in real life! As long as you can get the trigger values from SM64 and can adapt the hardware and software there are loads of possibilities. With the right modifications you could even use real world events to trigger things in SM64. You could also use triggers from other software that isn't even a game. Lastly, the electronic HUD can be used as a general purpose display for other projects or for testing.

As an overview, Project64 runs an emulated version of Super Mario 64 while a piece of software on the PC grabs the HUD values from the game by reading certain memory values. These values are pushed to the PC's serial port via USB and then are picked up by an Arduino which then updates the attached LED displays.

This project demonstrates a number of techniques which can be reused in other projects; for one thing, it shows how a program running on Windows can extract values from another piece of software that was never designed to interact with another program in that way. Additionally, the project shows how software can push values to a serial port that other hardware can then pick up and use. Lastly, the project looks at driving multiple displays based on values from a serial port.

You can watch a video I did of this project:

Displays

The HUD in SM64 displays Mario's current lives, coins, stars, health and camera options. To simplify things in this version the camera options are ignored. The lives is limited to 100, the coins maximum value is 999, the stars count usually reaches no more than 120 and the power display is made up of 8 segments.

In the early stages I had an Arduino Uno directly update a single 7-segment LED display based on Mario's current coin total. I found that when collecting coins the LED display updated exactly in time. However, when it came to driving 2 displays I came across problems with multiplexing since I couldn't get the timing right which resulted in the digits changing randomly. It seems that the Arduino's Serial.parseInt function was the problem as it is slow and waits until it gets an int from the serial port. I did try to parse the values myself and even adapted the timing of the displays to make up for the delay but I still couldn't get a solid display.

I then thought about using shift registers which not only would fix the display issue but would also greatly reduce the number of I/O needed to the Arduino. I happened to have a 5451 LED driver IC which has 35 outputs and built-in brightness control which removes the need to have limiting resistors on the LED display segments. The 5451 is updated with just 2 control signals (data and clock) but on the downside it can only drive common anode displays. Here is the link to the datasheet:

http://www.micrel.com/_PDF/mm5450.pdf

To update enough displays and LED's for the HUD two 5451's would not be enough-71 outputs are needed. However, as lives is limited to 100 and the normal star total is 120, the total number of outputs is 61 which can be achieved with 2 5451's. This means one 5451 can drive the power display and the coins display (which can display 0 to 999) and the other 5451 can handle the lives and stars displays (which only need to show not much over 100). So, we only need segments b and c of the left hand digits of the lives and stars displays.

By connecting the data inputs of the 2 5451's together we select the 5451 to update by using its individual clock. This means we only need 3 I/O lines from the Arduino to update all the outputs. This is the big advantage of using shift register type IC's but on the downside there are a lot of connections between the displays and the IC's, something that isn't so much a problem when using multiplexed displays.

Prototyping on breadboard was a big task but I was sure to test each display in turn rather than connecting everything and then have lots of problems. For the power meter I just used another 7 segment display but for a more finished version you could use individual LED's arranged in a circular manner. Note that multi digit displays are usually multiplexed but because the circuit doesn't use multiplexing you will need to use individual LED displays.

Below you can see the breadboard version of the HUD in action:

Project64 memory access

Unlike older computers in which it was fairly easy to access data in RAM and the programs used fixed addresses for variables, modern computers have protected RAM and variable addresses can be different each time a program runs (because of dynamic allocation). We can however access a program's data in RAM using API's of which programs such as HxD use. HxD is a great, free disk editor/memory viewer that helped me get this project going. Note that you may have to run HxD as an administrator to be able to open up Project64's RAM.

So, I made use of HxD to find where Mario's coin total is stored in RAM by first searching for a unique hex string and then calculating the offset which in turn led me to the coin total variable. With this address I was then able to get the current coin total and forward it to the Arduino. However, when I next switched  on my computer the program no longer worked because the coin total address had changed (because of dynamic memory allocation).

To get round the problem I looked for a pointer to the start of the emulated N64's RAM and that pointer, it turns out. is always at the same address. This way it is a matter of getting the value of that pointer which tells you the start address of the N64's RAM to which you can add an offset to get whatever variables you need.

The value of the coins, lives, etc are then sent to the serial port (in text form) that the Arduino is connected to by using a C++ program I wrote. Sending just one value wasn't a problem but when it came to sending multiple values the process memory reader program froze up. It turned out the problem was that the .NET framework serial port component is set to infinite timeout, but lowering the timeout value the did not help. What seems to have fixed it is to only write values to the serial port if one of the values (coins, lives, etc.) changes which lowers the frequency of writing to the serial port.

The values are written to the serial port in the following order:

health,coins,lives,stars,

The ',' are important because they separate the values; the last one ensures the last value is interpreted as a number immediately otherwise the Arduino will wait.

Circuit diagram

The circuit is quite simple and can be viewed below:

I arranged the LED displays so that no single display is split across the two 5451 IC's. As you can see, the LED display segments have no limiting resistors as the LED current is set by R1 and R2 for each of the two 5451 chips. According to the datasheet the current to each LED segment is roughly x 20 the current entering the BRI pin of a 5451. To give you some real world values I measured:

18 segs on (6 segs IC1, 12 segs IC2):

220mA total current draw

IC1 current draw: 3mA

IC2 current draw: 4.4mA

IC1 BRI current: 0.56mA

IC2 BRI current: 0.35mA

Current to single segment connected to IC1: 18.6mA

Current to single segment connected to IC2: 10.2mA

Oddly, it seems the total current flow is a lot less than it should be and this is without even considering the Arduino (which measuring its current draw is more difficult). However, by measuring the current flow through different LED segments I found some variation so it we were to add up each individual LED segment current the total would be less that the total current draw.

We can see that the LED segment current drops the more segments that are on (~18mA for 6 segments vs ~10mA for 12 segments) as well as varying slightly for each segment. While you may not notice a difference in segment brightness for LED's driven by a single 5451 you may detect a difference between two 5451 IC's.

Software

There are two pieces of software that are needed for this project; the code for the Arduino which takes the values from the PC serial port and updates the displays and the software for the PC which gets the values from Project64 and passes them to the computer's serial port. The 

Arduino software is attached to the bottom of this page and is called 'SM64_Real_HUD.ino' - this should be downloaded to the Arduino.

There are only three signals we need to update the displays:

LED_DIS_CLK_1    5451 no.1 clock input

LED_DIS_DATA     5451 no.1 and no.2 data input

LED_DIS_CLK_2    5451 no.2 clock input

These are set as outputs and put to their default values in the setup function. This is where we also init the serial port and update all the displays using updateHUD(), so that they show the default HUD values at start up.

During the loop function we continually check the serial port and if there are enough bytes we try to convert the values from the serial port (which are in text form) to numbers and store the converted values into the HUD variables (power, coins, lives and stars). The updateHUD function is called again, which writes to both 5451 LED drivers in turn. To update a 5451 IC you must first send it a start bit and then 35 more bits to turn on or off each segment. The outputBargraph function, which is called from updateHUD(), lights the power segments by sending a number of logic 1's based on the current power value. As for outputNum3(), also called from updateHUD(), this function lights a group of three 7 segment displays based on a value. It works out the 1's, 10's and 100's values and sends them to outputNum() which outputs a number to a 7 segment display using the segment bit values from the look-up table charValues[].

Every time we send a bit to a 5451 we must send it a clock pulse which is handled by the clk function. As we have two clocks, one for each 5451, we must specify to clk() which clock to use.

Next is the program that runs on the PC which I wrote in C++ using Visual Studio 2010; the project is attached to the bottom of this page as 'processMemoryReader.zip'. The project file is called 'processMemoryReader.sln' and the two programming files are 'processMemoryReader.cpp' and 'Form1.h'.

The components, which you can view in Design view, are: the form (Form1), a status label (lblstatus), a serial port (serialPort), a timer (mainTimer), a group box (regionGroupBox) and the radio buttons for PAL (PAL_RadioButton) and NTSC (NTSC_RadioButton). The Interval property of the timer is set to 10 so that the displays are updated quick enough; any higher value and there may be a delay between changes in the game and the displays.

When Form1 opens it calls a setup function which looks for a window called 'Super Mario 64 - Project64 version 1.6' and if it finds it it gets its process ID. When we have the process ID we can open the Project64 process and then read from its memory; this handle to the process must be released when finished with which is dealt with in the form's FormClosed event.

Because memory requested by a process will be at a different address each time we need a way to easily find the start of the emulated N64's RAM. Fortunately Project64 has a pointer to the RAM which is always located at the same address in memory and so in the setup() function we get that pointer using the value of baseAddrPointer.which in turn gives us the actual start address of the N64's RAM of which we store in startRAM_addr.

Also in setup() we try to open the serial port and change its write timeout value to 1 instead of the default of infinite which can cause a hang if there is trouble writing to the serial port. To change which serial port is used you will need to alter the PortName value of the SerialPort component.

In the timer's Tick event we call the function getSendValues which firstly gets the correct HUD variable offsets based on the PAL/NTSC selection. These offset values are stored in offsetValuesPAL[] for the PAL game and offsetValuesNTSC[] for the NTSC game and have been adjusted for the differences in endianness between the N64 and PC. The N64 stores data in Big-endian format but the emulated version using Project64 on a PC stores data in Little-endian format and so each set of 32 bits are reversed which results in slightly different address values. So, for e.g. for the PAL game Mario's health is stored at 003094DE (803094DE on a real N64; the '80' can be treated as 00) but with raw access to Project64 the health address is 003094DD. This may seem confusing as cheat codes stay the same between a real N64 and the emulated version but when we access Project64's memory directly we are seeing it in a different format.

In getSendValues() we get each of the HUD values from Project64 by adding the appropriate offset value to startRAM_addr and these HUD values are stored in temporary variables. If any of these new values have changed since last time then we send all of the HUD values to the serial port. It's important to send the values to the serial port as infrequently as possible otherwise you may get exceptions because the serial port can't keep up. Note that we send the HUD values to the serial port in text form with each value separated with a comma, including the last value, so that the Arduino can quickly interpret the values as numbers. You could send the values as raw bytes but I found it easier to send them in text form, which is also easier to debug as when testing I sent test string manually using the Arduino IDE's built-in serial monitor.

In this prototype form there is only the basic error checking for such circumstances as the serial port not being available or if Project64 has not been opened. However, there are a number of things that could be added to improve, such as being able to resume after an error without having to close the process memory program and open again.

Using the HUD

When you have the sketch downloaded to the Arduino (see Software section) and the circuit built and connected to the Arduino, when you connect the Arduino to your PC after a few seconds you should see the coins, lives and stars set to 0 and none of the power segments on. It would be a good idea to use the Arduino IDE's serial port monitor to test some values. For e.g., send:

8,563,4,101,

to turn all power segments on, set coins to 563, lives to 4 and stars to 101.

Try some other values to make sure all segments are working and then after testing close the Arduino IDE. Next open Project64, load the SM64 ROM and load a save file, save state or start a new game. In Visual studio load the process memory reader project and run it; a window should appear (if you are using the PAL version of the game select the PAL option). The HUD displays should show the current HUD values and change as you collect coins, lose some health and so on. Note that the game doesn't clear the coin counter when you enter a non-enemy level - the HUD displays can be quite revealing of how the game works.

If there are any problems you will need to stop the process memory reader program, fix the issue and run the program again. A possible error can come about because of a program already accessing the serial port the Arduino uses. You may also run into difficulties if you ran the process memory reader program before opening Project64.

All content of this and related pages is copyright (c) James S. 2015-2022