Heads Up Witness Camera
I've now competed in the Sparkfun Autonomous Vehicle Competition (AVC) for two years running and, each year, I come away with questions as to exactly what was going through my car's computer brain when it performed some unexpected maneuver. Even though I typically log data from each run to an onboard SD card, the data is often hard to interpret because it's just stream of out-of-context numbers. What I really want is to do is watch a video playback from the POV of the car which also shows various important bits of information, such as compass heading, speed, next waypoint, etc. overlaid on top of and time synchronized to the video. So, I set out to see just how complex, or easy it might be to put such a system together and make it small enough to fit into my autonomous car, Johnny Five.
I started by purchasing a few items off eBay, such as a very compact, NTSC color video camera module that can run from 3.3 to 5 volts of DC power. The module I selected cost about $23 and measures less that 1 inch by 1 inch. Here's a picture so you can see how small it is:
Next, I purchased compact digital video recorder (DVR) for about $53 at Hobby King that records to SD cards up to 32 GB in size. The DVR can directly record an NTSC (or PAL) video steam and can be controlled by a small IR remote (included), or configured to automatically start recording on power up. Here's a picture of the DVR unit:
The DVR also has a video output and uses on screen setup menus to configure it, so I decided to splurge and get a small LCD video display from Adafruit for $50 to use as a monitor during development. However, you can use the composite video input on almost any TV for the same purpose. Here's the monitor setup on my lab workbench:
Finally, to overlay text and graphics onto the video stream, I purchased Video Experimenter Shield (kit) from Nootropic Designs for $25. Here's a picture of the Video Experimenter Shield fully assembled:
The Video Experimenter kit was easy to assemble, as it uses all through hole parts and has a very simple circuit design. It's designed to work with a modified version of the Arduino TVOut library that has been modified to sync to the incoming video signal and overlay white pixels on a small, 136x96 pixel grid. The code can do simple graphics, such as lines, circles and rectangle and can also write text in a few fonts sizes. To get an idea of how all this works, here's a short video I shot shortly after hooking up all the components and using one the demo programs to generate a video overlay:
and, here's a sample of the actual video, as captured and recorded by the small, SD card DVR (Note: there are some artifacts in this video that seem to result from the upload to YouTube due to the conversion from the DVR's native ASF format. I do not see these artifacts when playing back the video using the VLC Video Player on my Mac.) Based on my tests, the DVR uses about 16 megs to record one minute of video, so a 32 GB SD card (about $20 on Amazon for a name brand car) should hold over 30 hours of video, if my calculations are correct.
So, the total cost of this project, so far, is $151, but would cost only $100 if you left off the LCD monitor.
My Custom Video Overlay Board
The whole thing worked so well, I decided to make it more compact so it could fit easily into my car. I started by taking the circuit from the Video Experimenter, which is a simple video sync extractor based on the LM1881 chip, and combined it with a ATmega328P (in a TQFP32 package) using a circuit like this:
I then designed a simple PCB to contain this circuit and also provide a place to mount the camera module. The result PCB is only 1.3 x 1.3 inches in size. The board is currently still being fabled at OSH Park, but the layout looks like this:
and, the backside, where the camera module attaches, looks like this:
I hope to have a further report, with actual data from field tests once I get the final PCB assembled and figure out how to place all the components into by autonomous car, Johnny Five. Stay tuned for further developments.
First Assembled PCB
I received the first PCBs back from OSH Park and, after a few struggles, managed to get one assembled and working. The PCB I picked had an open connection on one of the through hole vias that prevented the RESET line from working and which, in turn, made it impossible to program in the boot loader. Turns out this problem only afflicted the one PCB (of the 3 I received from OSH Park) I picked to assemble. That's typical luck for me, but I was able to work around the problem by adding a bit of trusty Kapton-insulated wire wrap wire as a jumper. Also, I screwed up several things on the design (holes too small for the JST and FTDI connectors) so I plan to remake the board before calling this a final design. However, here's a photo of the patched, but working PCB:
Click for a larger view
Update - Finished Board, Rev2
Revised PCB Front and Rear View
Parts List for Overlay Camera PCB
Qty Part # Mouser Part Each
1 ATMEGA328P-AN 556-ATMEGA328P-AN $3.28
7 C0603C104Z4VACTU 0-C0603C104Z4V $ .01
2 C0603C180J5GACTU 8 0-C0603C180J5G $ .02
1 LM1881MX/NOPB 926-LM1881MX/NOPB $2.94
1 161-2505-E 161-2505-E $ .69
1 1N4148W-E3-08 78-1N4148W-E3-08 $ .10
1 7A-16.000MAAJ-T 717-7A-16.000MAAJ-T $ .94
Part # Digi-Key Part Each
1 S3B-PH-K-S 455-1720-ND $ .17
1 PHR-3 455-1126-ND $ .09
10 SPH-002T-P0.5L 455-2148-1-ND $ .50
Controlling the Recorder
With the camera nearly ready to go, the next step was to figure out how to start and stop the SD Card Recorder once it was mounted inside Johnny Five. The recorder was designed to be operated with an infrared remote that controlled on screen menus visible on a monitor attached to the video out connector. However, even though I purchased a small LCD monitor, I decided that using the monitor and IR remote to control the decoder was not going to be practical in an outdoor setting and would just add unwanted drama at race time. One option was to configure the recorder to begin recording when power was supplied to the recorder, but taking this approach would require adding a power control circuit, which seemed overly complex. I wondered if there was a better way. With a little bit of Internet sleuthing, I was able to find a document that described some of the wiring inside the recorder and, most usefully, described some unused, bonus features I could tap into with a little bit of rewiring. The original document I based this on can be found here, but here's an extracted diagram that shows the relevant parts:
Click for a larger view
There are several interesting things described in this diagram. First, it shows an unused set of pins where a USB cable could be connected. I didn't try connecting anything to these pins, but I suspect this might make it possible to use the recorder as a USB drive and directly access the SD card. Another note indicates that the battery is "used for interruption of power supply" which is good to know, as I was not sure about the purpose of the battery. I was also pleased to see that the suggested voltage range for the power input was "+5 to +30", as this meant I could power the recorder either from the +5 volt output from the ESC, or directly from the 7.5 volt battery pack.
However, another notation near the bottom of the page my full attention. One of the pins is labelled "TXD/Alarm-In" and a note pointing to this pin reads "It'll record when grounding the point (Alarm_in) and stop recording when undo." This seemed like a really convenient way to control the recorder, but I would need to to find a convenient way to route this signal out to the exterior of the recorder. Since I did not plan on recording audio, I decided to repurpose the Audio-In pin which is on the same connector as the Video-In signal. Doing this requires moving only a single wire from one position on the recorder's main PCB to another, as shown in this annotated image:
Click for a larger view
Well, That's a Big Fail...
However, after making the "open pin recording" mod I described above, I made several attempts to get it to work, but with no luck. I'm not sure if this is because of some detail I missed, or if the SD recorder I purchased is different from the one described in the document I found but, after spending considerable time on trying to make it work, I could not. If anyone with a similar recorder manages to get this approach to work, please let me know.
Before returing to the idea of using the record on power on feature to start record, I remembered that there was another diagram in the same document that described how to build a wired keyboard to mimic the functions provided on the IR remote. The approach used a resistor ladder and looks like this:
Click for a larger view
From the resistor diagram I could see that connecting a resistor chain containing the values 33 + 3K + 3.9K + 6K + 11K (23.93K) from pin ADC_KEY1 to ground was supposed to be equivalent to pressing the OK button the IR remote, which is the button that put it into Record mode. Likewise, connecting the values 33 + 3K (3.03K) from ADC_KEY2 to ground would STOP the recording. The following annotated photo shows the places where these connections need to be made:
Click for larger view
Fortunately, when I wired up this approach using a 22K resistor for the OK (Record) signal on ADC_KEY1 and a 3K resistor for the STOP signal on ADC_KEY2, it worked! While not as convenient as having a single line to control both starting and stopping the recorder, having two line to start and stop record is better than having to control the power supplied to the recorder, as this method provides to clean way to stop recording other than to cut the power. As a test, I wired up a pair of pushbutton with approximate resistor values (27K and 2.7K.) The following video demonstrates the successful result:
Of course, I still need two control lines to make this work and I don't really want to have to mod the nice aluminum case, or take over the audio output line on the other connector. So, after trying out a few approaches, I settled on the following circuit, which I believe I can tuck inside recorder between the two circuit cards:
Note: in a prior version of this diagram, the +/- inputs on each comparator were swapped
The circuit uses a dual, open drain output comparators to detect when the input line is pulled high, or low. When INPUT is pulled low, the upper comparator's output is driven low, which operates the OK/RECORD key. Likewise, when INPUT is pulled high, the lower comparator's output is driven low, which operates the STOP key. When INPUT is left to float, it's pulled to 1/2 Vcc so that neither comparator's output is driven low and so no key is pressed. The circuit will need to be powered by roughly 5 volts to operate reliably with 5 volt logic levels on the input pin. However, I can easily adjust the resistor values to support other combinations of logic and power (Vcc) voltages. The whole circuit fit nicely onto a PCB roughly .575 x .400 inches, which only costs $1.15 to fab at OSH Park (honestly, how can he do that so cheaply...)
Then, to operate the circuit and control the SD Recorder, I'll use code like this in Johnny Five's Arduino Mega controller:
#define INP_PIN ??
void setup () {
pinMode(INP_PIN, INPUT);
}
void startRecord () {
digitalWrite(INP_PIN, LOW);
pinMode(INP_PIN, OUTPUT);
delay(500);
pinMode(INP_PIN, INPUT);
}
void stopRecord () {
digitalWrite(INP_PIN, HIGH);
pinMode(INP_PIN, OUTPUT);
delay(500);
pinMode(INP_PIN, INPUT);
digitalWrite(INP_PIN, LOW);
}
The actual code I'll probably use in Johnny Five will be slightly different, as I won't want the CPU in a 500 ms wait loop at the start of he race. However, the above code better explains how to drive the circuit to select the RECORD and STOP actions.
Finished Controller Board
I received the fabricated PCB from OSH Park today. I populated the parts, then did a quick bench test to make sure it was operational, added some hookup wires and then covered the PCB with clear heat shrink tubing. Here's the completed subassembly ready to install inside the SD Recorder:
Click for larger image
Overview of the Complete System
Here's a short video I made just shortly after completing the first working pass of the software and getting it linked into the telemetry subsystem in my autonomous car, Johnny Five. I would up having to rewrite substantial portions of the modified TVOut library in order to free up processor time to write all the graphical information to the screen at the video frame rate. To do this, I removed a lot of code I didn't need, such as the ability to switch the X/Y resolution of the overlay video, as well as algorithm improvements to the line and circle drawing routines. I also changed the text handling code to extend the Arduino Print class in order to reduce duplicated code as well as gain some additional features that are only available in the Print class. The code for the camera board and the modified TVOut2 library are linked below, as well as a zip file with gerber files you can upload to OSH Park, in case anyone want to duplicate my work.
First Field Test of the System
After trying a few different approaches, I finally decided to mount the camera system in the front number of the car. There turned out to be two convenient holes moulded into the bumper through which I could attach a mounting plate I cut out of 1/8 inch plywood with a laser cutter. Here's a photo showing the whole subassembly:
Click for larger image
Then, it was off to the parking lot at a local school for the first test of the system. The update rate for the overlay data is too slow, so it's hard to follow. And, it appears the "steer to" line (vertical bar) is not working properly. However, later in the video, at about 2:45, you can see my robot, Johnny Five, start to steer erratically when the GPS data seems to drop out (notice that, at the same time, the number of sats drops, then changes rapidly, and the ACC 'accuracy' figure starts going up to indicate the GPS is computing less accurate readings.) I'd seen this behavior in other test runs, but this is the first time I've been able to directly link it to issues with the GPS readings. So, all in all, it seems the system will be worth the effort as a post diagnostic tool.
Weird Issues
The development of this camera system went quite well, at first. in fact, it was only when I had all the components built and assembled that I noticed a really odd problem, which is that the video recorder would endlessly reboot when all the pieces were power on at the same time. I eventually determined that this did not happen if I powered up the video recorder first and then, after about a 10 second delay, powered up the video camera. At the current time I have a rather bodged up circuit made from a PNP transistor and resistor built up on a perf board that's used to power on the camera after a delay when Johnny Five is powered up. I'm in the process of redesigning one of the circuit boards on J5 to included this circuit and make for a neater layout.