In his article in the July/August 2012 issue of QEX, entitled "A High-Performance Sound-Card AX.25 Modem", Sivan Toledo 4X6IZ mentioned that he picked up an idea for testing his software from this very website. The idea he referred to was to use Stephen H. Smith WA8LMF's recording of on-air Los Angeles APRS traffic to simulate received audio from a 2M transceiver; this is used as the input to a sound card demodulator or APRS gadget under test, in order to see how effectively a given device (or decoding routine, interface circuit, etc.) can extract AX.25 APRS packets from a busy channel. To be fair, this was not originally my idea - I stole it from Scott Miller N1VG, proprietor of Argent Data Systems.
Here are some other testing methods and related ideas I have used in building and testing packet programs and equipment. None of these are ground-breaking, but they might prove useful if you are doing this kind of work.
Use audio instead of RF. This one is very simple, but maybe not obvious. Most digital modes (including packet) are sent using audio over a radio channel - an audio generator is connected to the input of a radio, which transmits the audio over the air to be received by another radio, which demodulates the signal, outputting the original audio (or some facsimile thereof) into a receiving device. For simple tests, you can skip the radios entirely - just use a plain old audio cable to connect the generator to the receiver. For APRS, this might mean connecting the audio out pins of a gadget to the audio in pins of another gadget. It could also mean using a single 3.5mm stereo cable from one PC to another ("line out" to "line in", or "speaker out" to "microphone in"). Most intuitively, it can mean just hooking up a speaker to the output of your radio to see if the audio is distorted, over-driven, under-driven, or non-existent.
Some advantages:
Fewer variables than RF (not likely to have any fade, multipath, adjacent signals, RF buzz in the audio, etc.).
Dead-simple signal chain - no gain structure issues, for example.
Can allow you to tune by ear (especially in terms of volume levels).
Gross signal distortions or signal routing problems are immediately obvious.
Doesn't pollute the airwaves.
And some complications:
As with any audio interface, care must be taken not to over-drive or under-drive the audio.
This is doubly true when using the microphone inputs of a sound card!!
These devices do not respond well to being over-driven, and can even be
permanently damaged by too much incoming signal.
Be wary of ground loops. Ideally, all devices in the test chain should share a single ground.
The audio will be too pristine. Real world audio over RF will contain a bunch more noise, so a direct audio connection represents a best-case test.
Speaking of audio, you can also couple your devices acoustically. Try placing the receiving device's microphone near the transmitter's speaker (admittedly, this works best with PCs, since trackers usually do not have microphones). Besides providing complete electrical isolation, acoustic coupling can simulate marginal reception, background noise, and other evils that can effect RF performance. This works really well for testing PSK, SSTV, and other noise-tolerant modes (it may be too noisy for packet).
Use your sound card as a data acquisition device. Once again, you need to be careful with what you pipe into your sound card so you don't blow it up, but you can use a sound card to record more than just sound.
Example: I have some code that runs once per sample time, either on an interrupt (AVR/Arduino) or in a separate COG (Propeller). Since these executions kick off 13,200 times per second, it is not always feasible to write a character out to the serial port on every execution. Instead, I toggle an LED on every execution. If the LED is lit but somewhat dimmed at runtime, I have a sense it is running regularly with a duty cycle of less than 100%, but I cannot tell if the code is running early or late, or if it skips every third execution because of some other behavior. Instead of just staring at the dim LED, or dragging out the oscilloscope to count hash marks between leading edges (and dividing clock rate into time, etc.), I can instead connect the LED pin to the input of my sound card. I turn the volume WAY down on that input, set it to "line in" mode (not "microphone", if I can help it), and record a few seconds of the square wave input with Audacity. Then I select a section of the "audio" I've recorded , click Analyze... Plot Spectrum, and viola - I can see that the strongest signal is at 6600 Hz, right where it should be (half the sampling frequency, since each sample toggles the pin either on or off).
Another example: If I'm monitoring the microcontroller at runtime, and want to record the transitions from one AX.25 symbol to another (MARK to SPACE, vice versa), I can use a pin to represent the current state (1 for SPACE, 0 for MARK) and let 'er rip. Here again, I connect the pin to the soundcard, set the project rate to equal my sampling rate, and record the results. To analyze, I use Audacity to zoom waaaaaaaay in on the signal, and can see the individual data points at which the pin was high or low. This scenario is not quite as clean as the previous case - the symbol changes occur at uneven intervals, during which the charge on the sound card's ADC rolls off - therefore symbols that last for five, six, or seven sample periods gradually drop in voltage, instead of staying steady at a high logic state, then dropping crisply at the next symbol change. Still, the data is easy to decipher visually, and within limits might even be accurate enough to be converted back to binary values through analysis of the recorded "audio" file (see below).
Use sound editors to simulate ADC input. If you are working with microcontrollers (or even PC software) to process incoming audio, it helps to be able to examine your DSP's input signal in great detail. You might find that your sample rate is too low, or that there are some harmonic effects within your algorithm that cause certain input frequencies to break your math. To get this kind of insight, it helps to decimate the audio input down to the actual sample rate of your equipment or software.
For example, my favorite ADC sample frequency for 1200 baud AX.25 is 13,200 Hz (a partial multiple of 1200 x 2200, the two audio frequencies for MARK and SPACE). As part of my testing regimen, I frequently record APRS audio on my PC. The audio might come from a VHF receiver, sound card modem like AGWPE, or directly from a TNC. I take that audio recording and down-sample it to the frequency of my ADC routines (typically converting it from stereo @ 44.1kHz to mono @ 13.2kHz). I can then analyze the PCM directly at the same frequency as my software. Some audio tools will allow you to record audio at a non-standard frequency, saving you the conversion step. The tool I use most often for this work is Audacity, which supports both techniques (re-sampling, and recording at odd frequencies).
Use PC tools to analyze the audio data. Ok, so as described above we've got a WAV file with a weird sampling frequency - what do we do with it? If you just want to get an idea what the waveform looks like, try zooming waaaaaaay in on a tiny time-slice of the recorded audio. In Audacity, for example, you can zoom in far enough to see the voltage levels of the individual samples that make up the waveform. If you want to analyze the audio data numerically, you can use something like the Perl module Audio::WAV to read in those voltages. Now you are seeing the data as it would be read (hopefully) by the ADC of your device. From here, it is a simple matter to write that data to a text file that you can use with a spreadsheet program.
Yes, a spreadsheet program! By using worksheet formulas, it is easy to put together a mock-up to test a decoding algorithm. For example, the ADC routine I use is essentially this:
(voltage of this sample) * (voltage from 6 samples ago) = (multiple)
SUM( last 7 multiples ) = (score)
IF (score) > 100 then (AX.25 symbol) = (SPACE)
ELSEIF (score) < -100 then (AX.25 symbol) = (MARK)
ELSE (AX.25 symbol) = (the previous AX.25 symbol)
All of this math can be easily duplicated over a few columns, using basic functions. By performing one function per column, you can see in fine detail how your routine handles signal transitions, noise, timing errors, and other minutia. And of course, since your input is a static text file, you know that you will have the same input data for every variation of the routine you want to test. You could never hope to analyze the performance of your math routines at this level of detail when your data source is over-the-air RX audio, processed in real time.
Try your code on an emulator. This one is tricky, and may not be possible (or feasible) for your particular toolset. However, if you are lucky, you may be able to run a virtual machine on your PC that can emulate the behavior of the chip or board you are using. Emulators and/or simulators are available for PICs, AVRs, and most ARM chips; and for a variety of hardware platforms for mainstream OSes like Linux and Android. Some of these tools allow you to feed data in to the VM from a file, for example to simulate incoming audio, or data from an ADC or decoder chip. You may also be able to simulate the execution of your program one CPU cycle at a time, to really dig into register values and other memory contents, to make sure everything is working as expected.
Try your code on a PC. Of course a PC will not run the firmware you just compiled for a microcontroller (at least not directly), but with a few tweaks you can re-compile your code for the PC instead, and use that program with the same text file you analyzed with Excel (I am assuming here that you are using C, Java, Basic, or some other language that is available on both the microcontroller and your PC). This can provide results similar to an emulator, in that you can feed data into the program from a file, write variable values to STDOUT or a data file, or pause between every line of code if necessary.
One example: Some routines depend on the quirks of a strongly typed language in order to work correctly. Does the math between two 16-bit unsigned integers work out the way you wanted? Does that subroutine loop correctly when the 8-bit counter rolls over to zero? Did I de-reference my pointer correctly, or am I trying to calculate the square root of a memory address? These are all details you can check on the PC, and write out meaningful messages instead of just blinking some LEDs at runtime.
Another example: I think much faster in Perl than I do in C. If I have an idea for a new routine, I can hack together a rough sketch in Perl in a few minutes' time without having to look up and decipher some obscure detail in a header file. Perl is also much easier to use when manipulating data files and text, which can be a challenge to read, write, and parse in C (for me, at any rate).
A third example: Consider the case of the Propeller. It has no built-in ADC, and the Sigma-Delta circuit recommended in the app notes and in many online sources cannot be used on a breadboard, due to the high sampling frequency. One simple solution is to use one of the (digital) pins as a single-bit ADC. You de-couple the audio with a capacitor, bias the AC voltage to Vdd/2 (right at the logic threshold), and let the upswings and downswings of the audio nudge the pin into alternating high and low logic states. This reduction to a one-bit data stream eliminates all of the power data, and the vast majority of phase data, from the audio you are trying to process. What does this do to your elegantly crafted Fourier math? How much error does this introduce into your clock detection routine? Is there enough signal left to even attempt to decode? If your only mode of programming is to compile for the Propeller and hope for the best, you might find many issues difficult to solve.
In one particular coding session, I was completely clueless as to what was breaking down and why. To troubleshoot, I moved my research efforts from the microcontroller to the PC, using nearly all of the techniques outlined above:
I recorded the audio of one packet of clean, correctly driven APRS from within my PC (recording the audio out mix data while it was still in the software domain).
I used Audacity to record this audio, with the project rate set to 13.2kHz, and the channel set to 16-bit mono.
I exported this audio to a WAV file, also at 13.2kHz 16-bit mono.
I used Perl and Audio::WAV to read the PCM data, convert it to a 1-bit stream (positive PCM value = 1, negative PCM value = 0), and write it to a text file.
I adjusted my Prop-GCC program to compile on a PC (using TCC), to read data from a text file (instead of an input pin), and to dump nearly every variable value for every input cycle.
At runtime, I redirected the output of my program to another text file.
I spent hours poring over the output file (often in Excel), tweaking the program, re-running, and re-analyzing.
This was an arduous process, but after two weeks of work, I was finally able to answer some complex questions about the quality of my data and the inner workings of my code. None of this would have been possible running the code real-time on the Propeller, trying to decode wildly varying on-air packet formats, timings, and signal qualities from a VHF receiver.