Introduction

I've been playing with the SparkFun Lightning Detector that uses the AS3935 Franklin Lightning Sensor for a while and find it to be a very interesting  but frustrating part.  A good introduction to the AS3935 can be found here, here or here.  Unless you live in a place where there are a lot of thunderstorms, you don't get many opportunities to see that it is even working.  However, I can tell you that it is extremely sensitive and I don't see how you could ever get it to work near anything like a computer.  Unfortunately, that includes the Raspberry Pi!  Remember this chip was designed to be used outside in a key fob like product running on batteries.

The AS3935 is listening to 500kHz which is just below the AM broadcast band that ends at 530kHz.  To hear the RF interference I'm talking about, just tune an AM radio to the bottom of the band and hold it near a computer or monitor.  There is a ton of RF noise that will cover up any lightning signals.  From my experience the AS3935 should be 10 meters away from anything electrical and that includes a USB wall power supplies.  

AS3935 Bluetooth Controller

Fortunately, an Arduino Nano is quiet enough to be used as an interface.  But you can't connect the USB cable to it because that will bring all that computer noise right back down to the AS3935 again.  What to do?  I added a Bluetooth serial port to my Arduino and wrote a little program that monitors the state of the AS3935 and allows modification of the registers.  I keep the AS3935 controller as far as possible from the house and run it on  an old linear wall wart power supply.  USB adapters are a switch mode power supplies and they generate a lot of interference.   Optionally, it can run for a while on a 9V battery, but you can't expect to leave it on for days that way.

Wiring Connections:

The control program listing is below.  My particular Bluetooth serial device had all ready been configured to communicate at 9600 baud.  You might need to change this for your device.  I've never had much luck with much higher than 9600 for software serial ports even though it is supposed to be possible.   Note: I'm setting the capacitor value (tunCap = 64) to make my AS3935 as close to 500kHz as possible.  This might not be the right value for your AS3935 so you should comment that line out if you aren't sure.  Also, the original SparkFun library I downloaded had an error in the Energy value function that was fixed in version 1.4.1.  Make sure you are using  library version 1.4.1 or higher.

A simple command line interface lets you change various parameters on the AS3935 and get strike information back.  I use the Android Bluetooth Serial app to talk and listen to the controller with an old smart phone.  It is a great free app and a nice way to use an old smart phone for data logging 24 hours a day.  All I have to do is keep the phone plugged in  and close enough to get the Bluetooth signal from the controller.

You always send a 5 digit number (followed with a crlf) to configure the AS3935 controller.  Each number represents one of the values of the configuration registers in order: Spike(1-9), Watchdog(1-9), MinStrike(1,5or9), AFE(-10), and Noise(1-7).  So a typical command is 12141 for Spike=1, Watchdog=2, MinStrike=1, AFE=14, and Noise=1.  Note the AFE value for Outdoor is 14 but you only enter 4 since it is single digit.  Minimum Strike can only be 1, 5 or 9.  I didn't bother with option  for 16.  The program will echo back each of the set values SWMEN.  Again the AFE value is E and will show the actual AFE value minus 10.

The Android Bluetooth Serial terminal program has a lot of settings.  I turn on the timestamp option which you see in white text in the screenshots below.  Behind the three dot menu (upper right) is an option to log data to a file.  Also, make sure it is set to send a carriage return and line feed after each transmission.   I assume other terminal programs could be configured to provide a similar interface but this app is exactly what you need.   Setting parameters is illustrated by the first screenshot.  I've entered 12141 and hit send which is the right facing arrow.  The program reads that string and puts the numbers into the right registers.  Then it reads each register and prints what is actually there.

The next screenshot shows what it looks like during operation.  The lines with all the 4s are multiple disturber interrupts that usually end with a 8 which is the interrupt flag value for a strike.  Interrupt flag equal 0 occurs when the distance value has been updated but there wasn't a new strike.  If your watchdog value is set higher than 1, or if there isn't much time between strikes, then there will be just an 8 or 0.  When there is an 8 or 0, the program sends a newline and prints the new distance and energy.  

Here is what a piece of what a log file looks like.  It is pretty much the same as what the app shows on the screen in a txt file.  The nice thing is that the setting information is also recorded.  If it has been a long time since a strike, there can be lines with a pile of 4s for disturber or 1s for noise.  Using the data requires reading this txt file into a spreadsheet using space as a delimiter.  Set both the Watchdog and Noise values high enough so that you don't get a pile of nuisance interrupts.  I'm not going to get into the details of the rest of the spreadsheet processing.

AS3935 Lightning Generator

Unless you live where there are a lot of thunderstorms you never see the AS3935 doing anything.  It would be nice to generate something that made it think it heard lighting. Just a spark will not do.  It is looking for a particular signature.  The manufacturer sells a development kit for $230 that includes a generator, but I'm not going to spend that kind of money.  There also a few other generator products, but they cost more than the AS3935.  So I decided it couldn't be that hard to just make one.  Here is what I came up with.

This is a very simple circuit that probably uses parts you already have laying around.  An Arduino Nano is used to generate the timing pulses needed to create fake lightning that the AS3935 will accept.  You could use an Arduino Uno or probably other basic Arduinos as well.  I'm not sure about some of the bigger variants.  The emulator makes use of an internal microprocessor timer/counter that might be different with other boards.

The only odd part is the 330uH inductor that is used as a broadcast antenna.  I made mine by winding 220 turns of #26 enameled wire on a AA battery size tube.  Originally I used the most inductive section of a loopstick antenna salvaged from an old transistor AM radio, and it worked just as well.  Of course you can buy a new inductor, but make sure the one you get is unshielded and basically looks like a resistor.  I needed a little more damping for the inductor than it had naturally, so R2 is in parallel with it.  D2 is across the transistor to prevent the inductor from swinging the collector of the transistor below ground.  

Driving a transistor at 500kHz is a little different that you might have seen for buffering larger DC loads like relays.  There you would just use a 1k resistor to the transistor base.  Here a 100nF cap is used to couple the high frequency signal.  Resistor R1 keeps the transistor off if there isn't any signal from D11.  A good idea considering L1 is basically a dead short to DC.

I built the circuit on a little prototype board shown in the box above.  It took a couple iterations on the RF design section so it is spliced onto the end.  You can see the inductor/antenna on the right.  Diode D2 is on the bottom of the PCB where you can't see it.  Notice that I run the generator from a battery.  Wall USB power supplies make too much electrical noise to bring anywhere near an AS3935.  The black round thing on the lower left is a 5V piezoelectric buzzer that beeps out some status information and is connected to D13 which already has the on board LED connected.

So what we are trying to produce is a waveform that looks like the one below.  It is solid blue because it is actually 500kHz oscillating so fast it is painting the area under the curve on the oscilloscope trace.  The basic shape of this I got by looking at the code from some commercially available AS3935 lightning generator.  They used considerably more complex DACs and drivers than my design.  The shape is a basic exponential decay that lasts about 600us.  The selection of C2 and the damping resistance of the inductor create this shape.

The 500kHz is derived from the 16Mhz microprocessor clock.  Timer 2 divides this clock and makes an output available on D11.  Below is the waveform right at D11.  The output gets turned on and off by the program so it isn't there all the time.  The 500kHz is shown below at both a high sweep speed and slower during the strike pattern generation.

D12 only needs to be pulsed on for a about 25us to recharge C2.  Below you can see the pulse and the value on the capacitor while the strike is being generated.  You can see how this creates the envelope with the right shape to look like the single strike pattern above.

Apparently the way you make the AS3935 think the strike is closer is to just repeat the pattern.  The program creates up to 4 pulses this way as shown in the waveform below.  I've tried increasing the time between pulses to about 20ms which is more realistic, but it doesn't seem to matter.

And now for the program.  The only thing you probably haven't seen here before is the setup for a 500kHz hardware output.  The rest is just a glorified "blink" example program.

The microprocessor has several hardware timer/counters.  Timer 0 is used by Arduino to take care of time functions like delay() so we are using Timer 2 which is similar but not used.  It has an output that has to be pin D11.  Originally I just left the output on all the time, but some of the 500kHz RF would "leak out" and cause the AS3935 nearby to get confused, so I turn it on and off by just making the pin an Input when I don't need it and an Output when I do.

The RF section is powered, at least for about a millisecond , by discharging a 0.1uf cap C2.  The cap is recharged by output pin D12.  All you need to do is turn on the output for about 25us to fully charge it.  Then as it discharges it creates the pattern the AS3935 recognizes as lightning.  A single lightning event is a series of these patterns.  The program creates a single pattern then two, three, and four patterns.  

For user feedback, the builtin LED and optionally a piezoelectric buzzer hooked to D13 are flashed the number of times the next pattern will have.  That way you can watch the "Energy" number to see it steadily increase if things are going correctly.

To use the generator, the antenna must be placed about 3cm  from the AS3935.  You can move it closer but it might start creating errors and if it is too far away you just wont pick anything up.  I find the parameter values that work best are Spikes = 1, Watchdog = 2, Number Strikes = 1, Gain = Outdoor or 14, and Noise = 1. 

AFE Gain Experiment

The data sheet says the AS3935 is optimized for two gain (AFE) settings Indoor=10010=18 and Outdoor=01110=14.  It looks like indoor has higher gain because they assume some degree of signal attenuation to the inside of a building.  Unfortunately, indoors is also where all the electronics are generating piles of noise and you probably don't want extra gain.  I doubt a wood frame house attenuates 500kHz very much anyway.  So just what is the AFE gain doing anyway?  Can you put values different than 18 and 14?  

First off, the original SparkFun library had an error in the Energy value function that was fixed in version 1.4.1.  Make sure you are using  library version 1.4.1 or higher.

Also, with the SparkFun AS3935 library, you can NOT adjust the AFE to anything except INDOOR or OUTDOOR.  I chopped out the value checking part of the setIndoorOutdoor library function in my copy, but you don't need to that if you stick with INDOOR and OUTDOOR .   Spoiler alert! You probably don't need anything but OUTDOOR.

With a controlled source for lightning signals, I can now change the gain value and look at the reported energy for what should be the exact same strike.  The table and plot below shows how the reported energy goes up as the gain is increased.  I assume you can set values below 10.  However, setting them above 18 or Indoor doesn't work right so I assume that is the maximum.

Looking at the Table on the right, I've taken AFE=14 or Outdoor as the base energy and you can see how Indoor or 18 is about 3 times gain while AFE=10 is about 1/3.  So a 10 to 1 swing in gain is possible from 10 to 18.

Energy and Distance Experiment

Now here is the rub with tinkering with the AFE gain and not knowing what you are doing.  How does the AS3935 establish distance in the first place?  By using the strike energy.  A small amount of energy must be far (up to 40km) away while something really big must be overhead (1km).  So if you tinker with the AFE gain, you are also effecting the measured energy.  That will screw up the distance estimation!  Normally the distance value is to the edge of the storm, but if you reset the statistics, the initial strike distance is also the distance to the edge of the storm.  So by adjusting the AFE gain and constantly resetting the statistics, I was able to plot the strike distance vs strike energy shown in the plot below.

Apparently with the little recommended antenna and other PCB layout considerations, an AFE gain of 14 or Outdoor has been calibrated to an average lightning strike.  Indoor probably assumes 1/3 attenuation due to a metal building and so has the gain boosted.   Now suppose I use the AS3935 with a bigger antenna?  That probably means you need to turn the gain down to compensate since you don't have access to the energy/distance equation.  I'm sure the AS3935 is actualy using a lookup table, but there should be some kind of energy equals  inverse distance squared formula that converts energy to distance.  With a little curve fitting, the equation appears to be: D = 2100/sqrt(E) Remembering that it never goes beyond 40 or less than 1 kilometres.

So what I've learned from all this is; You want the AFE gain set to 14 or Outdoor all the time.   Unless you know something about how the lightning signal is effected by shielding or other circuit change you will probably just screw things up.  Setting it to Indoor just opens up the opportunity to amplify all that electronic noise and make things worse even if the sensor really is indoors.  Unfortunately, Indoor is the default value, so no matter what else you adjust, make the Indoor-Outdoor AFE setting to Outdoor.

Watchdog Value Experiment

The watchdog value is a threshold set to limit which potential strikes are processed.  Setting the value higher means that smaller energy strikes will be ignored and the AS3935 would tend to not record more distant strikes.  On the other hand, more Distrubers will be ignored as well.  In my experience the value needs to be set to at least 1 but the default of 2 isn't too bad either.  If you need to set it even higher than that, you really need to figure out how to operate in a lower noise environment.  Separating the AS3935 from noise sources is a much better plan than just setting the watchdog to a high value so you don't get any disturber interrupts.

I've observed several storms with four or five times as many actual strikes within a 40km radius than were reported by the AS3935.  If you look at the plots in data sheet, depending on settings, only 45% of near strikes are detected at best and less than 25% of far.  Many strikes are probably rejected as not fitting the pattern or are added together to make single strikes.  I've had many close strikes where I saw the flash, but the AS3935 didn't record anything.

However, the question is, does the watchdog value only limit the minimum energy or maximum distance, or does it effect the recorded energy somehow?  By holding everything constant except the watchdog value, I made a plot that shows the reported strike energy while changing the watchdog value.  In the plot below, you can see that the watchdog doesn't change the reported energy.

Antenna Spacing Experiment

The spacing between the lightning generator and AS3935 should directly effect the reported energy.  Holding everything except the distance between the two antennas constant, I can make a plot that shows how the signal drops with increasing distance.  Although I've plotted the relationship on a log scale to look straight, the Energy is actually dropping with the expected inverse distance squared relationship just like real lightning.  Only the RF power of the generator is many orders of magnitude smaller than real lightning and the whole range becomes cm not km.

A Real Storm

I recorded about 400 strikes during a small thunderstorm that passed near my house.  Empowered with all this new knowledge, I made a plot that shows how the AS3935 was reporting the distance to the storm and the individual strike energies.  You can kind of see how the distance algorithm is just setting the distance shorter whenever a strike with higher energy comes along.  However at around an hour and 45 minutes into the storm, the distance was revised upward because only smaller strikes had been seen for a while.  Actually this storm never really went overhead and it dissipated about three hours in.  You can kind of see the cluster of energy values slowly increasing while it came closer, but then it fizzled out.

My settings were Spike=1, Watchdog = 1, Min Strike=1, AFE=Outdoor, and Noise=1.  I was getting a disturber hits about once a minute and probably should have set Watchdog to 2.

Minimum Number of Strikes

Occasionally, I get strikes recorded where there isn't a storm within 400km from me let alone 40km which is the maximum range for the AS3935.  What I think I'm seeing are very large strikes that are also a long distance away.  For example, here is screenshot from a web page called weatherbug.com.  My home is where the blue marker is, and the green circle is about the 40km range.  I was getting a few strikes an hour that I think were from the big storm you can see way off to the North and West.  Now that doesn't bother me because I'm not making a product that is going to start false alarming.  However, you can set Minimum Number of Strikes to 1, 5, 9 or 16 so that the AS3935 will wait till it has seen that many strikes before starting to report a storm coming.  I kind of like the extra activity so I always leave my value at one.

Distance Estimation Part 2

I was looking over some distance to the storm data I had been collecting over the last couple days and noticed that some distances were never recorded.  The histogram below shows the data.  Then I remembered that the AS3935 data sheet had a table of Distance Estimation that also skipped values.  I always thought the skipping was odd because the value is supposed to be just the distance in km except 63 was "Out of range".  Turns out the values in the table are the only distances you get.  Apparently 5 is possible, but I had not caught one in this data.  So using the equation D = 2100/sqrt(E) derived above to convert strike energy to strike distance will result in a finer estimation than the AS3935 distance to the storm value.

Another Storm

Putting everything together, here is a plot of the start of a really big thunderstorm.  I've plotted the distance to the storm estimate and the individual strike energy converted to a distance for the first hour and half.  Actually the storm was so violent I lost power for five hours and that ended the experiment.  There are over 1400 strikes recorded on this chart.

Is Tuning Important?

There is a lot in the data sheet about tuning the AS3935, but just how important is getting it right?  I set up my AS3935 controller on the bench and connected the interrupt pin to my oscilloscope.  Using the SparkFun example tuning program I changed the tuning capacitor value up and down while looking at the output frequency.  I probably should have recorded the factory setting first, because I have no idea what it was originally shipped with.  Anyway, the output can be changed from 513 to 488kHz as can be seen in the plot below.  For my board, 500kHz is with a tuning capacitor value of 64pF.  Fortunately, the chip remembers this value so you only need to program it once.

Now suppose you change the tuning capacitor value to something other than the one for 500kHz.  The next plot shows how the energy value is effected by the tuning.  Clearly you don't want to use the wrong value or your energy (and then distance) will be screwed up.  However, my simulated lightning signal has the envelope shape the AS3935 is looking for, but it is a single frequency and not the broad spectrum generated by real lightning.  The tuning might not effect real lightning nearly as much as it does here.

Why 500 kHz anyway?

I was wondering if 500kHz was some magical lightning emission frequency or something.  Turns out that the peak of the lightning RF spectrum is way below that.  The real reason is that 500kHz is a special hole in the international allocation of the frequency spectrum.  Originally it was a dedicated frequency only to be used by ships in distress for the famous SOS call.  Today it isn't used for that anymore, but to this day 495-505kHz should be empty most of the time. I read a proposal somewhere on the web that the frequency should be made into a kind of RF maritime memorial.  A maritime safety service called NAVTEX still broadcasts on 518kHz in the US.  There are only a few stations on the coasts and their transmissions are intermittent. 

Another close neighbor is at 530kHz which is the bottom of most AM radio dials used for Traveler's Information Stations.  These radio stations are low power and are used in parks and airports to convey information about their facility.  Mostly these stations are on 1610kHz, but about 40 broadcast on 530kHz.  There are a handful of low power Canadian radio stations on this frequency as well.  Probably none of these transmitters are going to interfere with a lightning detector.  It isn't till 540kHz that you start to get real high power AM radio stations.  

On the other side, non-directional beacons used for aircraft navigation can broadcast up to 435kHz.  There are fewer of these stations over time as GPS has eliminated their utility.  I think as they breakdown they are just decommissioned. I still have two that I can receive at 389 and 379kHz.

So around 500kHz the only thing that should be broadcasting is lightning.  As long as your receiver has a narrow enough bandwidth there isn't anything too close by to interfere.  The good news is that at this low frequency, the Q of a RLC tank circuit used by the AS3935 is naturally high enough.  Using the component values listed in the schematic, the -3dB bandwidth should be about +/- 8kHz from 500kHz.  The plot below shows just how narrow the tuning is.

How good is it?

If lightning strikes were all the same, then measuring the strength of the radio signal would be a pretty accurate method for measuring distance.  But they aren't the same at all.  The plot below shows the distribution of strike amplitudes.  The most common strikes have around 4.5 kA peak current, but the range is from 1 to 100kA.  The average value is around 9kA.  I assume this average value is buried in the distance energy formula somewhere.

So a really large and distant strike can register as the same distance as a really small and close strike.  I'm trying to get access to real lightning strike distances from companies that use time-of-arrival signal analysis from hundreds of locations to triangulate on the exact strike location.  Sadly, they all want a lot of money for this information.  

There is a free website weatherbug.com that shows distance to the last closest strike over time.  I wrote a Python program that scrapes that distance from the site every minute and saves it in a file.  This timeline can be plotted along with the AS3935 log to compare the two distance estimates for the same storm as shown in the plot below.  The plot shows comparable estimates, but the WeatherBug number is for the closest strike in 30 minutes so it really lags the end of the storm.  It looks like the AS3935 has more like a 15 minute window.  The strikes are from 800 stroke energy values.

Storm Window Time Experiment

Even without a nearby storm, occasionally the AS3935 reports a strike with an interrupt flag equal to 8.  Especially if you have the Watchdog set low and the Minimum Strikes to 1.  Maybe it is a large distant strike or just noise that looked enough like a lightning pattern.  When it does, some internal timer starts to keep track of following strikes to form the distance to the storm estimate.  However if nothing comes along, after some time the part updates with an interrupt flag equal to 0.  Then the distance will read 63 and the energy will be 0.  The table below shows the log over many hours.  You can see six initial storm estimates and six updates to 0.  If you subtract the time difference between the initial strike and the final update it is always 17 minutes.  So 17 minutes is the AS3935 storm window time which can also be seen in the storm timeline plot above.

Minimum Time Between Strikes Experiment

I had a particularly intense thunderstorm that had 2000 strikes in a three hour period.   That is about 10,000 seconds or a strike every 5 seconds on average.  However, most of the strikes were compressed in to more like an hour time window as you can see in the plot below.  An average lightning strike consists of three or four strokes about 40ms apart so most strikes last about 160ms.  Looking at the time between adjacent strikes in the data log, I found the minimum was 182ms.  With 288, 293, 297 and 325ms rounding out the shortest five events.  I realize 182ms might not be the absolute minimum, but it does show the potential for the AS3935 to capture single strike data.  

Comments or suggestions:   michael.gasperi@gmail.com