Thermal Cameras

Geiger Bot's map design with false-color lookup tables to indicate relative intensity of a measurement borrows from thermal cameras; the film "Predator" (1987) was my specific inspiration. LUTs are not unique to thermal cameras, but it was due to thermal cameras that I initially researched and began to implement LUTs.

So, while thermal cameras are somewhat unrelated to Geiger Bot itself, there is very little information on thermal camera design or use online, let alone caveats or criticism (as few people own them). I figured I'd add some.

Before I begin, while I try to be constructively critical, don't take that an endorsement against thermal cameras in general or these specific units. It isn't. Most of what I take issue with is software, not the hardware itself, which has evolved significantly since 1987.

FLIR TS24

(image taken with FLIR T420)

Introduction

The FLIR TS24 is a very rugged, IP67-rated (and floats on water) monocular-style thermal camera with a FLIR "Tau" core marketed for consumer hunting applications; other "H" series cameras that are more or less the same are marketed to nautical, law enforcement, and security use. It is essentially competing with image intensifiers (I2), or night vision scopes, though for the most part you cannot legally hunt at night, so the thermal camera's daytime use becomes more practical.

The Sensor

The nice part about the H-series (including the Scout) is you get a lot of sensor for the dollar. In the TS24's case, that's a 240x180 VOx microbolometer with a < 50 mK NETD sensitivity.

The sensor is also limited to a max 8 Hz refresh rate, compared to the 60 Hz of the T420. This is an artificial limitation so it can be exported without a State Department license (due to the US ITAR laws). Practically speaking, I don't think 8 Hz is a significant limitation vs. 60 Hz. It's noticeable, but it's not terrible. You too are governed by ITAR, so for example I could never take my T420 out of the country, or let a foreign national use it, without violating ITAR. (note that China produces the exact same sensors so this makes little sense and probably just harms the US economically)

The sensor is indeed as sensitive as you'd think from the specs (50mk = 0.05 deg C). You can trace your finger along the wall and see a heat trail. Identifying things outdoors at range is pretty easy (within its resolution limits) and the biggest obstacle here is optical zoom; thermal cameras use rather expensive germanium optics as glass/plastic is completely opaque to thermal wavelengths. Still, with the default optics it certainly outperformed the detect range on my Gen II image intensifier.

Caveats

However, there are a few major things to note here. The Tau cores on the Scout line are optimized for their intended "tactical" use-- seeing mammals at range outdoors. There is no adjustable focus and the automatic scaling / contrast is in a fairly limited range. Basically, the idea is to keep prey always visible without messing with manual settings.

So, while this is probably overall a good design decision (I have to lock the T420 to a manual range to detect mammals in a similar fashion without frustration), it also makes the cameras relatively inflexible (think of Predator 1 when Arnold covers himself in mud). A much cheaper and lower-resolution FLIR i3 is actually better at showing closer, high-contrast detail on objects because of this. So I would recommend keeping your end-use in mind.

The LUTs on the Scout series are white-hot, black-hot, and some hot-only isotherms. The Tau core itself actually has a much better selection built-in but most of the Tau core's functionality is disabled, which is unfortunate. The amount of options and image customization you can do is pretty much non-existent, a curious omission on a $3k USD camera, considering any $3k USD digital camera would have extensive manual options and there would be riots, fire, and looting if it did not.

You can, and I have, used it to do rudimentary home energy audits (again, more flexible than an image intensifier). FLIR in fact advertises the Scout line for this purpose. It is sensitive but the fixed focal range and limited dynamic contrast range limit its usefulness here, as you can see in the E60 comparison link below. It is really unable to do medical or veterinary imaging altogether.

One final caveat is that of nature itself. Some animals have incredibly good natural insulation and are almost invisible to thermal imaging, at least under the right conditions. An example of this is looking at a pheasant in the winter with the Scout; aside from the head, the rest of the bird was the same temperature as the environment because of the high efficiency of feathers and down as insulation. Conversely, deer, dogs and horses were all easily spotted under the same conditions.

Variants

There are two variants; standard and pro, which are shared elsewhere in the H-series line. The difference is a frankly, rather insulting $1000 software switch to enable the built-in image and video capture features. Only someone in marketing would think that was a good idea. Even the standard version comes with a SD card. I've seen "Pro" owners recommend using the video output instead of the digital capture abilities -- probably due to the compression artifacts. (see T420 section) If nothing else you can buy a cheap Chinese JXD MP4 player, many of which have composite video input, for about $50 on eBay and just record with that, if you don't mind the annoyance of a dongle.

There is also the PS- line which is 2/3 the price, smaller and lighter; the drawbacks being no zoom optics, a non-removable battery, no digital image capture option, and no video out. Although, amusingly enough, the latter isn't really true. You can tap into the hotshoe connector on the bottom of the PS device and get analog video out. There is no official hotshoe connector, but the terminals are there and connecting two leads for video is not difficult. UPDATE: I have been informed by a PS series owner that the video output terminals were not active on his unit. This may reflect a hardware revision or a factory configuration change to the core itself. Unless you can personally verify the unit you are purchasing has this capability, I would not expect analog video output functionality on the PS series in light of this new evidence.

Practical Outdoor Use: Range

The default 19mm optics are equivalent to "3x zoom" on a 35mm or digital camera. The QD100 lens on the BTX model, 20x zoom. For taking pictures of wildlife or trying to spot things outdoors, will 3x zoom get the job done? Sometimes. Other times, not so much. I find the main limitation outdoors in rural areas is the range more than anything, even the resolution.

As an example of why range matters, I know a soldier in the US Army who recently deployed to Kandahar Province, Afghanistan as a forward observer. His FOB comes under rocket attack semi-frequently. One would think this would be a perfect opportunity for thermal imaging to find the source but the range of Katyusha rockets exceeds any handheld thermal imager I'm aware of. So, even if he wanted and was able to buy a thermal imaging system with his own funds (he bought his own ACOG), much like a M4 carbine, they just lack the necessary range.

That's an extreme case, but the point is, you have to have the range to the target you're interested in.

More Information - Sample Images from Scout

I have one sample image below from it, but for others I would recommend looking at TNVC's product pages for the H-series.

For a Scout vs. E-series comparison (same sensor), you might want check this comparison out. (and, with the same target images, an i3 vs. i7 comparison)

FLIR Scout TS24 Summary

• • Good:

    • About same price as an image intensifier (I2) ("night vision")

    • Very good thermal sensor for the price

    • Generally, superior detection range to an I2

      • This depends a lot on the thermal contrast of the scene. Thermal contrast for finding mammals and avians is much better at night and during the colder parts of the year.

    • Defeats some camouflage and cover, can see through fairly heavy smoke, light water vapor

    • Much more versatile than an I2, can be used safely in daylight

    • Very rugged design

    • Fast startup times from standby (for a thermal camera)

  • Bad:

    • Heavier, larger, and much shorter battery life than an I2 (6 hours vs. 40 hours); I2s are still used for a reason

    • Software-locked core settings artificially limit its versatility (the "Guardsman" is somewhat less restricted in this regard)

    • Absurdly expensive software-locked digital capture options

    • Video output is composite video only

    • Cannot manually focus 19mm optics (without a tool, breaking the water resistance, and warranty)

    • Changing out AA batteries is cumbersome and requires a screwdriver

    • FLIR logo plastered into corner of image 100% of the time rather unnecessarily

    • "Freeze frame" on the "standard" models lacks UI indication and is very confusing to most people who use mine

    • Viewfinder seems to develop a large number of vertical bar display artifacts if used at > 50% brightness for extended periods of time

Above: What happens when the Scout is left on too long. Note the vertical colored artifacts, which are also present on the "FLIR" cityscape loading screen. These are *not* non-uniformity/flat field artifacts and are not present with video out. Time and cycling the LUT rapidly seems to slowly clear them up (a cold reboot does not). Sort of like pseudo burn-in on a plasma TV.

FLIR T420

Introduction

The T420 is a very different camera from the Scout. The Scout is made by FLIR's North America security products division (Inframetrics?) whereas the T420 comes from their Swedish (Agema) division. I really like the T420. It comes in a rather impressive case that looks like it would survive crashing into the island on Lost, and a bunch of travel international AC adapter plugs. Which is ironic, because it's a felony to take the camera outside the US.

The OS - Windows CE

Whereas the Scout uses a pretty minimal and stripped version of Linux, the T420 runs Windows CE, once the cutting edge but now starting to show its age. Still, it is used pretty well to extend the capabilities of the camera. I managed to crash the core app "uicore.exe" several times, though I could not control the cursor or interact with it, so there's no haxxoring into some Windows CE Solitaire.

Note I haven't encountered that since the last update

The Sensor, and Comparison to Scout

The difference between the Scout and T420 is pretty much night and day, though the sensors are close. (the T420 has a 320x240 core with a NETD of < 45mK) A lot of this is due to the adjustable focus and less restricted contrast range. (the Scout has a minimum focus distance of about 2m) The bottomline is that the T420 produces very, very impressive results.

Relative to the Scout the T420 has pretty advanced software functionality, such as capturing a simultaneous visual spectrum image and doing thermal fusion and picture-in-picture. There are also numerous measurement features for actually reading the temperature.

There's even a laser pointer, to help find exactly WTF you're looking at in the infrared spectrum. Like the CMOS sensor for the visual spectrum, it's rather elaborately mechanically matched to the thermal array's objective. Three lasers in a triangle would've been much more amusing, but oh well. You can buy your own Tau core, but good luck ever developing the mechanical parallax tracking and FOV stuff like this does. Seriously, some pretty crazy engineering with the spectral fusion modes. This is why FLIR kind of wins the thermal market by default.

The built-in LCD has decent brightness, contrast and viewable angles and an anti-glare filter (ie, matte not glossy). A sunshade is also included for outdoor use. While it's among the better TN displays I've seen, I would've liked to see 8-bit IPS panel used instead which would have likely provided superior contrast on the LUTs and display intermediate shades better than a 6-bit TN panel.

Image Quality

With numerous LUTs, manual settings, adjustable focus, and a good sensor, the T420 produces and displays some very nice images.

The problem with the image quality isn't on the camera's display though. It is what happens to the images after you save them. They get overcompressed, and the quality suffers.

Though you can't screenshot to remove compression artifacts from video capture, unfortunately, as the raw radiometric data is not saved. And, would you enjoy having to screenshot every single picture you take with your digital camera to get decent quality from it? I suspect probably not.

The curious thing is that the raw sensor data is uncompressed entirely -- so most of the filesize isn't even the JPEG image data. I've been unsuccessful at importing them into ImageJ using the raw import feature (unfortunately I believe the format is proprietary and unpublished), but I do see a couple repeated images which would only happen with uncompressed data. So if the size is such a concern, why not compress the raw sensor data with lossless compression? You could have lossless PNG images with smaller filesizes than are seen today!

Original "moon" JPEG from camera (overcompressed): 177 KB

Re-saved as PNG (losing raw uncompressed sensor data): 115 KB

Misc and Speed Issues

There are some relatively cool features of it, such as remote control from an iPad / iPhone (or Android device). From what I saw during the firmware update patch, this seems to be driven by .asp web services running on the camera. You can use it as an interactive remote control or dump all the images from the camera via WiFi to the mobile device.

However, there are some speed issues. The framerate for the remote control is not consistent with measurement tools (the box measurement tool, in particular, is very CPU intensive) or range autoscaling enabled. The bottleneck here is processing speed. The SOCs available for Windows CE have pretty limited CPU speeds. I also don't believe one can vectorize code with ARM's NEON SIMD unit with Windows CE.

Update: The measurement tools and general UI responsiveness have been greatly improved in the latest firmware update. Very impressive!

Another place speed is an issue is the photo gallery. In addition to very low video quality playback (software decoder, perhaps), after taking an image it can take a good 10 - 20 seconds to show up with a class 6 SD card. The gallery more or less breaks at > 1k images... which makes the image overcompression make even less sense. You cannot create subdirectories etc so the effective file count limit is across the entire card.

Anyway, I don't want to seem too critical; there are certainly flaws with it (like everything else in life) but overall it is a pretty amazing piece of hardware. Much more so than the Scout. The actual sensors are similar, but it's all about the adjustable focus and contrast range.

Lens Cap Lanyard Mod

One simple "mod" I recommend is a lens cap lanyard, which is oddly missing from the accessories included. The lens cap is pretty prone to falling off and has a lanyard hole already. I took a single strand of inner 550 paracord, colored it black with a Sharpie, and secured it with a single knot to the lanyard attachment point on the front of the camera, then reinforced the knot with a dab of superglue. 550 paracord, great stuff.

Competition

FLIR isn't the only game in town. There's a very similar line of thermal cameras from Fluke, the Ti50s.

Looking at the case, camera, and included accessories the products almost seem to be clones of each other. I don't know who cloned who. Fluke claims they have a patent pending on the color fusion stuff, which is ridiculous. The US Army Natick Research Center and various defense contractors were working on "color fusion" since at least the early 1990s, if not even earlier. This eventually led to the AN/PSQ-20.

Anyway, other than a translucent overlay in the fusion modes, the Flukes don't seem to do anything the FLIR cameras don't, and cost over 3x as much. The ergonomics aren't as good (imo), there's no WiFi or mobile device connectivity, and the file saving is annoying in a different way. The individual sensors are not quite as good as FLIR's, and the highest resolution available is 320x240.

I mean, the Fluke Ti50 is $22k USD. An equivalent FLIR Tau core (320x240; <50mK NETD) is $3k USD. So, $19k for the software, housing, battery, optics, and screen? Seems a bit much. I'm all for making a profit but that's just crazy talk.

Thus, the T420 seems to be rather competitive. I think FLIR will win this market segment quickly because of the massive pricing difference. Assuming they haven't already.

FLIR T420 Summary

• • Good

    • Produces consistently amazing thermal images.

    • Surprisingly useful for a great variety of things.

    • Excellent manual image controls (focus, dynamic range, LUT, measurement tools, etc)

    • Focusable optics

    • Excellent ergonomics, looks like a digital camera (some thermal cameras have a pistol grip... and a laser pointer. what could possibly go wrong?)

    • Very well-engineered parallax tracking for visible spectrum camera and laser pointer

    • WiFi capabilities allow for fast data transfer to mobile devices in the field or for you to assume direct control of the camera

    • Free and rather good "FLIR Tools" mobile app available

    • Majority of bugs (WiFi, Rainbow LUT) have been addressed in firmware patch

  • Bad

    • All images and video saved by camera are very overcompressed as shown above, requiring each individual image to be manually screenshotted and cropped to be saved with a sane level of compression

    • Video streaming and image gallery have processor speed issues. Image gallery issues make effective card size limit < 1 GB. Subdirectories, lol wut?

    • Quality of visible spectrum camera is very subpar (may be a size limitation with the mechanical parallax tracking though)

    • Video output is composite video only

    • FLIR logo plastered into corner of image 100% of the time rather unnecessarily

Sample Images

Hot resistor on irrigation control panel. An interesting thing to note is that the camera has compressed the cold end of the LUT to pretty much nothing, which means the camera automatically changed the scaling; I'd not seen it do that before.

Is that the ITB? Well, I'll go find my anatomy textbook later. It looks a bit inflamed, whatever it is.

Other medical-type things I've found in humans:

1. Fever -- hotspot on camera matched medical IR thermometer (the forehead scanning kind). Individual's skin temperature was also 3 degrees hotter than mine in general. They ended up testing positive for strep throat and were Rx'd an antibiotic.

2. Inflammation on foot -- individual noted pain near heel of foot. Looked kinda like the fat pad at the heel of the foot was inflamed, though again I'm no doctor so who knows. Would not have noticed without comparing it to the other foot to find the asymmetry.

Hot dog. Oddly the front of the tongue was the hottest measured temperature in this image.

Dog that was entirely too interested in the laser pointer on the T420. No licking the Germanium lens!

That would've been a vague white blur with the Scout. A MacBook Pro looks much less dramatic because the aluminum conducts the heat into a relatively wide area.

There are 4 humans in this picture. Two on the left, two on the right. And it wasn't even that hot out... so let's just say "Predator 2" wasn't actually filmed in a hot city during the day.

I hate to say it, but the "White Hot" LUT probably works best in situations like this.

That x-treme hotspot by the GPU is two transistors. Are they supposed to be that hot? I'm not sure. Without a doubt, the hottest component in my desktop system. (Asus P4P800 Core2Duo motherboard)

Flip clock in sunbeam. The normal plastic shield has been removed. (thermal cannot see "through" plastic or glass). The dark blue artifacts near the 8 are due to the Rainbow LUT being bugged, which was fortunately fixed in the recent firmware patch. It took about 15 seconds after the flip digit changed for the digit to be clearly visible in the thermal spectrum.

It's almost like someone was sitting there. An entire Ghost Hunters episode just waiting to happen.

This is the fixed Rainbow LUT, showing a thermal anomaly in a breaker panel. Slight barrel distortion is apparent. Supposedly this is a potential fire risk although I don't really see how 44 degrees C is going to start a fire.

I'm Commander Shepard, and this is my favorite case on the Citadel. (metal case, painted) The case was exposed to sunlight for a few moments before the picture was taken. It is impressive that the thermal camera could see that at all.

I'd say he needs more coffee.

Now this is a good Predator-ish image. Though the crosshairs obscure it a bit, that's my neighbors dog whining to get in at night. (they never did let him in.)

Helena, MT. Daytime, summer. Solar heating effects overpower most all other thermal contrast.

Dogs are messy drinkers. (spilled water is easily visible due to evaporative cooling) Note that water is opaque to LWIR thermal wavelengths; the "depth" you see in the water is simply a reflection of the top half of the bowl. It is an optical illusion. (though, you can indirectly see water levels, such as in the coffee picture above)

Thermal images are rarely pretty, but this one is.

Water leak under sink in newly-purchased home. (not mine) Was also wet to touch and noticeable in visual spectrum with a flashlight. Though no one else saw it until I pointed it out -- the official home inspector didn't find it either (no thermal camera). It doesn't take many detected water leaks for the camera to pay for itself... given the long-term cost of repairs and mold remediation.

Thermal fusion on image, on my dog dying of cancer. His abdomen had been shaved for a splenectomy. Previously he had anemia from a hemangiosarcoma bleeding into his peritoneum, which was not visible on the thermal camera, probably due to the fur. Parallax alignment errors are present.

This is an image capture from the Scout, rather than the T420, and colored with ImageJ's "Fire" LUT. You'll note the blown highlights in the image, an artifact of Scout's more "tactical" contrast range. The quality is in general somewhat lower than you'd see directly through the Scout's viewfinder due to the lossy nature of composite video out (and quality of my capture hardware).

To help work around the contrast limitations of composite video (and grayscale LUTs) my suggestion is to take every image capture in both white-hot and black-hot.

Thermal Cameras In the "Predator" Series

Personally I've been wanting a thermal camera ever since I saw Predator 1. I haven't seen any "making of" or "director's commentary" that discusses thermal cameras at all, so here's what I've been able to piece together.

Predator 1

The camera used was an Inframetrics (now FLIR), unknown model (535?), but it was a liquid-nitrogen cooled one with a single pixel detection element. A mechanical scanning system would rapidly "scan" each scene onto that pixel. The LUT was a 7-color version of Rainbow; black -> blue -> cyan -> green -> yellow -> magenta -> white. The thermal camera work is the most realistic in this film, probably because they had a scientific consultant. At the end where the Predator removes his mask and the contrast is greatly decreased, I believe what happened is some amount of liquid nitrogen was removed from the dewar.

At the time, the use of thermal imaging in such a manner was pretty revolutionary.

Predator 2

Very similar to Predator 1. LUT was slightly different, not real sure where magenta went to? (a third shade of blue maybe? hard to tell)

The "fetal ultrasound" scene was obviously fake. For the scene in the slaughterhouse where the vision mode was altered to see the lights, I believe that was simply a recolored image from the visual spectrum. With a 1w blue laser (which burns stuff pretty quick) I couldn't see it with a thermal camera hitting either dust or water in the air.

"...Danny Boy..."

AVP

A modern uncooled microbolometer-based camera was used here, with what appears to be the "glowbow" LUT (or "red hot" in ImageJ).

How can I tell? Like in AVP:R, the vertical non-uniform artifacts that are characteristic of microbolometers.

Every couple of seconds a thermal camera will do a "NUC" or "FFC" (non-uniformity correction / flat-field correction) where a reference image is taken with the shutter closed because the microbolmeter elements are individually very chaotic and unstable. They have to be continually recalibrated, unlike a CMOS or CCD sensor in the visual spectrum. This is especially true when the camera is first started. Anyway, despite that artifacts persist, especially at lower thermal intensities.

AVP:R

One of the weaker films overall but actually one of the better ones for thermal work, in my opinion. Again, like AVP, a real modern microbolometer-based camera was used and there is far more thermal footage than in AVP. It's pretty well done, in that regard.

They appear to be using the default Rainbow LUT or one similar to it, preserving the OG Predator look. Respect.

Compare the vertical non-uniformity artifacts (visible on the right 25% on the image) to the above sample images I posted. You can see, it is indeed a microbolometer array sensor.

Predators

Well-written and a good movie (certainly better than AVP:R), almost on par with the original.

But, the film did not use a thermal camera; rather, CG was substituted. This had the unfortunate effect of making the ending sequence with fire rather unrealistic-- a human body is easily visible against fire, as it is much colder than the fire. The thermal camera does not "glitch", they are used routinely for furnace inspection at very high temperatures. It doesn't even make sense in a fictional context; there are a great many things hotter than a small brush fire visible on the thermal camera in previous Predator films.

I hope they use a real thermal camera in the sequel.

THIS LOOKS SHOPPED