October 22, 2016 - Reefmaster, Lessons Learned

Post date: Oct 23, 2016 1:24:55 AM

I finally found the time to write down some details about "lessons learned" from building the custom depth map in Reefmaster. I will break it down into sections: Tides, Time Stamps, Boundaries, Bugs & Limitations, Conclusion.

Tides

Since Reefmaster has to accommodate measurements taken in the ocean, it has the ability to compensate for tidal changes in sea height. It does this by importing tidal files, which are a comma-separated file containing a date/time value, and a water elevation adjustment from a normalized "Mean Sea Level" height (eg. 8/21/2016 23:15,-5.00). For adjusting data collected at sea, you can simply download these files for the desired location and time period. For lakes and other interior bodies of water, no such files exist. Fortunately, there is a USGS gauging station located on Stillwater Reservoir that takes a water height measurement every 15 minutes. The tab-delimited data looks like this:

USGS 04256500 2016-08-21 23:15 EST 1675.00 P

I utilized this data to create my own Stillwater tidal file. The USGS gauge data contains an absolute elevation, such as 1675 feet, and we want a difference from normal for the tidal file. To compensate for this, the first step is to choose a normalized water height. I chose 1680 feet, the top of range for Stillwater, for reasons I will discuss in the "Boundaries" section below. I loaded the USGS data into Excel, changed the formatting, selected only the required columns, subtracted from the normalized height value, and exported to a CSV file. With the resultant file loaded into Reefmaster, the program can then match the closest time in the tidal file to any collected data point from the depth finder file, and adjust the depth value accordingly. Without this adjustment feature, accurately mapping Stillwater would be next to impossible as the data collection was performed over two summers, in water heights varying up to seven feet. This is one feature where Reefmaster has a distinct advantage over the Insight Genesis online map. When uploading data files to IGSM, you can only apply one adjustment value per track file, as opposed to an updated value every 15 minutes in Reefmaster.

Time Stamps

The application of tidal adjustments detailed above is only made possible by matching time stamps between two data sets (collected vs tidal). The quality of your map can quickly suffer if the accuracy of these time stamps are off. When I first created the Stillwater map in RM, I was immediately disappointed because I couldn't see the original river bed in the map like I could in the IGSM online map. You can see this map in one of my first posts on the matter here.

It took me quite a while to figure out that a subset of the collected data files had time stamps from 2013, even though collection took place in 2016. The file names included the correct date of the collection (good work Dave!), however the file time stamps were off because the depth finder had to be reset in the field after the firmware locked up, and the system date had not been automatically set by the unit. This should be trivial for the unit to do since it has access to the highly accurate GPS time. A system firmware bug perhaps? Dave thinks it may have been a result of frustrated button mashing. Either way, once the tidal files were copied and adjusted to properly account for the false 2013 dates, the issue was eliminated, and the river bed appeared quite clearly.

The handling of time stamps is made even harder by Lowrance's choices in their proprietary file format. They do not write a true time stamp for each point in the data itself. Upon starting collection, the depth finder operating system tags the file with a creation date and time. In the file, each data point contains a "seconds since start of collection" value. If you want to perform tidal corrections on the data, you have to be very careful to preserve the file's creation date. Reefmaster developers even go so far as to recommend writing down the start time of each file in case the file creation date is accidentally overwritten by your computer's operating system. Lowrance just came out with a new "closed source" proprietary format. Hopefully they've improved their approach to time stamps in the new format, and will also open the format of their files so other programs like Reefmaster can utilize them.

Boundaries

Reefmaster has the ability to import KML (Google Earth) files that define the shoreline and island boundaries for a body of water. Fortunately, the USDA GIS page has an excellent boundary data set for Stillwater reservoir that saved me a lot of time. It did take me some time to convert their file format to the required KML format though. The USDA shoreline boundary was nearly perfect, and I did not have to make any changes to it. The island boundaries, however, came up a little short. They had created boundary data for approximately half of the islands in the reservoir. I manually added about 40 additional islands. I used the 2013 Google Earth imagery for this purpose, since it's water height value (1677 ft) was as close to 1680 feet as I could get.

The boundaries are very useful in building a map, because they can be assigned a "depth" value for interpolating the contour map. The default value is, of course, zero feet. It is particularly useful, because often you can't get in close to shore to collect data, and the boundary layer allows an interpolation between your closest collected data point and shore. Rather than an extrapolation using no knowledge of the shoreline distance. Island boundaries also create a virtual barrier that prevents the interpolation algorithm from utilizing data across these areas. The overall result is a much more accurate map.

This is why I had to normalize all the collected data using tidal files to 1680 feet. If you normalize to something lower, you now have an immediate discrepancy where the collected data meets the defined boundary with depth of zero. With normalization, 1680 feet water height is synonymous with 0 feet depth. This approach seems obvious to me now, but that did not "click" for me until I began attempts at normalizing all the data to lower and lower water heights. I was attempting to show what historical heights of the reservoir looked like. I could get the depth map to look almost identical to historical drawn maps, with the exception of near the shorelines and islands, where everything looked terrible. It was then I realized what I was doing wrong. This setting also alleviates the need for multiple boundary sets with different values.

Bugs & Limitations

As good a program as Reefmaster is, it is not without issues. A major bug I found was in the export of vector contour maps in KML format. The color scheme that you precisely setup in the program is not conserved upon export of this particular type of map. The shaded relief KML map export worked properly, and that is what is shown on this site.

Another missing export feature, is Reefmaster's inability to output contour maps in MBTILE format. They do have some MBTILE support, as you can load a file as underlying imagery for reference while creating your project. Unfortunately you cannot create a contour map in this format.

There is currently no easy way to adjust the final water height before exporting a map. Even though I normalized all the data to 1680 feet, the reservoir is rarely that high. I would like to quickly export a map for every 1 or 2 feet of elevation change, to provide an accurate map for any time of year. Again, the developers claim this capability is coming in a future release. With the current setup, I have to re-import all the data tracks, applying a different tidal file normalized to the desired water height, and re-create the map. This would take hours to create all the desired maps.

Another frustrating thing I've learned, is that their next major version release will require an upgrade fee. I understand the need for this from a business perspective, but it seems they are just creating an all encompassing rule for upgrades. I think they should set a window of time, say two years from purchase, where you get free upgrades whether it's a minor or major release. Some people (like me), have just recently purchased the program, and we're already left in the dust on upgrades. While others that bought 3 years ago, and have been receiving free upgrades for an extended period of time. It also irks me a bit that they nickel and dime you on plugins to add additional features that should already be included in a "professional" version. In my opinion their website is very misleading on this fact, bordering on false advertising. It wasn't until after I purchased the software, that I realized it couldn't do half the things I was expecting it would, and went back and read the website and forums very carefully.

Conclusion

I don't mean to come across too harshly though, as all in all, Reefmaster is a wonderful program. The developers have stated the bug and missing features will be remedied in the near future, albeit for an upgrade fee. They are quite responsive to users on their forums, and have really put together a great piece of software.

I much prefer Reefmaster's ability to customize your own map to Simrad's IGSM online "one map fits all" view. I eventually go cross-eyed looking at their maps with ten shades of blue. I also don't care for Simrad's IGSM business model. They're leveraging the efforts of their own customer base to collect data for them and improve upon their own commercial map product, while also charging those customers additional money to utilize the full service (certain aspects are free). I could see charging customers for the service, but then paying back, or subsidizing, those customers that contribute large data sets that improve their product.

Overall, I am happy with Reefmaster, with the few caveats listed above. It certainly produced a beautiful map of Stillwater Reservoir.

Update: April 4, 2017

With the recent release of Reefmaster 2.0, all the bugs and feature limitations previously mentioned have been remedied. The new release is very impressive, and highly recommended.