05 Mission Flights and Analysis

Matthew Berk, Tyler Jordan, Michael Vegh

Initial mission flight tests began the week of June 1. Numerous issues involving incorporating the mission code onto the Pixhawk were overcome, including frustrating bugs such as grid mismatches which caused the airplane to drop water on the wrong cells. Our flights confirmed earlier predictions that battery current would not be a significant limitation to flight time, as every mission with a score below 100 (save one) used every drop before consuming the allotted 650mAh.

Figure 1: Power Usage and Current Used vs Altitude and time. Shows that even in a 7+ minute mission there is plenty of battery capacity still available.

Nearly all flights were initiated from the east end of Lake Lagunita, down the gentle hill from the barbecue area. This area seemed to have fewer tall weeds which had previously caught the airplane during landings and occasionally caused minor damage. Additionally, this area allowed for an initial climb into the prevailing wind (often out of the West) which improved the aircraft starting altitude altitude at the beginning of the search path.

Mission indices 3-16 were flown on Friday, June 9th. The team arrived with a well-prepared flight strategy to pack as many flights as possible into a single morning. A car was brought to the lake to provide auxiliary power to several battery chargers to allow continuous flying with runners replacing freshly charged batteries with discharged ones and watching the batteries to ensure safety. Tools and a work surface were brought to the field to allow quick field repairs if necessary, and minimal interference with the equipment from ground debris. Overall the team was able to maintain a steady flight pace from 5:30AM onward until other teams showed up and began to fly, breaking cadence and slowing the flight routine down considerably.

16 official missions were flown, with the first two being flown using the "lawnmower" search path which made crosswind sweeps starting at the upwind side of the field. The latter 14 were flown using the "Linkin Park" search path which was an upwind-biased self-intersecting path. It gave much more consistent coverage of the middle of the map, which was discovered to be very important in enabling early control of dangerous central fires. More info can be seen in 04_Final Mission Strategy. All flights will be referred to by index and can be seen in the Mission Gallery

Lawnmower Path, Indices 0 and 1:

The first two official flights were performed using the "lawnmower" type sweeping path. This type of path seemed to make sense as a heat map of most dangerous cells strongly implies that the upwind cells need to be searched more quickly and thoroughly than downwind cells (as one might expect). However, post-flight analysis showed that this path was highly ineffective against central fires which were still more than dangerous enough to go out of control. The animations showing these two flights clearly illustrate this fact, as the airplane finds the fires far too late to mount an effective defense against them. This discovery led the team to develop a new search path

Linkin Park Path, Indices 2-15:

Rethinking the path, it was considered that as the fire spreads it will become easier to see. This implies that "perfect" coverage is both unnecessary and possibly counterproductive as it greatly slows the aircraft's overall search of the lake. Further, in testing it was found that turns in the early part of the search harmed climb rate significantly, and should be minimized. These two goals led to the development of a much more unconventional looking search path which does make its first turn upwind, but then takes a much more "random" seeming path from point to point on the map. This path has redundant coverage of the center, meaning that any fire which started upwind would be spotted before it crossed the center of the map. Comparing this path the the previous one it is clear that it typically has a much lower "time-to-spot" time than the naive downwind sweeping path would have had. In simulation the Linkin Park path regularly outperforms any of the sweeping paths that Spaero had developed. Typical time-to-spots were in the range of 1 minute, which was almost always early enough that if the algorithm performed properly it could easily control the fire it spotted.

However, not everything was working properly with the algorithm in these missions. Careful observation of the flights in the mission gallery show two critical problems:

• 1. The airplane does not always see the adjacent cells when an image is taken. This was a known issue that largely boiled down to being unable to guarantee position accuracy within 3 meters of a waypoint. Possible changes to the algorithm to mitigate the danger of this error include forcing the airplane to search downwind of the most downwind fire cell known, and search upwind of the most upwind fire cell.

  2. The airplane contains the wrong side of the fire! It can be seen that rather than extinguishing the fire-front n the downwind side of the fire, it extinguishes the front along the side 90 degree CCW to the downwind edge. This was simply an implementation mistake causing the algorithm to believe that "downwind" was the wrong direction. It is unclear how much the score could have been improved by fixing this bug, but qualitative analysis of the mission flight tracks seem to imply that substantial improvements to fire containment and control would have been made if it were fixed.

Additionally, even in the absence of these errors, there was an exceptionally wide disparity in results for the mission simulation for identical code between Windows and Linux operating systems.To illustrate, the code, run over 1000 samples, results in an average mission score of 99.8885 with 10.846 drops on Linux. On Windows, on the other hand, the average score was 81.3476 with 17.059 drops (noting that the aircraft was limited to 21 or less on a given mission). This is primarily due to the differences in the "rand()" function between code that is compiled on Windows vs. Linux, which tends to bias certain cells over others in the initial fire distribution.

Furthermore, hardware limitations proved significant in that, time to process information varied wildly throughout the mission; earlier tested flights showed that decision-making-time could take as long as 9.5 seconds; parts of the decision-making-algorithm were then ported over to the high-control loop (specifically, the code which checks whether the aircraft has reached a cell or not). The aircraft then circles the waypoint until it receives a new destination and flight mode from the algorithm. Changing this functionality improved speed considerably, although the aircraft was found to occasionally circle waypoints, causing it to lose valuable flight time. Time-to-process was not continuous, so it was impossible to model this discrepancy in the simulation.

Overall, in spite of these major shortcomings, performance was considered satisfactory. Clearly there is room for future improvement, but given the short time span the team is proud of the score and flight capabilities. The airframe itself seemed extremely robust and the control system performed admirably in real flights, and the advanced containment algorithm was (nearly) ideally implemented. Considering the difficulty found in obtaining good performance in the embedded programming environment, this in itself was a significant accomplishment.

Previous: 04_Final Mission Strategy

Next: 06_Overall project review and Lessons Learned