Research and campaign site for PhD Trygve Olav Fossum - Applied Underwater Robotic Laboratory (AURLab) - Norwegian University of Science and Technology (NTNU), Department of Marine Technology (IMT), Trondheim, Norway.
For citation of my work please visit my Google scholar profile. Some of my work is also available on Researchgate.
Researchgate Profile (with available full-texts)
Source code for our paper in Annals of Applied Statistics (MESLAS)
Last updated: March 2021
Seminar talk 2nd of February at the University of Washington and the Paul G. Allen school of computer science and engineering.
Here I present the work from our paper in Science Robotics and give a brief introduction to adaptive sampling, the challenges related to ocean sensing, and why robotics can help us improve our understanding of the changing oceans.
I recently published our work from Runde, which made the front cover of the journal! The work concerns adaptive robotic sampling of phytoplankton outside this Island called Runde, near the city of Ålesund (where they previously have held the world championship in underwater photography) . At Runde there has been a famous bird-mountain that is home to a number of different seabirds that forage on different fish in the ocean outside. However, during the last decades there has been a sharp decline in the number of birds. The decline is noticeable across the North Atlantic, and includes the well-known Puffin. So to better try to understand why this is happening, we tried to collect data across the whole ecosystem. From the seafloor, through the water column, all the way up to the birds and their behavior, making connections across different disciplines and sources of data.
Picture: The systems, scales, and platforms used in the field experience: remote sensing (Chlorophyll a), AUV (biological and physical measurements), and R/V Gunnerus (FRRf [fluorometry] and SilCam [particle camera]). Focusing in on a volume of water: starting at satellite observation, AUV mapping in 3D, and finally images and measurements of individual particles in the water-column, covering a unique range of scales in the coastal microbial community.
The specific contribution which is on the cover of Science Robotics, is concerned with the the mapping of the water column in this ecosystem, using robotic platforms. The water column is important because it is where the basis for the marine (and human) food web is situated. This basis arise from the fact that phytoplankton is produced in vast quantities here, based on nutrients and sunlight. The produced phytoplankton is eaten by zoo-plankton, which is eaten by fish, which is the main source of food for the birds. So mapping phytoplankton is important because it tells us what we are starting with (it is sort of the energy input from the sun gets transferred into eatable food), and if there is changes here, it will eventually be felt by the birds.
To get the right detail of this complex system, we needed to cover a range of different scales. So our engineer and science team collected data from large scale features using satellite and buoys, and combined them with more detailed and local data from ships, cameras, as well as autonomous robotic platforms. Specifically, we used a type of robot called an AUV, which stands for autonomous underwater vehicle. The AUV looks like a torpedo, but only it does only carry a payload of different types of environmental sensors (you can see pictures further down on this page). The AUV can move fast and can be programmed to “think for itself” using software algorithms that we developed, so that the system can make decisions based on what it observes during the mission. This way we are utilizing all the information that is available to us. I think it is important to note that mapping phytoplankton is very difficult, as it is very heterogeneously distributed; we call this heterogeneity "patchiness". The paper discusses a way to map this patchiness effectively in 3-dimensions using a statistical model of the plankton distribution, that updates during the mission, on which the subsequent data collection is planned on line inside the computer of the robot (usually referred to as adaptive sampling). The robot is not controlled, but makes these choices on line on its own (autonomy). Radiowaves can not propagate far under water, only a couple of meters depending on signal strength. So it has to be able to operate on its own (which is why they are called autonomous). We can, however, through acoustic follow its position and send simple commands such as "stop, come to the surface", "what is the distance from you to the boat".
The robots goal is to optimize its route to give the best map of the amount and distribution of phytoplankton back to the scientists, which are doing parallel sampling with other instruments on a nearby research vessel. One of these sensors is actually a camera that counts and detects individual plankton particles, telling us something about the creatures that live inside this patchiness.
Link to the paper here: http://robotics.sciencemag.org/content/4/27/eaav3041
Onboard the new R/V Kronprins Håkon, the objective of the cruise is to identify and quantify the processes that are important for the heat budget in northern Svalbard waters. Together with the University of Bergen, Univ. Centre of Svalbard, NTNU, the national Meterological Institue, IFERMER, and FOC (Canada), a large scale deployment of both buoys, gliders, models, and AUVs is conducted near the sea ice edge.
Picture: Picture of the survey area. This is the Northwestern side of Svalbard. The red track is the Seagilder sections. (*) Marks the mooring stations. Black lines are the survey lines for the ship R/V Kronprins Haakon.
We have developed a way to track these fronts using an adaptive sampling behavior based on temperature thresholds and hysteresis. A large challenge is the limited communications, which at these latitudes are limited to satellite (iridium). We are also operating close to the ice edge, making flexible and unknown sampling a problem as it is hard to foresee the sampling path of the AUV as this depends on data.
Picture: The AURLab team out at sea at 82 deg. north picking up the AUV (LAUV Harald) after a successful mission.
From the R/V Kronprins Haakon, we ventured out with the small RIB boat Polarsirkel. The ice edge is visible in the background. The fog is due to the cooling and freezing of seawater to ice.
Picture: The results from the front tracking algorithm together with the interpolated data from the AUV. The AUV can be seen crossing the front six times before heading back in the surface (straight southwesterly line).
The AUV did successfully follow the Arctic front between cold Arctic and warm Atlantic water masses, crossing the front six times. The mission took 4 hrs, and the AUV dived to a depth of 90m (yoyo pattern). Sharp temperature trends can be seen from the interpolated (kriged) temperature measurements.
As a part of the Hugin HUS 1000 collaboration between NTNU and FFI (Forsvarets Forsknings Institutt - Norwegian Defense Research Establishment) , the "Hugin Autonomy Project" aimed to strengthen the collaboration through development of software and hardware enabling adaptive mission execution onboard Hugin 1000 AUV. This was achieved by using an NVIDIA Jetson TX1 and a software bridge to the DUNE system (Pinto et al., 2012) and the TREX autonomy layer (Py et al., 2010; Rajan and Py 2012; Rajan et al., 2012). The bridge developed and tailored to work with DUNE and TREX by PhD Øystein Sture (www.ntnu.edu/employees/oystein.sture) with help from Martin Wiig (www.ntnu.edu/employees/wiig).
The autonomy tests were conducted in the Korsfjord outside Trondheim and aimed at tracking and mapping internal processes in the Fjord using CTD (conductivity, temperature and depth) measurements as input to an onboard adaptive algorithm, developed to focus sampling efforts to the most interesting cross-section of the Fjord. The autonomy tasks were controlled from TREX and executed through the DUNE-Hugin bridge.
Picture: Picture of adaptive TREX mission using Hugin 1000 AUV. The goal is to identify changes across the Korsfjord. Notice that the AUV turns to map the warmer influx of water.
Picture: FFI Hugin 1000 AUV with instruments exposed during field trials in the Trondheimsfjord.
From the FFI homepage: https://www.ffi.no/no/Forskningen/Avdeling-Maritime-systemer/hugin/Sider/default.aspx
"The HUGIN system was developed in a collaborative effort involving FFI, Kongsberg Maritime, Statoil, Norsk Undervannsintervensjon (NUI) and the Royal Norwegian Navy and is used by the offshore survey industry for detailed seabed mapping and data acquisition, and by navies for mine counter measures (MCM) and intelligence, surveillance and reconnaissance (ISR). Today there are 30 scientists and engineers at FFI involved in the effort of increasing the effectiveness and area of uses for the system."
Funded by the Research Council of Norway and the Centre of Autonomus Marine Operations and Systems (AMOS), the ENTiCE project (2016 - 2018) is led by Ingrid Ellingsen at SINTEF Ocean and is focused on using and improving ocean models in ocean sampling. Through collaboration between the Norwegian University of Science and Technology (NTNU), SINTEF Ocean, Scottish Association for Marine Science (SAMS) a unique mix of both oceanographers, biologist, and robotics came together to work on these problems.
As a part of the robotic team, our goal was do develop adaptive sampling strategies with AUV capable of focusing data collection to regions of high scientific interest.
A paper from this work can be found here: https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.21805
Deploying the AUV (LAUV Harald) from the Telemetron. The AUV is specially made for mapping the upper water column.
The resulting path selected and executed autonomously by the AUV itself. The AUV uses an onboard AI planner architecture (T-REX: Rajan, K. and Py, F. (2012). T-REX: Partitioned Inference for AUV Mission Control).
The ENTiCE team onboard the Gunnerus research vessel: (from left) Ingrid Ellingsen, Martin Ludvigsen, Kristine Steinhovden, Geir Johnsen, Zolt Volent, Glaucia Fragoso, Emlyn Davies, Trygve Olav Fossum (me), Øystein Sture, Kay Skappy and Joel Pederick.
Accumulation of phytoplankton is central to nutrient, carbon and energy cycling, as well as providing the basis for the marine --and human-- food chain. Hence, mapping and predicting these processes are of vital for understanding the governing mechanisms in the coastal ocean. Sampling these signatures is difficult due to a number of reasons. Phytoplankton is patchily distributed and may be present in very small vertical layers, or in bigger plumes. Conventional surveys from boats with nets and water sampling are bound to have their data affected by the patchiness and distribution. Hence, knowing where and when to sample is vital in order to have an accurate representation of the process taking place.
Using an Autonomous Underwater Vehicle (AUV) equipped with a CTD (conductivity, temperature, depth) sensor, and a optical ecological sensor dubbed 'ECOpuck', capable of directly measuring Chlorophyll concentration, we have developed a algorithm for detection, tracking and mapping of phytoplankton concentration. The goal of the adaptive algorithm is to provide information enabling more efficient measurement campaigns with costly sampling resources, such as ship based surveys. In experiments conducted in Ålesund, Rune, June 2017, the algorithm was used to autonomously detect and track the peak chlorophyll layer providing a distribution map of the concentration.
The project had a larger scope of mapping the entire ecosystem around the Runde island and was a collaboration between: Runde Miljøsenter, NTNU, Maritime Robotics, and SINTEF.
The AUV tracks chlorophyll plumes using Gaussian Processes and spatial interpolation.
Deployment of the AUV Fridtjof. In addition to tracking chlorophyll, seabed mapping using side scan sonar was conducted in collabortation with Kartverket (https://www.kartverket.no/) .
