Space-Time Reconstruction of Ocean Waves

During my Ph.D. thesis I developed methods to obtain space-time (4-D) reconstructions of ocean waves from stereo videos. The main purpose was to develop new instrumentation and remote sensing techniques to be able to measure and extract properties of the waves over large areas (instead of point-wise) and in a non-intrusive way.

Overview

In recent years, remote sensing imaging systems for the measurement of oceanic sea states have attracted renovated attention. Imaging technology is economical, non-invasive and enables a better understanding of the space-time dynamics of ocean waves over an area rather than at selected point locations of previous monitoring methods (buoys, wave gauges, etc.).

We present recent progress in space-time measurement of ocean waves using stereo vision systems on offshore platforms, which focus on sea states with wavelengths in the range of 0.01 m to 1 m. Both traditional disparity-based systems and modern elevation-based ones are presented in a variational optimization framework: the main idea is to pose the stereoscopic reconstruction problem of the surface of the ocean in a variational setting and design an energy functional whose minimizer is the desired temporal sequence of wave heights. The functional combines photometric observations as well as spatial and temporal smoothness priors. Disparity methods estimate the disparity between images as an intermediate step toward retrieving the depth of the waves with respect to the cameras, whereas elevation methods estimate the ocean surface displacements directly in 3-D space.

Both techniques are used to measure ocean waves from real data collected at offshore platforms in the Black Sea (Crimean Peninsula, Ukraine) and the Northern Adriatic Sea (Venice coast, Italy). Then, the statistical and spectral properties of the resulting oberved waves are analyzed. We show the advantages and disadvantages of the presented stereo vision systems and discuss future lines of research to improve their performance in critical issues such as the robustness of the camera calibration in spite of undesired variations of the camera parameters or the processing time that it takes to retrieve ocean wave measurements from the stereo videos, which are very large datasets that need to be processed efficiently to be of practical usage. Multiresolution and short-time approaches would improve efficiency and scalability of the techniques so that wave displacements are obtained in feasible times.

References:

Joint Surface (Ocean Wave) Reconstruction and Refinement of Camera Calibration

Top left: synthetic ocean surface generated using CUDA SDK; Top right: simulated perturbations of the camera's extrinsic parameters (rotation and translation). Bottom left: joint ocean surface reconstruction and camera calibration refinement. Bottom right: ocean surface reconstruction without camera calibration refinement.

Validating modern oceanographic theories using models produced through stereo computer vision principles has recently emerged. Space-time (4-D) models of the ocean surface may be generated by stacking a series of 3-D reconstructions independently generated for each time instant or, in a more robust manner, by simultaneously processing several snapshots coherently in a true “4-D reconstruction.” However, the accuracy of these computer-vision-generated models is subject to the estimations of camera parameters, which may be corrupted under the influence of natural factors such as wind and vibrations. Therefore, removing the unpredictable errors of the camera parameters is necessary for an accurate reconstruction. In this paper, we propose a novel algorithm that can jointly perform a 4-D reconstruction as well as correct the camera parameter errors introduced by external factors. The technique is founded upon variational optimization methods to benefit from their numerous advantages: continuity of the estimated surface in space and time, robustness, and accuracy. The performance of the proposed algorithm is tested using synthetic data produced through computer graphics techniques, based on which the errors of the camera parameters arising from natural factors can be simulated.

References:

Space-Time Reconstruction of Oceanic Sea States via Variational Stereo Methods

We present a remote sensing observational method for the measurement of the spatio-temporal dynamics of ocean waves. Variational techniques are used to recover a coherent space-time reconstruction of oceanic sea states given stereo video imagery. The stereoscopic reconstruction problem is expressed in a variational optimization framework. There, we design an energy functional whose minimizer is the desired temporal sequence of wave heights. The functional combines photometric observations as well as spatial and temporal regularizers. A nested iterative scheme is devised to numerically solve, via 3-D multigrid methods, the system of partial differential equations resulting from the optimality condition of the energy functional. The output of our method is the coherent, simultaneous estimation of the wave surface height and radiance at multiple snapshots. We demonstrate our algorithm on real data collected off-shore. Statistical and spectral analysis are performed. Comparison with respect to an existing sequential method is analyzed.

References:

Incorporating Wave Height Distribution Models in Variational Stereo Imaging of Oceanic Waves

Surface reconstruction for a given time (snapshot). Top row: radiance and height functions, f(u) and Z(u), respectively. Middle row: predicted intensities in the region of interest using the generative scene model, superimposed on original images. Bottom row: absolute error images in the region of interest.

We develop an image processing observational technique for the stereoscopic reconstruction of the waveform of oceanic sea states that also incorporates the enforcement of any given statistical wave law modeling the quasi-Gaussianity of oceanic waves observed in nature. The problem is posed in a variational optimization framework, where the desired waveform is obtained as the minimizer of a cost functional that combines image observations, smoothness priors and a weak statistical constraint. The minimizer is obtained by combining gradient descent and multigrid methods on the necessary optimality equations of the cost functional. Robust photometric error criteria and a spatial intensity compensation model are also developed to improve the performance of the presented image matching strategy. The weak statistical constraint is thoroughly evaluated in combination with other elements presented to reconstruct and enforce constraints on experimental stereo data, demonstrating the improvement in the estimation of the observed ocean surface.

References:

A Variational Stereo Method for the Three-Dimensional Reconstruction of Ocean Waves

We develop a novel remote sensing technique for the observation of waves on the ocean surface. Our method infers the 3-D waveform and radiance of oceanic sea states via a variational stereo imagery formulation. In this setting, the shape and radiance of the wave surface are given by minimizers of a composite energy functional that combines a photometric matching term along with regularization terms involving the smoothness of the unknowns. The desired ocean surface shape and radiance are the solution of a system of coupled partial differential equations derived from the optimality conditions of the energy functional. The proposed method is naturally extended to study the spatio-temporal dynamics of ocean waves and applied to three sets of stereo video data. Statistical and spectral analysis are carried out. Our results provide evidence that the observed omnidirectional wavenumber spectrum S(k) decays as k^-2.5 is in agreement with Zakharov's theory (1999). Furthermore, the 3-D spectrum of the reconstructed wave surface is exploited to estimate wave dispersion and currents.

References:

Euler Characteristics and Maxima of Oceanic Sea States

We present an application of a novel Variational Wave Acquisition Stereo System (VWASS) for the estimation of the wave surface height of oceanic sea states. Specifically, we show that VWASS video technology combined with statistical techniques based on Euler Characteristics of random fields provides a new paradigm for the prediction of wave extremes expected over a given area of the ocean.

References:

Wave Statistics and Spectra via a Variational Wave Acquisition Stereo System

We propose a novel variational Wave Acquisition Stereo System (WASS) that exploits new stereo reconstruction techniques for accurate estimates of the spatio-temporal dynamics of ocean waves. WASS has a significant advantage as a low-cost system in both installation and maintenance. A stereo camera view provides three-dimensional data (both in space and time) whose statistical content is richer than that of a time series retrieved from wave gauges, ultrasonic instruments or buoys, the latter being expensive to install and maintain. Indeed, wave spectra can be easily estimated from the multi-dimensional images obtained with WASS. The estimated spectra present an inertial range that decays as k ^-2.5, k being the wave number, in agreement with wave turbulence theory (Zakharov 1999, Socquet-Juglard et al. 2005). Further, the empirical probability density functions derived from the reconstructed surface data compare very well with theoretical models (Tayfun & Fedele 2007, Fedele 2008). The variational WASS is a promising technology with broader impacts in offshore engineering since it will enrich the understanding of the statistics of waves for an improved design of offshore structures.

References: