During my Ph.D. thesis I developed methods to obtain spacetime (4D) reconstructions of ocean waves from stereo videos. The main purpose was to develop new instrumentation and remote sensing techniques to be able to measure and extract properties of the waves over large areas (instead of pointwise) and in a nonintrusive way.
Overview
In recent years, remote sensing imaging systems for the measurement of oceanic sea states have attracted renovated
attention. Imaging technology is economical, noninvasive and enables a better understanding of the spacetime
dynamics of ocean waves over an area rather than at selected point locations of previous monitoring methods
(buoys, wave gauges, etc.).
We present recent progress in spacetime measurement of ocean waves using stereo vision systems on offshore platforms, which focus on sea states with wavelengths in the range of 0.01 m to 1 m. Both traditional disparitybased systems and modern elevationbased ones are presented in a variational optimization framework: the main idea is to pose the stereoscopic reconstruction problem of the surface of the ocean in a variational setting and design an energy functional whose minimizer is the desired temporal sequence of wave heights. The functional combines photometric observations as well as spatial and temporal smoothness priors. Disparity methods estimate the disparity between images as an intermediate step toward retrieving the depth of the waves with respect to the cameras, whereas elevation methods estimate the ocean surface displacements directly in 3D space. Both techniques are used to measure ocean waves from real data collected at offshore platforms in the Black Sea (Crimean Peninsula, Ukraine) and the Northern Adriatic Sea (Venice coast, Italy). Then, the statistical and spectral properties of the resulting oberved waves are analyzed. We show the advantages and disadvantages of the presented stereo vision systems and discuss future lines of research to improve their performance in critical issues such as the robustness of the camera calibration in spite of undesired variations of the camera parameters or the processing time that it takes to retrieve ocean wave measurements from the stereo videos, which are very large datasets that need to be processed efficiently to be of practical usage. Multiresolution and shorttime approaches would improve efficiency and scalability of the techniques so that wave displacements are obtained in feasible times. References:
Joint surface (ocean wave) reconstruction and refinement of camera calibration Top left: synthetic ocean surface generated using CUDA SDK; Top right: simulated perturbations of the camera's extrinsic parameters (rotation and translation). Bottom left: joint ocean surface reconstruction and camera calibration refinement. Bottom right: ocean surface reconstruction without camera calibration refinement.
Validating modern oceanographic theories using models
produced through stereo computer vision principles has recently
emerged. Spacetime (4D) models of the ocean surface may be
generated by stacking a series of 3D reconstructions independently
generated for each time instant or, in a more robust manner,
by simultaneously processing several snapshots coherently
in a true “4D reconstruction.” However, the accuracy of these
computervisiongenerated models is subject to the estimations
of camera parameters, which may be corrupted under the influence
of natural factors such as wind and vibrations. Therefore,
removing the unpredictable errors of the camera parameters is
necessary for an accurate reconstruction. In this paper, we propose
a novel algorithm that can jointly perform a 4D reconstruction
as well as correct the camera parameter errors introduced
by external factors. The technique is founded upon variational
optimization methods to benefit from their numerous advantages:
continuity of the estimated surface in space and time, robustness,
and accuracy. The performance of the proposed algorithm is
tested using synthetic data produced through computer graphics
techniques, based on which the errors of the camera parameters
arising from natural factors can be simulated.
References:
Spacetime reconstruction of oceanic sea states via variational stereo methods
We present a remote sensing observational method for the measurement of the spatiotemporal dynamics of ocean waves. Variational techniques are used to recover a coherent spacetime reconstruction of oceanic sea states given stereo video imagery. The stereoscopic reconstruction problem is expressed in a variational optimization framework. There, we design an energy functional whose minimizer is the desired temporal sequence of wave heights. The functional combines photometric observations as well as spatial and temporal regularizers. A nested iterative scheme is devised to numerically solve, via 3D multigrid methods, the system of partial differential equations resulting from the optimality condition of the energy functional. The output of our method is the coherent, simultaneous estimation of the wave surface height and radiance at multiple snapshots. We demonstrate our algorithm on real data collected offshore. Statistical and spectral analysis are performed. Comparison with respect to an existing sequential method is analyzed.
References:
Incorporating Wave Height distribution Models in Variational stereo imaging of oceanic waves Surface reconstruction for a given time (snapshot). Top row: radiance and height functions, f(u) and Z(u), respectively. Middle row: predicted intensities in the region of interest using the generative scene model, superimposed on original images. Bottom row: absolute error images in the region of interest.
We develop an image processing observational technique for the stereoscopic reconstruction of the waveform of oceanic sea states that also incorporates the enforcement of any given statistical wave law modeling the quasiGaussianity of oceanic waves observed in nature. The problem is posed in a variational optimization framework, where the desired waveform is obtained as the minimizer of a cost functional that combines image observations, smoothness priors and a weak statistical constraint. The minimizer is obtained by combining gradient descent and multigrid methods on the necessary optimality equations of the cost functional. Robust photometric error criteria and a spatial intensity compensation model are also developed to improve the performance of the presented image matching strategy. The weak statistical constraint is thoroughly evaluated in combination with other elements presented to reconstruct and enforce constraints on experimental stereo data, demonstrating the improvement in the estimation of the observed ocean surface.
References:
A variational stereo method for the threedimensional reconstruction of ocean waves
We develop a novel remote sensing technique for the observation of waves
on the ocean surface. Our method infers the 3D waveform and radiance
of oceanic sea states via a variational stereo imagery formulation. In
this setting, the shape and radiance of the wave surface are given by
minimizers of a composite energy functional that combines a photometric
matching term along with regularization terms involving the smoothness
of the unknowns. The desired ocean surface shape and radiance are the
solution of a system of coupled partial differential equations derived
from the optimality conditions of the energy functional. The proposed
method is naturally extended to study the spatiotemporal dynamics of
ocean waves and applied to three sets of stereo video data. Statistical
and spectral analysis are carried out. Our results provide evidence that
the observed omnidirectional wavenumber spectrum S(k) decays as k^{2.5}
is in agreement with Zakharov's theory (1999). Furthermore, the 3D
spectrum of the reconstructed wave surface is exploited to estimate wave
dispersion and currents.
References:
Euler characteristics and maxima of oceanic sea states We present an application of a novel Variational Wave Acquisition Stereo System (VWASS) for the estimation of the wave surface height of oceanic sea states. Specifically, we show that VWASS video technology combined with statistical techniques based on Euler Characteristics of random fields provides a new paradigm for the prediction of wave extremes expected over a given area of the ocean. References:
Wave statistics and spectra via a variational wave acquisition stereo system We propose a novel variational Wave Acquisition Stereo System (WASS) that exploits new stereo reconstruction techniques for accurate estimates of the spatiotemporal dynamics of ocean waves. WASS has a significant advantage as a lowcost system in both installation and maintenance. A stereo camera view provides threedimensional data (both in space and time) whose statistical content is richer than that of a time series retrieved from wave gauges, ultrasonic instruments or buoys, the latter being expensive to install and maintain. Indeed, wave spectra can be easily estimated from the multidimensional images obtained with WASS. The estimated spectra present an inertial range that decays as k^{ 2.5}, k being the wave number, in agreement with wave turbulence theory (Zakharov 1999, SocquetJuglard et al. 2005). Further, the empirical probability density functions derived from the reconstructed surface data compare very well with theoretical models (Tayfun & Fedele 2007, Fedele 2008). The variational WASS is a promising technology with broader impacts in offshore engineering since it will enrich the understanding of the statistics of waves for an improved design of offshore structures. References:

Research >