Excerpts from the Book

"No time in human history has ever witnessed such explosive influence and impact of image processing on the modern societies, sciences, and technologies."

"The effciency and advantages of a particular methodology often depend on the concrete tasks at hand, as well as the classes and data structures of the images provided."

"The current book attempts to present most modern image processing approaches, and reveal their qualitative or quantitative connections."

[Curvature] "The mean curvature H has been frequently studied in dynamical processing of shapes and surfaces, mainly due to its variational meaning."

[BV (by Rudin-Osher-Fatemi)] "Functions with bounded variation (BV) are ideal deterministic image models which allow the existence of jumps or edges, and are however still mathematically tractable."

[Statistical Mechanics] "From Shannon's information and communication theory [ref] to the pioneering works of Geman and Geman [ref], Zhu, Mumford, and Wu [ref] on Gibbs's random fields in image modeling, it is unsurprising to see the growing importance of ideas and techniques from thermodynamics and statistical mechanics [ref]."

[Statistical Mechanics] "The fundamental principle of classical thermodynamics is that a system of many microscopic particles in equilibrium could be well described by only a few key macroscopic quantities,such as the total energy E, temperature T, pressure p, volume V , and entropy S, etc. (The usage of these standard symbols therefore will be kept consistently throughout this section.)

To the community of image and pattern analysis, the astounding similarity in terms of missions and goals can be instantly felt: we are also trying to compress the information of complex visual patterns and distributions down to a few visually crucial features. The technical resonance between the two fields has thus been profoundly driven by the characteristics they happen to share. "

[Bayesian] "Bayesian method has played a central role across the entire spectra of image processing and visual perception. See, for example, the monograph “Perception as Bayesian Inference ”edited by Knill and Richards [ref]."

[Filtering] "Mathematically, (low-pass) filtering is closely connected to the mollification operator in real analysis and the diffusion phenomenon in the PDE theory."

[Specialties of Images] "Compared with most acoustic or sound signals, images differ in that they are discontinuous functions. The discontinuities in 2-D images are commonly associated with the boundaries of objects in the 3-D world, and therefore intrinsic and visually important."

[Wavelets] "An individual wavelet, literally speaking, is a localized small wave. It acts as a virtual neuron that fires strongly whenever localized visual features are presented to it. Due to localization, it can respond strongly only when its window captures the target feature within.

Wavelets analysis studies how to design, organize, and analyze such wavelets, and achieve efficient computational schemes. A significant portion of its mission is to scientically model human and machine vision, and to effectively perform various image processing tasks."

[Multiscale/Multiresolution Analysis] "From the designs of Gabor to Malvar-Wilson, wavelets still live in the mighty shadow of Fourier frequency analysis. Historically, the situation did not make an energetic turn until Meyer and Mallat introduced the independent and general framework of multiresolution analysis (MRA)."

[Image Modeling] "The goal of image modeling or representation is to find proper ways to mathematically describe and analyze images."

[Images as Generalized Functions] "Treating images as distributions or generalized functions is the broadest approach for deterministic modeling [ref]. Though necessarily meaning less structures, such concepts do have profound merits in image understanding as explained below."

[Besov Images and Multiscale Structure] "As a multiscale tool, wavelets are particularly powerful for studying a class of images known as Besov images, whose multiscale nature is intrinsic from their definitions."

[Images as Gibbs' Ensembles] "Even at the ensemble level, unlike the deterministic view, different ensembles of images unnecessarily have to carry clear cut decision boundaries. There do exist certain image samples that may look like both grass images and sand beach images."

[Visual Equilibrium of Two Images] "On the other hand, unlike statistical mechanics where equilibria (e.g., thermal, mechanical, or chemical) are naturally defined through physical contact (e.g., via thermal contact, a mechanical piston, or a permeable membrane), the definition of the equilibrium of two portions of an image or two separate images is much less obvious. What seems natural for image and vision analysis is that the notion of equilibrium must be in the sense of visual discrimination. Intuitively speaking, two images are said to be in (visual) equilibrium if one cannot distinguish one from the other when having them placed next to each other. "

[Images as Collections of Level Sets] "An image as a function can be understood as a collection of isophotes, or equivalently, level sets. This view leads to the level-set representation of images, and is closely connected to the celebrated level-set computational technology of Osher and Sethian [ref]."

[Mumford-Shah Segmentation] "As an inverse problem, segmentation is invariably achieved by two essentially equivalent methods: statistical estimation via maximum likelihood (ML) or maximum a posteriori probabilities (MAP), and deterministic estimation via energy based variational optimization. The celebrated Mumford-Shah segmentation model [ref] belongs to the latter category, and is closely inspired by and connected to earlier statistical models, for example, by Geman and Geman [ref], and Blake and Zisserman [ref]. "

[Don't Hate Noise] "Noise is ubiquitous, noisy, but not always annoying.

Noises or fluctuations are often not as bad as their names might suggest. In equilibrium thermodynamics as well as statistical mechanics, noise is the key ingredient underlying the Second Law (i.e., Law of Maximum Entropy), and is crucial for maintaining the stability of large systems."

[White Noise] "The first popular noise model is called white noise, as inspired by the notion of white color. A shade of white color results from approximately equal mixture of different visible color spectra, typically ranging from 400nm to 700nm."

[Stochastic PDE] "One important class of generative models for random signals are stochastic differential equations (SDE) [ref]. Many random physical signals can be simulated using SDE's."

[Wiener Filtering] "Ideally the (above) filter should be instead learned from or driven by the given image data, which leads to another historically important class of filters proposed by Norbert Wiener [ref]. "

[Wavelet Shrinkage of Donoho and Johnstone] "The intuition behind wavelets-shrinkage based signal and image denoising is as follows. Under Besov norms (Section 3.3.3), the magnitudes of wavelet coefficients are directly proportional to the irregularity of a given image. When noises are involved, such irregularity grows in the wavelet coefficients. Thus by properly separating such irregular growth due to noises from the wavelet coefficients, the goal of denoising can be naturally achieved."

[Perona-Malik Nonlinear Diffusion and filtering] "Linear diffusions or linear scale spaces (see Witkin [ref]) in the form of [eqn] unavoidably smear sharp edges embedded in u0 while filtering out noises. To remedy this shortcoming, Perona and Malik in their seminal paper [ref] allowed the diffusivity coefficient D to be adapted to the image itself, instead of being prefixed and uncorrelated: D = D(x; u; grad u): "

[Origins of Blurs] "There are three major categories of blurs according to their physical background: optical, mechanical, and medium-induced. "

[Nature of Deblurring] "Deblurring is a backward diffusion process and decreases the entropy [and hence is an ill-posed problem]."

[Hidden Symmetries of Double-BV Blind Deblurring] "First we show that there are several hidden symmetries in the double-BV deblurring model [eqn], as stated in the next three theorems. Such symmetries, as in many other areas of mathematics, could lead to the nonuniqueness of solutions."

[Interpolation and Inpainting] "Interpolation has been a significant topic in a number of areas including numerical analysis, computational PDE's, approximation theory, real, complex, and harmonic analysis, as well as signal processing. "

[Radial Basis Functions] "Radial basis functions are popular interpolants for interpolating scattered spatial data, such as in image processing, computer graphics, and statistical data analysis [ref]. "

[Complexity of Image Interpolation] "Compared with all the aforementioned classical interpolation problems, the main challenges of 2-D image inpainting or interpolation lie in three aspects: (A) domain complexity, (B) image complexity, and (C) pattern complexity."

[Geometric Image Inpainting] "The performance of geometric inpainting models crucially depends on what types of geometric information are incorporated and how they are actually integrated into the models."

[Image Segmentation] "Image segmentation is the bridge between low-level vision/image processing and high-level vision. Its goal is to partition a given image into a collection of “objects,” built upon which other high-level tasks such as object detection, recognition, and tracking can be further performed."

[Occlusion] "In human and computer vision, the occlusion phenomenon plays a key role in the successful retrieval of 3-D structural information from 2-D images projected onto the retinas."

[Active Contours Based on Features] However, visual identification of vague edges or boundaries has to rely on certain recognizable features. Therefore, we will generally refer to such models as Active Contour models driven by features. Of course image gradient is a special example of image features. Below we will make the term “feature” more specific and stochastic analysis shall be the main powerful tool. "

[Geman-Geman's Image Model] "It is a hidden Markov model that defines images as random intensity fields regulated by their hidden edge patterns."

[Asymptotics of the Mumford-Shah Model] "In the following three subsections, we shall consider separately three asymptotic limits of the Mumford-Shah model, all of which are important for image processing and often rediscovered later independently by other researchers in different contexts."

[Phase-Field Approximation of Length] "The main idea of Gamma-convergence approximation [to the Mumford-Shah model] is to encode the 1-D edge feature by a 2-D edge signature function."

September, 2005