Learning-based superresolution

under construction ...

Learning-based image upscaling and superresolution

•Super-resolution (SR): how to find missing details/HF comp?

•Interpolation-based:

•Edge-directed;

•B-spline;

•Sub-pixel alignment;

•Reconstruction-based:

•Gradient prior;

•TV (Total Variation);

•MRF (Markov Random Field).

•Learning-based (hallucination).

•Example-based: texture synthesis, LR-HR mapping;

•Self learning: sparse coding, self similarity-based;

•‘Deep Learning’ competes with shallow learning in image SR.

• Image Superresolution by Sparse Representation [Yang et al.'08, Zeyde et al.'10]

The SR algorithm in [Yang et al.'08]: Bi-cubic interpolation Sparse Coding SR

Flowchart of Sparse coding-based SR method in [Zeyde et al.'10]

•Image Super-resolution by Learning Deep CNN [Dong et al.'14]

•Learns an end-to-end mapping btw low/high-resolution images as a deep CNN from the LR image to the HR one;

•Learn a mapping F, consists of three operations:

•1. Patch extraction and representation;

•2. Non-linear mapping;

•3. Reconstruction.

•Sparse-coding-based SR viewed as a deep CNN, but handle each component separately, rather jointly optimizes all layers.

•LR image upscaled using bicubic interpolation as Y; then recover from Y an image F(Y) similar to ground truth HR image X.

Origin (640x360) Bicubic interpolation (720p) CNN learning-based SR (720p)

Origin (960x540) Bicubic interpolation (1080p) CNN learning-based SR (1080p)

Origin (960x540) Bicubic interpolation (1080p) CNN learning-based SR (1080p)

Origin (360x240) Bicubic interpolation (720x480) CNN learning-based SR (720x480)

Origin (950x600) Bicubic interpolation (1900x1200) CNN learning-based SR (1900x120)

Origin (681x384) Bicubic interpolation (1362x768) CNN learning-based SR (1362x768)

Origin (950x600) Bicubic interpolation (1900x1200) CNN learning-based SR (1900x1200)