|
Research Description
A new technique for film grain noise extraction, modeling and synthesis is proposed and applied to high definition video coding. Film grain noise boosts the natural appearance of pictures in high definition video and should be preserved in coded video. However, the coding of video contents with film grain noise is expensive. In this work, we extract film grain noise from the input video as a pre-processing step (at the encoder) and re-synthesize the film grain noise and add it back to the decoded video as a post-processing step (at the decoder) as shown the below figure.
Fig. 1. The overview of the proposed film grain noise processin system
Under this framework, the coding gain of the denoised video is higher while the quality of the final reconstructed video is still high. To implement such a scheme, we develop a technique to remove film grain noise from image/video without distorting its original content as well as a parametric model consisting of a small set of parameters to represent the extracted film grain noise. This model generates film grain noise close to the real one in terms of power spectral density and cross-channel spectral correlation.
Resuts
Basically, we have tested our algorithm with three 1920x1080 HD sequences as shown in Fig. 2. Each sequence has different film grain noise characteristics and different type of contents. For example, 'Rolling tomatoes' sequence has lots of smooth regions, 'Playing cards' sequence is mostly consists of textured region, and 'Old town cross' has both smoothed sky and textured buildings so that they provide a set of good test examples to evaluate our proposed scheme.
a) Rolling tomatoes
b) Playing cards
c) Old town crossFig. 2. The first frames of HD test sequences.
Due to its huge size, only clipped sequcnes(or images) are provided in the below link. When you see the still image results, our proposed denoising algorithm effectively suppress the film grain noise, and our synthesis algorithm effectively generate 'look-like' film grain. When comparing to the generally decoded image with the denoised image with synthesized noise, the latter is more realistic and has less coding artifacts.
Fig. 3. The zoom-up image results of rolling tomatoes sequences
a) Original image
c) Decoded image(QP=24)
without pre-, post-processing
b) Denoised image (by TV method)
d) Decoded image(QP=24)
with denoising + grain synthesisIt is more obvious to see the video results for the superioty of the proposed film grain desnoing modeling algorithm. Due to the limitation of the web space, we only upload the cropped image with 30 frames, and encode each sequence with very high quality. In the results below, the final synthesized sequence is sum of decoded denoised image with QP=24 and synthesized film grain as Fig. 3-d).
'Rolling tomatoes' sequence : 'Playing cards' sequence :
'Old town cross' sequence
original-denoised original-synthesized decoded-synthesized
Besides of visual improvement as shown above, another goal of this research is reducing the bit-rate with suppressing film grain noise. We also experiment the sequence with chaging QP parameters. The following figures clearly show that proposed denoising method dramatically increase the coding gain.Fig. 4. The zoom-up image results of rolling tomatoes sequences
a) Rolling tomatoes
b) Playing cards
c) Old town cross
Reference
B. T. Oh, S. Lei and C.-C. Kuo, “Film grain noise analysis and synthesis for high definition video coding”, IEEE Trans. on Circuit and System for Video Technology, accepted. B. T. Oh, S. Sun, S. Lei and C.-C. Kuo, “Film grain noise modeling in advanced video coding”, Proceedings of SPIE, Visual Communications and Image Processing, 2007. Question
Any question? Please contact to btoh77@gmail.com