Xiongkuo Min's Homepage

Unified Blind Quality Assessment of Compressed

Natural, Graphic, and Screen Content Images

Xiongkuo Min^1,2, Kede Ma², Ke Gu³, Guangtao Zhai¹, Zhou Wang², Weisi Lin⁴

¹Shanghai Jiao Tong University ²University of Waterloo ³Beijing University of Technology ⁴Nanyang Technological University

Abstract

Digital images in the real-world are created by a variety of means and have diverse properties. A photographical natural scene image (NSI) may exhibit substantially different characteristics from a computer graphic image (CGI) or a screen content image (SCI). This casts major challenges to objective image quality assessment, for which existing approaches lack effective mechanisms to capture such content type variations, and thus are difficult to generalize from one type to another. To tackle this problem, we first construct a cross-content-type (CCT) database which contains 1,320 distorted NSIs, CGIs, and SCIs, compressed using the high efficiency video coding (HEVC) intra coding method and the screen content compression (SCC) extension of HEVC. We then carry out a subjective experiment on the database in a well-controlled laboratory environment. Moreover, we propose a unified content-type adaptive (UCA) blind IQA model that is applicable across content types. A key step in UCA is to incorporate the variations of human perceptual characteristics in viewing different content types through a multi-scale weighting framework. This leads to superior performance on the constructed CCT database. UCA is training-free, implying strong generalizability. To verify this, we test UCA on other databases containing JPEG, MPEG-2, H.264 and HEVC compressed images/videos, and observe that it consistently achieves competitive performance.

Database Description

The CCT database consists of 3 image content types: natural scene image (NSI), computer graphic image (CGI), and screen content image (SCI), with 24 reference images of each type. Two types of distortions are introduced, i.e., high efficiency video coding (HEVC) intra coding method and the screen content compression (SCC) extension of HEVC. Each distortion consists of 11 compression levels. There are a total of 24×11×1=264, 24×11×2=528, and 24×11×2=528 distorted NSIs, CGIs, and SCIs, respectively. NSIs are only compressed by HEVC. We carry out a subjective experiment on the database in a well-controlled laboratory environment. Each image is rated by 20 valid subjects. Difference mean opinion scores (DMOSs) are provided as the ground-truth quality for all distorted images.

Algorithm Description

We propose a unified content-type adaptive (UCA) blind IQA model that is applicable across content types. UCA incorporates the variations of human perceptual characteristics in viewing different content types through a multi-scale weighting framework. A diagram of the proposed UCA method is shown in Fig. 2. UCA is composed of two main steps: a feature extraction process executed at multiple scales and an adaptive multi-scale weighting process that pools the results into a single quality score.