EQSST (Estimation of Quality Scores from Subjective Tests), per-subject results Dataset

The dataset contains per-subject assessment results for the subjective test results presented in the GamingVideoSet dataset. Beyond the dataset, the SAS code used in the paper is also provided.

The dataset is available for non-commercial purposes.

Introduction to EQSST

Subjective tests for the assessment of the quality of experience (QoE) are typically run with a pool of subjects providing their opinion scores using a 5-point scale. The subjects’ mean opinion score (MOS) is generally assumed as the best estimation of the average score in the target population. Indeed, for a large enough sample, we may assume that the mean of the variations across the subjects approaches zero, but this is not the case for the limited number of subjects typically considered in subjective tests. In this paper, we propose an approach based on generalized linear models (GLMs) for the estimation of the population average QoE. The motivating dataset is composed of the individual scores assigned by 25 subjects to a set of gaming videos evaluated under different resolutions and compression ratios. The approach recognizes the multinomial nature of the data and allows for correlation between scores of the same subject. The resulting estimated average QoE is shown to follow more credible patterns than the MOS, particularly for higher bitrates, for which the model estimates present more coherent behavior. Similar convincing results are found on a second dataset, showing the validity of the approach.

Download 

Dataset 

Paper [KU Repository] [IEEE]

Citation

If you use this dataset in your work, kindly cite our paper:

S. Pezzulli, M. G. Martini and N. Barman, "Estimation of Quality Scores From Subjective Tests-Beyond Subjects’ MOS," in IEEE Transactions on Multimedia, vol. 23, pp. 2505-2519, 2021, doi: 10.1109/TMM.2020.3013349.