Midterm Update

Project Interim Report

Activity Summary

Ben has focused on understanding the inner workings of AV1 for his RS4. His paper is about the pitfalls other researchers made when testing AV1, the technical improvements and explanation of the features of AV1 compared to VP9, and some basic quality comparisons of AV1 to H.265, VP9 and VVC. He has also been looking into using some open-source implementations for the AV1 and H.266 Video Codecs for testing.

Nghia has been researching specifically into the differences in efficiency and performance between H.264, H.265 and AV1 video codecs in low-bandwidth environments. In particular, based on both academic research papers and technical blog posts, he has come to a conclusion that AV1 might be the most ideal candidate to target low-bandwidth requirements, and it is currently being used at the most prominent technology such as Google, Meta and Netflix in their video streaming applications. However, H.266, a recent video codec that has just come out in 2020, needs to be compared with AV1 to find out which fares better, and this will be his goal towards the next phase of the project.

Scope Change

As per the result of the research papers and technical blog posts, it seems that there is a clear win of AV1 against both H.264 and H.265 video codecs under low-bandwidth environments, Being a relatively new coding format, AV1 quickly rises to its fame by being used in a lot of prominent technology companies. Two years after AV1 came out, in 2020, H.266 was introduced. There is no existing research comparing H.266 and AV1 directly in performing under low-bandwidth requirements. Therefore, it is unclear who is the ideal candidate to be used in video streaming in classroom settings. As a result, we propose to narrow the scope to particularly comparing these two codecs instead of focusing on a variety of codecs as outlined in the project proposal. Moreover, subjective analysis, which was not mentioned the in preliminary project proposal, will need to be included in the project as well.

Project Direction

As agreed by both members of the group, the project direction will be shifted to testing H.266 and AV1 with the dataset being our own samples of recordings in the school environment. Since there are already academic works comparing H.266 against H.265 and AV1 against H.265, we propose to use these two bodies of work and extrapolate our predictions of performance between AV1 and H.266. As a result, our project will be to both compare and verify the extrapolation that we have derived.

Obstacles

One of the most challenging obstacles for the project is the lack of a simple H.266 implementation to use. Currently, the easiest open-source implementation of H.266 can be found at https://github.com/fraunhoferhhi/vvenc, but there is a lot of setup required to create samples and run quality analysis.

The AV1 implementation is available from https://aomedia.googlesource.com/aom/ but this implementation also has a long setup process and requires special permission to upload new test data. However, Nghia’s RS4 paper provides a method and instructions on how to test AV1 by using the FFmpeg framework.

As brought up by Ben’s RS4, the settings of the codecs can change the performance outcome, so our project in an ideal case will compare the video codecs in the fairest manner. However, since each video codec requires weeks of research to have an intermediate level of understanding, we don’t have time to understand all the complexities of H.266. Since we don’t really know the details of H.266, we will rely on the default configurations to encode the videos as we don’t have the information to create a truly fair comparison.

Another obstacle that has been slowing down the research is the newness of H.266. Since it has not been widely adopted at this point, there have been very few academic papers doing thorough comparisons of it.

Going forward, we hope to figure out the H.266 implementation by the next update so we can start to run some tests.

Reference

[1] Yue Chen et al. 2020. An overview of coding tools in AV1: The first video codec from the Alliance for Open Media. APSIPA Transactions on Signal and Information Processing 9, 1 (2020). DOI:http://dx.doi.org/10.1017/atsip.2020.2

[2] Thorsten Laude, Yeremia Gunawan Adhisantoso, Jan Voges, Marco Munderloh, and Jörn Ostermann. 2019. A comprehensive video codec comparison. APSIPA Transactions on Signal and Information Processing 8, 1 (2019). DOI:http://dx.doi.org/10.1017/atsip.2019.23

[3] Janusz Klink. 2021. A method of codec comparison and selection for good quality video transmission over limited-bandwidth networks. Sensors 21, 13 (2021), 4589. DOI:http://dx.doi.org/10.3390/s21134589

[4] Vincent Tabora. 2021. Av1 - why it matters to streaming services. (November 2021). Retrieved November 6, 2022 from https://medium.com/hd-pro/av1-why-it-matters-to-streaming-services-f7aea24d347a

[5] Anon. 2020. Google duo improves video call quality in low bandwidth using AV1 codec technology. (April 2020). Retrieved November 6, 2022 from https://tech.hindustantimes.com/tech/news/google-duo-improves-video-call-quality-in-low-bandwidth-using-av1-codec-technology-story-BrSs0DnSRx7mpGrFgSkqOI.html

[6] Tung Nguyen and Detlev Marpe. 2021. Compression efficiency analysis of AV1, VVC, and HEVC for random access applications. APSIPA Transactions on Signal and Information Processing 10, 1 (2021). DOI:http://dx.doi.org/10.1017/atsip.2021.10