HiFiC

Source: CLIC 2020 Dataset

What is HiFiC?

HiFiC stands for High Fidelity Compression. HiFiC is a neural network based approach to image compression and reconstruction. HiFiC uses GANs to reconstruct images, generating much higher quality images compared to Autoencoders, the simplest approach to image compression with machine learning.

Source: https://learnopencv.com/conditional-gan-cgan-in-pytorch-and-tensorflow/

What are GANs?

GAN stands for Generative Adversarial Network. A GAN is made of two neural networks, a generator, which generates new content/information that is similar to the input, and a discriminator, which attempts to discern the generated content from the real dataset. The generative network is trained to fool the discriminator.

A conditional GAN is able to conditionally generate content based on additional information.

HiFiC uses conditional GANs as a decoder, allowing it to generate much higher quality images compared to the Autoencoder and with very small encoded representation.

The Model

The HiFiC model is made of 4 parts: the encoder E, the generator G, the discriminator D, and the probability model P. The encoder here is similar to our autoencoder's, but also performs channel normalization. It effectivly downscales the image into a smaller tensor. The probability model is used to more efficiently store the output of the encoder. The probability model is able to losslessly store the output by combining machine learning techniques and arithmetic encoding, achiving a much smaller encoded representation of the encoder output. The generator takes the output of the encoder as input and is able to work together with the discriminator to "uncompress" the image.

Results

From the sample images in the interactive demo on the HiFiC website, we can see that HiFiC is able to completely outperform JPG when images are compressed to a similar compression ratio. At the same ratio, JPG has heavy image distortion and contains very present image artifacts. BPG, another image format, at the same ratio, performs much better than JPG, but is still outperformed by HiFiC in the user study performed in the HiFiC paper.

When observing HiFiC images against the original images very closley, we can observe that many textures in the image are not quite the same or are very different. This is because of the generative approach to image reconstruction, in that the generative model is creating or predicting the details of the image from training data. This issue is not apparent with traditional compression formats with sufficiently high reconstruction quality.

I encourage you to check out their website (linked at the bottom of the page) to check out this interactive demo.

Why We Can't Test it

HiFiC provides a nice Google Colab page to demo their trained model. However, we are unable to run their notebook because the model uses so much system ram, the session always crashes before the decompression can complete. We have tried this for both their default image and some of our metric dataset images. The limitation of not being able to decompress images pretty much prevents us from being able to compare the practical usage of HiFiC with our other compression algorithms. So, although HiFiC looks like a very promising compression tool, it is unfortunate that we do not have the system resources to run it.

Conclusion

HiFiC is a very competitive image compression method. It looks like it is able to outperform many popular lossy image formats. The main limitation with it is the large dependency on system resources. I was unable to run it with the 12 GB of system ram provided by the free version of Google Colab and do not have access to a computer strong enough anywhere else. The compression and decompression time is also heavily dependent on the GPU strength too, being a machine learning based tool, but is certainly takes much longer than jpeg to perform, taking near a minute to just compress an image.

HiFiC Website: https://hific.github.io/

HiFiC Paper: https://arxiv.org/abs/2006.09965

Page updated

Report abuse