Project

*** Congratulations to the winners of the 2019 competition! ***

Videos can be found Here

Video Communication

The project is an open competition in which the winning team will get to keep their radios and interfaces.

Task:

Your task in the project will be to send the best quality video in 75 seconds.

• There are no general limits on the technique you choose to transmit. For example, it could be analog or digital. We recommend that you leverage Lab5 for your project, but you don't have to.

There are a few exceptions though:

• You cannot use existing implementations of compression algorithms.
• You cannot use existing implementations of communication techniques.

You can use:

• Existing implementations of entropy coding compression: Zip, gzip, huffman coding
• **NEW** You can use PyWavelet, or Python DCT -- but if you implement your own, it will count towards more score in your scope of project.
• Existing implementations of error correcting codes

Specs

• You have 75 seconds for pure data communication, if you use afsk1200 with no overhead, that's 11250 bytes total. If we assume overhead of 1/3 of the communication that leaves 7500 bytes. With 24 bits/pixel that is about 2500 color pixel you could transmit using lab5 material. This is the minimum quality we will accept, and you can only improve on that.
• For your demo, you will be given a video (Tiff stack). The video could be of arbitrary size. The video will be chosen on the day of the project and presented to the competitors only on that day on a usb stick -- or a link on the web.
• The files will be multi-page PNG, whose filenames are "[FILENAME].png". Your code should be able to read them once you copied to the appropriate directory
• The receiver should produce a video of the exact same size as the one transmitted (not necessarily the same quality though)
• We will provide example videos on this page for you to practice on

Minimum passing grade:

• Use the 1200b/s AFSK packet modem developed in labs 4,5.
• Load the video, find its dimensions: W * H * T
• Resample each color channel in space an time to dimensions for which W_d * H_d *T_d <= 2500, the total number of bytes you can send in 75s. Quantize the result to 8-bit unsigned integer (0-255)
• Convert the pixel data to binary array. Packetise the data. (including information on how to construct the size of the original video)
• Send the data using your modem -- receive with another PI.
• Convert from packets to pixel data, and reshape.
• Interpolate the video to its original size

** NEW** Day of the presentation:

• Groups are scheduled with 15min time slots -- please come at least 30min before your presentation.
• The next group to present will set up while the current group is presenting (double buffer) -- Your time to set up will be 15min before your presentation.
• You have 15min for presentartion and demo -- split your time wisely (at least 5-10 presentation 5-10 demo). Practice demo!!!!! practice presentation.
• For presentation: You must:
  • Describe your system, what you tried, and what you ended up using, what worked, what didn't work.
  • Describe each member contribution.
  • Show results for sample videos, with reported SNR for Andy, Dog, Simpson and Milky Way.
• You will show a demo of compressing a new image as described by the Specs above.

** NEW** Final submission

• Final submission is 05/15/19 11:55pm
• You will need to submit a single zip file EE123_groupName.zip:
  • Your presentation
  • Code
  • 3 min video describing your project, results and demo.

What you could work on (partial list…)

• Improve the modem:
  • Increasing the rate beyond 1200b/s (factor of 2 is quite achievable, and faster than that is possible but difficult)
  • Using a different modulation for faster rate. 🌶
  • Removing overheads in packet protocol (not too difficult), Removing packetization alltogeather. 🌶
  • Error correction -- for making communication more reliable at higher rate.

• Compression:
  • Downsampling of the image in space and time.
  • Color transformation and decimation -- Eyes are more sensitive to grayscale resolution, less for color.
  • Implementation of image based compression (like JPEG or JPEG 2000)
  • Implementation of video based compression (MPG), which leverage differences between frames
  • Implementation of video compression which uses motion vectors before differencing. 🌶
  • Non conventional compression: Like PCA, dictionary based. 🌶

• Post processing
  • Filtering to improve sharpness and reduce artifacts.

** NEW** QA

• Q: Can I use more than one radio, or RaspPi?
  • A: Yes, but... you only have 75 seconds to transmit, the data for reconstructed video should be on 1 pi immediately after transamission. You could transmit with 2 radios and recieve with the SDR, but this is 🌶🌶🌶🌶
• Q: Can I use other language than python?
  • Yes, but it needs to work on the RaspPi 🌶🌶

Evaluation:

• For each of the video below, you should have a result of a transmission and the resulting PSNR that you got. This should be ready at the time of the presentation. In addition, you will submit the resulting images on bcourses.
• At the demo, we will ask you to show us two transmissions. In addition, we will provide you with one or two new videos (TIFF stack format) to demo.

The video quality evaluation will be done by

• Qualitative assessment 1-10 by the staff of the class.
• Peak Signal-to-Noise Ratio measure (PSNR) which is defined as:

, where MAX is the maximum possible image pixel value (255 for 8bits) and MSE is the mean-square error, or can be expressed as:

This assumes RGB videos are compared, where each color channel has M-by-N pixels and the total number of frames is T.

* Make sure that the radio transmit timeout setting (TOT Menu + 9) is set to at least 90 seconds.

Resources:

• Here's a link to explanation how Jpeg works: Link
• ICIP tutorial on JPEG200 Link
• EE225B lecturenotes on JPEG2000 Link
• Slides on Video compression Link, and Link
• Slides on Video Standards Link
• Here's how to read an image in python:

• Here’s some helper functions that we wrote to help you do the project (Link), the usage are summarized below:

Example Videos: Right click and save to download

(If you see static photos, just click on the Animated PNG links to view the GIFs)

Example of down-sampled videos

These are the same video, down-sampled, bz2 compressed to fit the transmission bandwidth (<7500 bytes), and up-sampled back - this would be the minimum accepted quality without compression, improvement in communication and post processing