Demos & Notes

Demos

Demo 1: UPS Delivery [from ring.com]

To speed up the image processing, I resize the original cropped frames into 400*300. After rejoin them, we have the original video on the left. The right one is improved video. We can see that the edge of the white pillar is more sharper, the whole picture is less blur than the original one. The human face is more recognizable, also, the improved one restores more detail of the texture of clothes.

One essential problem of improving that has not solved is the blur causing by capture devices and first compression. Like when the people is moving, especially moving fast, the picture still very ambiguous. This problem is beyond the scope of this project.

Demo 2: Low Resolution Video [from YouTube]

This is an extension of the project: double scale the original video. The original video (320*240) is on the top, the left-bottom corner one is enlarged by zooming each frame directly, and the right-bottom corner one is improved and enlarged with neural network. It's not easy to figure out the difference between them, more modifying and debugging work need to be done.

Notes

Python 3.5 for all codes and environment configuration concerns two parts: ML part and I/O part.
- ML part: To meet the high computational demand, I configured the environment with Docker in Ubuntu and the training part will use GPU instead CPU to speed up the process.
- I/O part: The configuration of ML part is able to meet the demand.

Insignificant improvement with offered pre-trained models.
- Train neural network with frames from the video .

Training will take 10 more hours with 20 training samples and 20 epochs.
- Reduce sample amount and resize each sample to low resolution images.
- Reduce training epoch times.
Process frames with self-trained models has a poor output.
- Crop original frames into a 800*600 image, so that the circle ring bell mask can be cut out and leave the most important part. This operation influence the quality a lot in image processing.
- Add various light condition sample (including night, day time, sunny, cloudy, etc.) into the training sample.
- Modify each parameters.

Audio lost in processing
- Since OpenCV only processes video, the audio part is lost during the process. It's possible to solve audio problems with FFMpeg and other Python packages, and I might follow up in the future.

Fixed Image pre-process

Google Sites

Report abuse