Motivation and a bit of history:
Gonzalo was an undergraduate student in Electronics Engineering at the Universidad de Concepción and also a football fan. While looking for a capstone work project, he talked to me that stark shadows annoyed him during live football broadcasting. Gonzalo had a point!
Stadium infrastructure, such as the canopies and illumination towers, project shadows on the pitch, obscuring parts of it sometimes almost completely. This phenomenon mainly occurs when matches are broadcast without overhead lights, and as a result, it is difficult to distinguish a play or which player is carrying the ball.
The first two videos show the results of our stark shadow removal algorithm. For comparison, the original raw video is shown on the left-hand side, and the corrected video is shown on the right-hand side of the images. The third video shows the working FPGA implementation of stark shadow removal in football video streams. The left screen shows the raw video, the right screen shows the corrected video, and the FPGA board streaming the raw video and performing the correction is shown in the middle of the screen.
Rationale and scientific results:
In brief, the removal of stark shadows in visible images is a problem that can be tackled using digital image/video processing and low-light illumination techniques. When images or parts of the scene exhibit low-light conditions, the acquired images often suffer from low visibility because they have a high dynamic range, i.e., images have simultaneously areas of very low and very high luminance. A camera tries to solve this problem by balancing the light levels at an intermediate point, but such a solution is not always practical.
Using our image processing and hardware design expertise, we developed an algorithm and a special-purpose hardware architecture, which analyzes and corrects the images. We replace the color information with a representation of luminance, where a shadow is simply the absence of light. We generate a mask of pixels that isolates the shadow areas, correct their luminance levels on a pixel-by-pixel basis, and merge the result with the original image. Our prototype can process multiple 1920x1080-pixel (Full HD) video streams at 60 frames per second with a delay of only microseconds.
We have published our results in a paper called "A hardware architecture for real-time shadow removal in high-contrast video," P. Verdugo, M. Figueroa, and J. E. Pezoa. In Proc. of SPIE Optical Engineering + Applications, San Diego, CA, USA, 6-10, August 2017.
Other applications:
We have tested our system on football broadcasts. However, we genuinely believe there is a significant business opportunity in security cameras. They are always in the same position and, during the day, different parts of the scene are illuminated, and others are in the shade. This system can significantly improve those images. We think that the correction could be carried out inside the camera or in the DVR equipment that records the images.
Credits:
This project is an ongoing work that has been carried out the undergraduate Electronics Engineer students Gonzalo Fernandez and Pablo Verdugo, and supervised by Drs. Jorge Pezoa and Miguel Figueroa, at the Electrical Engineering Department, Universidad de Concepción, in Chile.
Videos were provided by Canal del Fútbol (CDF) in Chile.