The latest advances and integration of several key technologies such as wireless communications, low-power sensing, embedded systems, Internet protocols and cloud computing, have enabled the emergence of the Internet of Things (IoT) paradigm. However, the ever-growing deployment of the visual sensing applications within IoT deployments already strains the network and cloud infrastructures used to deliver and store massive amounts of visual data. Along with conventional image and video content, we are currently witnessing a surge in 360^o video traffic originating from virtual reality, augmented reality as well as interactive multiview video which significantly contribute to the volume of communicated data. At the same time novel sensing paradigms emerge which depart from the conventional frame-based sensing. A prominent example are neuromorphic visual sensors, a.k.a. dynamic vision sensors (DVS), which have been produced in the last five years. Instead of the conventional raster scan of video cameras, DVS devices record pixel coordinates and timestamps of reflectance events in an asynchronous manner, thereby offering substantial improvements in sampling speed and power consumption. In order to accommodate for the surge in the visual content communicated within the context of IoT applications and deal with emerging types of visual data, appropriate transmission and storage mechanisms need to be developed that would take advantage of the visual data properties to achieve even higher bandwidth, power and storage efficiency. The ENVISION project aims at developing such data-driven delivery and storage algorithms based on advanced coding techniques for data acquired by both conventional frame-based video cameras and DVS devices. Specifically, ENVISION pursues the following interconnected research objectives: (i) design of advanced content-driven delivery mechanisms for the transmission of the visual content captured by both neuromorphic and conventional visual sensors to the cloud service under bandwidth and power constraints, (ii) development of novel data-driven methods for storage of the visual content under the cost-performance optimization framework, and (iii) design of low-complexity encoding/decoding techniques to robustly compress the visual information for storage and transmission.

This research is funded by the European Union Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 750254.

Publications

Preprints

• E. Bourtsoulatze and D. Gunduz, "Cache-Aided Interactive Multiview Video Streaming in Small Cell Wireless Networks," Apr. 2018 [arXiv]

Peer-reviewed Journal Papers

• Y. Bi, A. Chadha, A. Abbas, E. Bourtsoulatze and Y. Andreopoulos, "Graph-based Spatio-Temporal Feature Learning for Neuromorphic Vision Sensing," IEEE Trans. on Image Processing, vol. 29, pp. 9084 - 9098, Sept. 2019 [arXiv] [PDF]

• P. Maniotis, E. Bourtsoulatze, and N.Thomos, "Tile-Based Joint Caching and Delivery of 360^o Videos in Heterogeneous Networks," IEEE Trans. on Multimedia, vol. 22, no. 9, pp. 2382 - 2395, Sept. 2020 [arXiv][PDF]

• E. Bourtsoulatze, D. Burth Kurka, and D. Gunduz, "Deep Joint Source-Channel Coding for Wireless Image Transmission," IEEE Trans. on Cognitive Communications and Networking, vol. 5, no. 3, pp. 567 - 579, Sept. 2019 [arXiv][PDF]

Peer-reviewed Conference Papers

• Y. Bi, A. Chadha, A. Abbas, E. Bourtsoulatze and Y. Andreopoulos, "Graph-Based Object Classification for Neuromorphic Vision Sensing," in Proc. of IEEE Intl. Conf. on Computer Vision (ICCV'19), Seoul, Korea, Oct. 2019 (25% acceptance rate) [arXiv]

• P. Maniotis, E. Bourtsoulatze and N. Thomos, "Tile-Based Joint Caching and Delivery of 360^o Videos in Heterogeneous Networks," in Proc. of IEEE Intl. Workshop on Multimedia Signal Processing (MMSP'19), Kuala Lumpur, Malaysia, Sept. 2019 [PDF]

• E. Bourtsoulatze, D. Burth Kurka and D. Gunduz, "Deep Joint Source-Channel Coding for Wireless Image Transmission," in Proc. of the IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP'19), Brighton, United Kingdom, May 2019 [PDF]

• E. Bourtsoulatze and D. Gunduz, "Cache-Aided Interactive Multiview Video Streaming in Small Cell Wireless Networks," in Proc. of IEEE Intl. Symp. on Personal, Indoor and Mobile Radio Communications (PIMRC'19), Bologna, Italy, Sept. 2018 [PDF]