Project
The scientific goals of PREMIER pertain to advances in the field of video forensics. Videos strongly contribute to the viral diffusion of information through web channels, and can play a fundamental role in the digital life of individuals and societies. The forensic analysis of videos therefore represents a crucial tool to preserve the dependability of digital media.
Although promising approaches have been explored in the latest years, research in video forensics is much less advanced than for still images and the proposed solutions suffer from several shortcomings. This includes the need of large amounts of data for training, the difficulty of interpreting the outcoming results, and the vulnerability to adversarial attacks, which particularly affect AI-based techniques employing deep neural networks.
PREMIER will pursue novel hybrid approaches whereby data-driven AI techniques are enriched with a model-based signal-processing view point, with the goal of obtaining forensic detectors that are more interpretable and secure. The research activity is organized in several workpackages, addressing the different project goals and closely interacting through the development of the project.
WP1: Model-based solutions for AI in data-limited conditions
The use of model-based priors is explored with the goal of relaxing the need of large amounts of training data and orienting the training process towards task-relevant features. This includes applying smart pre-processing and regularization strategies, and studying conditions of training/testing data mismatching.
WP2: Interpretability tools for AI-based detectors
AI-based detectors are enriched by tools for the visualization of the deep network outcomes, such as temporal and spatial heat-maps quantifying the importance of single pixels and areas. This eases the interpretation of the learning process and allows for the informed design of new architecture based on signal processing concepts.
WP3: Authenticity and integrity verification in adversarial settings
The security of AI-based detectors is assessed against malicious actions performed by attackers. Different threat models are analyzed, which define the possible interplay between attacker and defender. Countermeasures to be adopted either at training or inference phase are developed, with the goal of increasing security.
WP4: Data collection for authenticity and integrity verification of visual data
The need of representative data for training and testing the hybrid detectors is supported by a continuous collection of datasets composed of original and fake videos. This will aid the development of enhanced solutions and provide the scientific community with extensive data corpora for benchmarking.
WP5: Data collection for authenticity and integrity verification of visual data
A dissemination and raising awareness plan is implemented to promote the project activities among the scientific audience and the general public.
This includes holding presentations/demos at scientific conferences, co-organizing workshops and special sessions at established events, and co-editing peer review journals. Consortium members also regularly hold training lectures and participate in dissemination events and panels, recognized by local and national press. Moreover, a web-based demonstrator showcasing the developed forensic tools is designed throughout the project.