Embedded Files
Diffusion-based Video Inpainting
Diffusion-based Video Inpainting
The goal of this project, which is is the PhD work of Nicolas Cherel, in collaboration between Télécom Paris (Yann Gousseau, Alasdair Newson) and Université Paris Cité (Andrés Almansa) is to use diffusion models for video inpainting. It concerns the following publications :
Infusion: Internal Diffusion for Video Inpainting, Nicolas Cherel, Andrés Almansa, Yann Gousseau, Alasdair Newson, 2024, Preprint
Diffusion-Based Image Inpainting with Internal Learning, Nicolas Cherel, Andrés Almansa, Yann Gousseau, Alasdair Newson, EUSIPO 2024, Paper
For more information, see the webpage of Nicolas Cherel :
Patch-based Video Inpainting
Patch-based Video Inpainting
This was part of my PhD work, in collaboration between Technicolor (Matthieu Fradet, Patrick Pérez) and Télécom ParisTech (Andrés Almansa, Yann Gousseau). The goal is to use a patch-based approach to video inpainting. Here is an example of such an inpainting :
An input image frame
Inpainted image frame
Abstract
Abstract
We propose an automatic video inpainting algorithm which relies on the optimisation of a global, patch-based functional. Our algorithm is able to deal with a variety of challenging situations which naturally arise in video inpainting, such as the correct reconstruction of dynamic textures, multiple moving objects and moving background. Furthermore, we achieve this in an order of magnitude less execution time with respect to the state-of-the-art. We are also able to achieve good quality results on high definition videos. Finally, we provide specific algorithmic details to make implementation of our algorithm as easy as possible. The resulting algorithm requires no segmentation or manual input other than the definition of the inpainting mask, and can deal with a wider variety of situations than is handled by previous work.
Original project webpage
Citing this work
Citing this work
If you wish to use our work or code, please cite the following paper :
Video Inpainting of Complex Scenes
Alasdair Newson, Andrés Almansa, Matthieu Fradet, Yann Gousseau, Patrick Pérez
SIAM Journal of Imaging Science 2014 7:4, 1993-2019
You can download this paper here :
Video inpainting code
Video inpainting code
You can find the matlab code for this work here :
Video inpainting examples
Video inpainting examples
Input video - Fontaine, Chatelet
Download the input video :
Download the occlusion mask :
Fontaine chatelet occlusion, avi
Our inpainting result
Download our inpainting result. Please note that the result has been slowed down by a factor of two,
for better visualisation.
Input video - Les Loulous
Download the video :
Download the occlusion mask :
Les loulous occlusion mask, avi
Les loulous occlusion mask, .mat file
Our inpainting result
Download inpainting result. Please note that the result has been slowed down by a factor of two,
for better visualisation.
Les loulous - our inpainting result
Input video - Young Jaws
Download inpainting result. Please note that the result has been slowed down by a factor of two,
for better visualisation.
Input video - Museum
This example is from the work of Miguel Granados et al :
How not to be seen: Object removal from videos of crowded scenes
Miguel Granados, KwangIn Kim, James Tompkin, Oliver Grau, Jan Kautz and Christian Theobalt
Computer Graphics Forum (EUROGRAPHICS), 2012
To download the input and masks of this work, go to the following webpage :
How Not to be Seen, Inpainting - Miguel Granados
Here is a comparison of our result with that of Granados et. al. The result of Granados et al. is on the
top, our result is below.
Above, the result of Granados et al.
Below, our result.
Input video - Duo
This example is from the work of Miguel Granados et al :
How not to be seen: Object removal from videos of crowded scenes
Miguel Granados, KwangIn Kim, James Tompkin, Oliver Grau, Jan Kautz and Christian Theobalt
Computer Graphics Forum (EUROGRAPHICS), 2012
Page updated
Google Sites
Report abuse