Overview

Author: Thierry BOUWMANS, Associate Professor, Lab. MIA, Univ.  Rochelle, France.

A full overview of the background subtraction methods listed in this website are provided in:




Editors: T. Bouwmans, F. Porikli, B. Hörferlin, A. Vacavant.

Title: Handbook “Background modeling and Foreground Detection for video surveillance:  Traditional and Recent Approaches, Benchmarking and Evaluation".
Publisher :CRC Press, Taylor and Francis Group.

Publication Date :  July 1, 2014. (More information) [Purchase]







Further Improvements

If you would like to list your publication related to this topic on this website, please send me your publication in .pdf and I will add the reference.

Fair Use Policy


As this website gives many information that come from my research, please cite my following survey papers:
 
T. Bouwmans, “Traditional and Recent Approaches in Background Modeling for Foreground Detection: An Overview”, Computer Science Review, 2014. [pdf]

T. Bouwmans, E. Zahzah, “Robust PCA via Principal Component Pursuit: A Review for a Comparative Evaluation in Video Surveillance”, Special Issue on Background Models Challenge, Computer Vision and Image Understanding, CVIU 2014, Volume 122, pages 22–34, May 2014. [pdf]

T. Bouwmans, "Recent Advanced Statistical Background Modeling for Foreground Detection: A Systematic Survey",
Recent Patents on Computer Science, Volume 4, No. 3, pages147-176, September 2011.
[pdf]
 
My recent publications are available on Academia, Research Gate, Researchr, ORCID and Publication List.

Objective


The aim of this web site is to provide ressources such as references (1500 papers), datasets (25 datasets), codes (60 codes) and links to demonstration websites (60 websites)  for the research on background subtraction by grouping all related researches and particularly recent advances in this field. For this, it  is organized in the following sections:

    1. Background Modeling (1544 papers):
This section groups all the references that I found on background modeling. These references are listed in the                                                                              following categories:
  • Traditional Background Modeling (786 papers)
        Basic Background Modeling (35 papers),  Statistical Background Modeling (548 papers), Background Modeling via Clustering (103 papers),
       
Neural Network Background Modeling (75 papers), Background Estimation (25 papers).

  • Recent Background Modeling  (727 papers)
        Advanced Background Modeling (28 papers), Advanced Statistical Background Modeling (55 papers), Fuzzy Background Modeling (20 papers),
        Background Modeling via Deep Learning (33 papers), Background Modeling via Circulant Structures (2 papers),
       
Background Modeling via Discriminative Subspace Learning (3 papers), Background Modeling via Mixed Subspace Learning (1 paper),
        Background Modeling via Kernel Subspace Learning (1 paper), Background Modeling via RSL (2 papers),
        Background via other Robust Subspace Models (4 papers), Background Modeling by RPCA (342 papers),  Background Modeling via RNMF (6 papers),
        Background Modeling via RMC (18 papers),
Background Modeling via other Robust Subspace Recovery (6 papers)
       
Background Modeling via Subspace Tracking (17 papers), Background Modeling via Multiple Subspace Tracking (4)
        Background Modeling via RLRM (34 papers), Sparse Background Modeling (64 papers),
       
Background Modeling via RTSL (1 paper), Background Modeling via RPCA Tensors (35 papers), Background Modeling via RNMF Tensors (1 paper)
       
Background Modeling via RTC (6 papers), Background Modeling via other Robust Tensor Recovery (3 papers),
        Background Modeling via Tensor Subspace Tracking (1 paper), Background Modeling via LRT (3 papers),
        Background Modeling via Subspace Clustering (1 paper),  Transform Domain Background Modeling (36 papers)
  • Prospective Background Modeling (31 papers)
        Pixel Intensity Classification  (PIC) (5 papers), Pixel Change Classification (PCC) (1 paper)
        Co-occurrence (CO) (9 papers), Pursuing Dynamic Spatio-Temporals Models (STDM) (3 paper),
        Projection Pursuit Density Estimation  (PPD) (1 paper), Sample-based Models (1 paper),
        Confidence Measure Background Modeling (3 papers) , Linear Dynamical System (1 paper),
        Nested Models (1 paper), Polynomial Models (4 papers), PID Models (1 paper), Imbalanced Learning Models (1 paper)

    2. Background Initialization (76 papers):
This section groups all the references that I found on background initialization.
These references are listed in the                                                                        following categories:

      Temporal Statistics (TS) (17 papers), Subsequences of Stable Intensity (SSI) (19 papers),  Iterative Model Completion (IMC) (13 papers),
      Optimal Labeling (OL) (2 papers), Missing Data Reconstruction (MDR) (7 papers),Neural Networks (NN) (17 papers), Image Fusion (IF) (1 paper)

    3. Background Maintenance (14 papers): This section groups all the references that I found on background maintenance. These references are listed in the                                                                     following categories:

         
Maintenance rules (2 papers), Learning rates (9 papers), Adaptation rates (3 papers)

    4.
Foreground Detection (31 papers)

       4.1 Foreground Detection (24 papers):  This section groups all the references that I found on foreground detection. These references are listed in the
                                                                following categories:


       4.2 . Post-Processing (7 papers)        
        
        Basic Post-processing (1 paper), Statistical Post-processing (1 paper), Fuzzy Post-processing (5 papers)

    5. Features (455 papers):This section groups all the references that I found on the different features used for background modeling.

        5.1 Feature Size (37 papers)

              Pixel, Superpixel (9 papers), Block  (2 papers), Cluster (4 papers), Region (9 papers), Brick (5 papers), Patch (4 papers), Semantic (4 papers)

        5.2 Feature Type  (346 papers)
          
               5.2.1 Features in the Pixel Domain (311 papers)

               a) Crisp Features (262 papers)

              Intensity Features (38 papers), Color Features in Well Known Color Spaces (17 papers), Color Features in Designed Shape Color Spaces (13 papers)
              Edge Features (14 papers), Textures Features (125 papers), Motion Features (2 papers),  Stereo Features (37 papers)

               b) Statistical Features (25 papers)

               Histograms of color space (9 papers), Histograms of Gradient  (7 papers),
Histograms of Figure/Ground Segmentations (HFG) (1 paper) 
              
Entropy (4 papers), Statistical Reach Feature (2 papers), Statistical Texture (2 papers)

               c) Fuzzy Features (13 papers)

               Local Fuzzy Color Histograms (LFCH) (6 papers), Local Fuzzy Color Difference Histograms (LFCDH) (1 paper), Fuzzy Histon (FH) (2 paper),                                Fuzzy Correlogram (FC) (2 papers), Fuzzy Statistical Texture (FST) (2 papers)

               d) Other Features  (11 papers)

               Phase features (3 papers),  Spatial features (1 paper), Location features (7 papers)

               5.2.2 Features in a Transform Domain (29 papers)

               Frequency Features (19 papers), Video Features (10 papers)

              5.2.3.  Features in Deep Learning Domain (6 papers)

               Deep Learned Features (6 papers)

       5.3 Feature Strategies (57 papers)

       5.4 Feature Selection (9 papers)

       5.5 Feature Reliability (4 papers)

       5.6 Feature Relevance (2 papers)

    6. Strategies (143 papers): This section groups all the references that I found on the different strategies used in background modeling.

        6.1 Multiple Strategies (116 papers)

         Multi-Features (57 papers), Multi-Scales (6 papers), Multi-Levels (11 papers), Multi-Resolutions (4 papers),
         Multi-Backgrounds (19 papers), Multi-Layers (10 papers), Multi-Classifiers (3 papers), Multi-Cues (5 papers),
         Multi-Criteria (1 paper)

       6.2 Map Strategies (2 papers)

        Map Selection (2 papers)

       6.3 Hierarchical Strategies (9 papers)

        Hierarchical Strategies (9 papers)

       6.4 Coarse to Fine Strategies (3 papers)

        Coarse to Fine Strategies (3 papers)

       6.5 Compositional Strategies (1 paper)

        Compositional Strategies (1 paper)

       6.6 Consensus Strategies (3 papers)

        Consensus Strategies (3 papers)

       6.7 Markov Random Fields (9 papers)
              
        Markov Random Fields (9 papers)

      
  
7. Performance Evaluation (20 papers):  This section groups all the references that I found on the different method to evaluate background modeling methods.

       Background subtraction (18 papers), Background initialization (2 papers)


Additional Informations

In addition of the previous sections, I give references and links to:
  • Available datasets (33 datasets)
  • Results on the Wallflower dataset for statistical background modeling
         First Generation , Second Generation, Third Generation
  • Available implementations (76 implementations):
       1) Traditional Background Modeling (18 implementations)

           Basic Background Modeling (2), Statistical Background Modeling (11), Background Modeling via Clustering (2),
           Neural Network Background Modeling (3), Background Estimation (0)

       2) Recent Background Modeling (47 implementations)

           Advanced Background Modeling (3),  Advanced Statistical Background Modeling (2), Deep Learning Bakground Modeling (6)
           Fuzzy Background Modeling (1), Background Modeling via RSL (1),
           Background Modeling by Robust PCA (18), Background Modeling via RNMF (2),           Background Modeling via Subspace Tracking (5) , Background Modeling via LRM (4),
          
Sparse Background Modeling (2), Transform Domain Background Modeling (0),
                                                       

       3) Others Background Modeling (3 implementations)
        
          
Others Background Modeling (3 implementations)

       4) Background Initialization (5 implementations)

           Background Initialization (5 implementations)
      
      
5) Background Maintenance

       
       6) Foreground Detection (3 implementations)


         Foreground Detection (3 implementations)
       
      
7) Background Modeling - Codes :  Background Modeling - Packages (3 packages) -   Background Modeling - Librairies (2 librairies)

  • Real Time Implementations (89  papers):

          GPU Implementations (30 papers), Embedded Implementations (19 papers), Architecture Implementations (36 papers),

          Parallel implementations (4 papers)

  • Sensors (83 papers):
         Cameras (21 papers), Infrared Cameras (23 papers), Multispectral Cameras (1 paper), RGB-D Cameras (34 papers), Light Field Cameras (4 papers)
  • Journals - Conferences - Workshops
        This page listes Journals (7), Conferences (16), Workshops (5) where recent advances on background subtraction are mainly published.
  • Websites (62 Websites)

       1) Traditional Background Modeling (13 websites):

            Basic Background Modeling (0), Statistical Background Modeling (9), Background Modeling via Clustering (1),
            Neural Network Background Modeling (2), Background Estimation (1)


       2) Recent Background Modeling (35 websites)

           Advanced Statistical Background Modeling (2), Fuzzy Background Modeling (1),

           Background Modeling via Discriminative Subspace Learning (1) , Background Modeling via Mixed Subspace Learning (0)

           Background Modeling via Kernel Subspace Learning (0), Background Modeling via RSL (1),

           Background Modeling by Robust PCA (19), Background Modeling via RNMF (0),

           Background Modeling via RMC (1), Background Modeling via other Robust Subspace Recovery (0)

           Background Modeling via Subspace Tracking (4),  Background Modeling via LRM (4),

           Sparse Background Modeling (2), Transform Domain Background Modeling (0).

       3) Others Background Modeling (9 websites):

            Others Background Modeling (9)

       4) Background Initialization: Background Initialization (4)
     
       5) Background Maintenance

      
       6)
Foreground Detection:
Foreground Detection (3)
     
       7) Performance Evaluation: Performance Evaluation (1)     
  • Applications (84 papers) :
           Intelligent Video Surveillance (20 papers), Intelligent Visual Observation of Animals and Insects (15 papers), 
           Intelligent Surveillance of Natural Environments (21 papers), Optical Motion Capture (7 papers), Human Computer Interaction (1 paper),        
          
Content based Video Coding (20 papers)

  • Extension to moving cameras (46 papers)

Freely Moving Cameras (6 papers), Online Moving Cameras (2 papers), Moving Cameras (21 papers), Hand-held Cameras (1 paper),

Mobile Cameras (1 paper), UAV Cameras (1 paper), PTZ Cameras (14 papers)

  • Patents (1 patent)
Note: The number in parenthesis gives the total amount of papers or links for the related categories.

Success Story

You can see the activity of the Background Subtraction Web Site at this page Google Analytics.