Removing or weighting outliers (5 papers)
1. Removing outliers (3 papers)
D. Geiger, R. Pereira, "The outlier process", IEEE Workshop on Neural Networks for Signal Processing", pages 61-69, 1991.
M. Black, A. Rangarajan, "On the unification of line processes, outlier rejection, and robust statistics with applications in early vision", International Journal on Computer Vision, Volume 25, No. 19, pages 57-92, 1996.
P. Filzmoser, R. Maronna, M. Werner, "Outlier identification in high dimensions", Technometrics, pages 1694-1711, 2008.
2. Weighting outliers (2 papers)
K. Gabriel, S. Zamir, "Lower rank approximation of matrices by least squares with any choice of weights", Technometrics, Volume 21, pages 489-498, 1979.
L. Delchambre, “Weighted principal component analysis: a weighted covariance eigen-decomposition approach”, Monthly Notices of the Royal Astronomical Society, pages 1-12, 2014.
Robust estimation of mean and covariance (28 papers)
1. Affine-equivariant M-estimators of scatter (7 papers) [Not robust to many outliers]
R. Maronna, "Robust M-Estimators of Multivariate Location and Scatter", The Annals of Statistics, Volume 4, pages 51-67, 1976.
N. Campell, "Robust Procedures in Multivariate Analysis I: Robust Covariance Estimations", Applied Statistics, Volume 29, pages 231-237, 1980.
P. Huber, "Robust Statistics", Wiley, New York, 1981.
F. Ruymgaart, "A robust principal analysis", Journal of Multivariate Analysis", pages 485-497, 1981.
S. Devlin, R. Gnanadesikan, J. Kettenring, "Robust estimation of dispersion matrices and principal components", Journal of the American Statistical Association, Volume 76, pages 354-362, 1981.
G. Boente, "Asymptotic theory for robust principal components", Journal of Multivariate Analysis, Volume 21, pages 67-78, 1987.
M. Locantore, J. Marron, D. Simpson, N. Tripoli, J. Zhang, K. Cohen, “Robust principal components for functional data”, pages 1-28, 1999.
2. Positive-breakdown estimators (20 papers) [More robust, but limited to small to moderate dimensions]
2.1 Minimum Covariance Determinant (MCD) (5 papers)
V. Todorov, N. Neykov, P. Neytchev, "Stability of (high-breakdown point) robust principal components analysis", COMPSTAT 1994, Volume 10, pages 90-92, 1994.
P. Rousseeuw, K. Van Driessen, "A fast algorithm for the minimum covariance determinant estimator", Technometrics, Volume 41, pages 212–223, 1999.
R. Maronna, R. Martin, V. Yohai, "Robust Statistics: Theory and methods", Wiley Series in Probability and Statistics, 2006.
P. Huber, E. Ronchetti, "Robust Statistics", Wiley Series in Probability and Statistics, 2009.
M. Hubert, M. Debruyne, “Minimum Covariance Determinant”, Wiley Interdisci-plinary Reviews: Computational Statistics, Wiley, 2010.
2.2 Minimum Vector Variance (MVV) (1 paper)
D. Herwindiati, S. Isa, "The New Measure of Robust Principal Component Analysis", International Conference on Electronic Engineering and Computing Technology , February 2010.
2.3 S-estimators (4 papers)
L. Davies, "Asymptotic Behavior of S-Estimators of Multivariate Location and Dispersion Matrices", The Annals of Statistics, Volume 15, pages 1269-1292, 1987.
P. Rousseeuw, A. Leroy, "Robust Regression and Outlier Detection", NewYork: Wiley, 1987.
R. Maronna, R. Martin, V. Yohai, "Robust Statistics: Theory and methods", Wiley Series in Probability and Statistics, 2006.
G. Boente, M. Barrera, "S-estimators for functional principal component analysis", Technical Report, 2013.
2.4 Minimum Volume Ellipsoid (MVE) estimator (2 papers)
R. Naga, G. Antille, "Stability of robust and non-robust principal component analysis", Computational Statistics and Data Analysis", Volume 10, pages 169-174, 1990.
R. Maronna, R. Martin, V. Yohai, "Robust Statistics: Theory and methods", Wiley Series in Probability and Statistics, 2006.
2.5 Stahel-Donoho estimator (2 papers)
M. Debruyne, M. Hubert, “The influence function of Stahel-Donoho type methods for robust PCA” , 2005.
R. Maronna, R. Martin, V. Yohai, "Robust Statistics: Theory and methods", Wiley Series in Probability and Statistics, 2006.
2.6 Concentration Steps (CS) (1 paper)
P. Rousseeuw, K. Van Driessen, “An algorithm for positive-breakdown methods based on concentration steps”, Data Analysis: Scientific Modeling and Practical Application, Springer-Verlag, pages 335-346, 2000.
2.7 M-estimator (5 papers)
Geometric Median Subspace (GMS)
T. Zhang, G. Lerman, “A novel M-Estimator for robust PCA”, Journal of Machine Learning Research, 2014.
M. Coudron, G. Lerman, "On the sample complexity of robust PCA", Neural Information Processing Systems, NIPS 2012, 2012.
Extended Geometric Median Subspace (EGMS)
T. Zhang, G. Lerman, “A novel M-Estimator for robust PCA”, Journal of Machine Learning Research, 2014.
M. Coudron, G. Lerman, "On the sample complexity of robust PCA", Neural Information Processing Systems, NIPS 2012, 2012.
Fast Median Subspace (FMS)
G. Lerman, T. Maunu, “Fast, robust and nonconvex subspace recovery”, Information and Inference, 2017.
3. Orthogonalized Gnanadesikan-Kettenring (OGK) (1 paper)
R. Maronna, R. Zamar, "Robust estimates of location and dispersion for high dimensional data sets", Technometrics, Volume 43, pages 307-317, 2002.
4. Kronecker Spatio-Temporal Covariances (3 papers)
Kronecker PCA (KronPCA)
K. Greenewald, T. Tsiligkaridis, A. Hero, “Kronecker sum decompositions of space-time data", IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2013, pages 65–68, 2013.
K. Greenewald, A. Hero, “Kronecker PCA based spatio-temporal modeling of video for dismount classification", SPIE, 2014.
Diagonally Loaded KronPCA (DL-KronPCA)
K. Greenewald, A. Hero, "Robust Kronecker Product PCA for Spatio-Temporal Covariance Estimation", IEEE Transactions on Image Processing, Volume 63, No. 23, pages 6368 - 6378, 2015.
Projection Pursuit Techniques (9 papers)
L. Ammann, "Robust singular value decompositions: A new approach to projection pursuit", Journal of the American Statistical Association, pages 505–514, 1993.
C. Croux, A. Ruiz-Gazen, "A Fast Algorithm for Robust Principal Components based on Projection Pursuit", COMPSTAT 1996, pages 211-217, 1996.
M. Hubert, P. Rousseeuw, S. Verboven, "A fast method for robust principal components with applications to chemometrics", 2001.
C. Croux, A. Ruiz-Gazen, "High breakdown estimators for principle components: the projection-pursuit approach revisited", Journal of Multivariate Analysis", Volume 95, pages 206-226, 2005.
C. Croux, P. Filzmoser, M. Oliveira, "Algorithms for projection-pursuit robust principal component analysis", Chemometrics and Intelligent Laboratory Systems", Volume 87, pages 218-225, 2007.
M. Hubert, P. Rousseeuw, S. Verboven, "A Fast Method for Robust Principal Components with Applications to Chemometrics", Chemometrics and Intelligent Laboratory Systems, Volume 60, pages 101-111, 2002.
L. Bali, G. Boente, D. Tyler, J. Wang, "Robust functional principal components: a projection-pursuit approach", Annals of Statistics, Volume 39, pages 2852-2882, 2011.
N. Kwak, "Principal component analysis based on l1-norm maximization", IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 30, No. 9, pages 1672–1680, 2008.
M. McCoy , J. Tropp, "Two proposals for robust PCA using semidefinite programming", Electronic Journal of Statistics, pages 1123–1160, 2011.
Mixed techniques (5 papers)
ROBPCA [Row-wise outliers]
M. Hubert, P. Rousseeuw, K. Vanden Branden, "ROBPCA: A New Approach to Robust Principal Component Analysis", Technometrics, Volume 47, No. 1, pages 64-49, February 2005.
M. Hubert, P. Rousseeuw , S. Van Aelst, "High-breakdown robust multivariate methods", Statistical Science, Volume 23, pages 92-119, 2008.
M. Debruyne, M. Hubert, “The influence function of Stahel-Donoho type methods for robust PCA” , 2005.
MROBPCA [Row-wise outliers] [missing data]
S. Serneels, "Principal component analysis for data containing outliers and missing elements", Computational Statistics and Data Analysis, Volume 52, pages 1712-1727, 2008.
MacroPCA [Cell-wise outliers][Row-wise outliers] [missing data]
M. Hubert, P. Rousseeuw, "MacroPCA: An all-in-one PCA method allowing for missing values as well as cellwise and rowwise outliers", 2018.
Robust error estimation (6 papers)
P. Verboon, W. Heiser, "Resistant lower-rank approximation of matrices by iterative majorization". Computational Statistics and Data Analysis, Volume 18, pages 457-467, 1994.
R. Maronna, V. Yohai, "Robust lower-rank approximation of data matrices with element-wise contamination", Technometrics, Volume 50, pages 295-304, 2008.
S. Brubaker, "Robust PCA and clustering in noisy mixtures", ACM-SIAM Symposium on Discrete Algorithms, SODA 2009, pages 1078–1087, Philadelphia, USA, 2009.
H. Xu, C. Caramanis, S. Mannor, "Principal component analysis with contaminated data: The high dimensional case", COLT 2010, pages 490–502, 2010.
RSL (2 papers)
F. De La Torre, M. Black, “Robust Principal Component Analysis for Computer Vision”, International Conference on Computer Vision, ICCV 2001, Vancouver, Canada, July 2001.
F. De La Torre, M. Black, “A framework for robust subspace learning”, International Journal on Computer Vision, pages 117-142, 2003.
Regression-type Optimization Problem (6 papers)
1. Least Trimmed Squares (LTS) (3 papers)
L. Liu, L. Hawkins, D. Ghosh, S. Young, "Robust singular value decomposition analysis of microarray data, National Academy of Sciences of the USA, Volume 100, pages 167-172, 2003.
M. Wolbers, “Linear unmixing of multivariate observations”, PhD thesis, 2002.
R. Maronna, “ Principal components and orthogonal regression based on robust scales”, Technometrics, 2005.
2. Least Absolute Deviation (LAD) (3 papers)
LAD (1 paper)
D. Hawkins, L. Liu, S. Young, "Robust singular value decomposition", National Institute of Statistical Sciences Technical Report 122, 2001.
Weighted LAD (2 papers)
C. Croux, P. Filzmoser, G. Pison, P. Rousseeuw, "Fitting multiplicative models by robust alternating regressions", Statistics and Computing, Volume 13, pages 23-36, 2003.
K. Jung, "Robust Singular Value Decomposition based on Weighted Least Absolute Deviation Regression", Communications of the Korean Statistical Society, Volume 17, No. 6, pages 803-810, 2010.
lp minimization (18 papers) [Outliers][High dimension]
1. l1 minimization
G. Lerman, T. Zhang, "lp-Recovery of the most significant subspace among multiple subspaces with outliers”, Constructive Approximation, Volume 40, No. 3, pages 329–385, 2014.
S. Kundu, P. Markopoulos, D. Pados, "Fast computation of the l1-principal component of real-valued data", ICASSP 2014, pages 8028–8032, 2014.
P. Markopoulos, G. Karystinos, D. Pados, "Optimal algorithms for L1-subspace signal processing", IEEE Transactions on Signal Processing, Volume. 62, No. 19, pages 5046–5058, 2014.
N. Tsagkarakis, P. Markopoulos, D. Pados, "Direction finding by complex l1-principal-component analysis" IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAW 2015, pages 475-479, 2015.
P. Markopoulos, S Kundu, S. Chamadia, D. Pados, "Efficient l1-norm principal-component analysis via bit flipping", Preprint, 2016.
S. Chamadia, D. Pados, "Outlier processing via l1- principal subspaces", Florida Artificial. Intelligence. Research Society, FLAIRS 2017, Florida, May 2017.
P. Markopoulos, D. Pados, G. Karystinos, M. Langberg, "l1-norm principal-component analysis in l2-norm-reduced-rank data subspaces", SPIE Defense and Commercial Sensing, DCS 2017, 2017.
N. Tsagkarakis, P.. Markopoulos, D. Pados, "L1-norm principal-component analysis of complex data", IEEE Transactions on Signal Processing, March 2018.
D. Chachlakis, P. Markopoulos, "Reduced-rank filtering with complex L1-norm principal-component analysis", IEEE Transactions on Signal Processing, 2018.
M. Dhanaraj, P. Markopoulos, "Novel algorithm for incremental L1-norm principal-component analysis", European Signal Processing Conference, EUSIPCO 2018, Rome, Italy, September 2018.
P. Markopoulos, S. Kundu, S. Chamadia, N. Tsagkarakis, D. Pados, "Outlier-resistant data processing with l1-norm principal component analysis", Chapter in Advances in Principal Component Analysis: Research and Development, Springer, 2018.
F. Nie, H. Huang, C. Ding, D. Luo, and H. Wang, “Robust principal component analysis with non-greedy l1-norm maximization", IJCAI 2011, pages 1433-1438, 2011.
J. Brooks, J. Dula, E. Boone, "A pure l1-norm principal component analysis", Computational Statistics & Data Analysis, Volume 61 pages 83-98, 2013.
R1-norm [Rotationally invariant]
D. Ding, D. Zhou, X. He, H. Zha, “R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization”, ACM, pages 281-288, 2006.
l1-norm [Rotationally invariant]
N. Kwak, "Principal component analysis based on l1-norm maximization", IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 30, No. 9, pages 1672–1680, 2008.
Half-Quadratic PCA (HQ-PCA) [Rotationally invariant]
R. He, B. Hu, W. Zheng, X. Kong, “Robust principal component analysis based on maximum correntropy criterion, “IEEE Transactions on Image Processing, pages 1485–1494, 2011.
2. lp minimization
N. Kwak, "Principal component analysis by lp-norm maximization"; IEEE Transactions on Cybernetics, Volume 44, No. 5, pages 594-609, 2014.
Z. Liang, S. Xia, Y. Zhou, L. Zhang, Y. Li, "Feature extraction based on lp-norm generalized principal component analysis", Pattern Recognition Letters, Volume 34, No.9, pages 1037-1045, 2013.
Maximum Entropy (2 papers)
R. He, B. Hu, X. Yuan, W. Zheng, "Principal component analysis based on non-parametric maximum entropy", Neurocomputing, Volume 73, pages 1840-1852, 2010
R. He, B.. Hu, W. Zheng, X. Kong, "Robust principal component analysis based on maximum correntropy criterion", IEEE Transactions on Image Processing, Volume 20, No. 6, pages 1485-1494, 2011.
Generalized Mean (1 paper)
J. Oh, N. Kwak, "Generalized mean for robust principal component analysis", Pattern Recognition, Volume 54, pages 116-127, 2016.
How many components k need to be selected? (3 papers)
Predicted Residual Error Sum of Squares (PRESS) (3 papers)
I. Jollife, "Principal Component Analysis", New York: Springer-Verlag, 2002.
R-PRESS
M. Hubert, S. Engelen, “Fast cross-validation for high-breakdown resampling algorithms for PCA, 2004.
S. Engelen, M. Hubert “Fast cross-validation for robust PCA”, Computational Statistics, Physica-Verlag, pages 989–996, 2004.