Feature selection on brain voxel space using regularizations

Post date: Mar 2, 2013 8:53:25 AM

Summary

• Ridge regularization (l2-norm) --> the importance (feature weights) are shared among correlated features giving small weight to each of the feature.
• LASSO (l1-norm) --> too strict, the the importance to only one from a group of correlated features, hence gives very minimal number of voxels
• LASSO+Ridge regularization (elastic-net penalty) --> selected correlated voxels midway between l1 and l2 regularization. However, it does not necessarily select spatially correlated voxels which might be the case for fMRI data. This can be solved using fused LASSO regularization.
• fused LASSO --> constrain on spatial/local consistency of the coefficient --> bring adjacent voxels too
• graphical LASSO --> give a sparse graph of voxels, this is similar to graphical Gaussian model.

Good resources:

• Machine learning book by Kevin Murphy [book webpage], in chapter 13, explains very well about each type of sparse regularization. Corresponding Matlab codes (PMTL) are available online too, but doesn't have fused lasso though.
• UW's machine learning for Big Data slides on sparse regularization.
• fMRI-Based Image Reconstruction Using the Elastic Net [pdf] -- a very nice tutorial of regression in general very well explained. Show some examples of using elastic net.
• This Gaussian identity tutorial can be useful.
• Slides on family of LASSO by Tibshirani

Using LASSO, Elastic-net, fused LASSO, generalized sparse penalty

• Regularization Paths for Generalized Linear Models via Coordinate Descent [pdf] -- discussing mainly about the efficient implementation of elastic-net in the Glmnet toolbox (Matlab).
• Sparse logistic regression for whole-brain classification of fMRI data [url] ***-- This paper uses LR with elastic-net penalty to classify speech and music region in the whole brain.
• Interpretable Classifiers for fMRI Improve Prediction of Purchases [pdf] -- using logistic regression with elastic-net penalty as feature selection. They optimized over lambda and alpha.
• Estimation of functional connectivity in fMRI data using stability selection-based sparse partial correlation with elastic net penalty [pdf] -- Using elastic-net with sparse partial correlation (SPC) to detect brain connectivity.
• Constrained sparse functional connectivity networks for MCI classification [pdf] ***-- using l1-norm to find the sparse connectivity within a subject, then use l2-norm to make the non-zeros connectivities consistent across subject.
• Generalized Sparse Regularization with Application to fMRI Brain Decoding [pdf] -- using generalized sparse regularization to decode the brain.
• UNCOVERING SPARSE BRAIN EFFECTIVE CONNECTIVITY: A VOXEL-BASED APPROACH USING PENALIZED REGRESSION [pdf] -- the paper gave very good overview on several penalty functions, and proposed a more generalized penalty function by combining all the penaty functions together.
• LASSO logistic regression model [pdf]

Fused Lasso penalty

• Sparsity and smoothness via the fused lasso [pdf] -- the original fused Lasso paper.
• A coordinate-wise optimization algorithm for the Fused Lasso [pdf] -- Derive efficient fused lasso, and extend fused lasso to logistic regression
• good paper talking about fused LASSO

software package for fused Lasso

Tibshirani's fused lasso R package cghFLasso [url]

group fuse Lasso for signal segmentation GFLseg Matlab.

SLEP: A sparse learning package using Matlab with mex, with very good code documentation. This package include LR with fused Lasso!!!

===============================