These are combined notes and images pulled from
Tor Wager's lectures on fMRI data analysis on Coursera,
Natasha S. Hansen's 2013 "MRI Quality Control Workshop" slides presenting XNAT (open-source tools to stream-line database management and quality assurance checks of neuroimaging and related data developed by the Buckner Lab at Washington U, now at Harvard U)
MRIQC documentation, toolbox for data quality checks developed in Russell Poldrack's lab at Stanford U's Center for Reproducible Neuroscience (CRN).
the SPM community via the SPM list/forum
papers (REFS)
the whole wide world wide web (even if people think Google is evil, it does scour the whole wide WWW pretty well....)
There are some tools out there to facilitate data checking, e.g.
quick subject movement/spikes in signal
go through volumes in movie mode and look for "jumps" in images -- now possible in most MRI analysis toolboxes
SPM: Check reg > load all EPIs > right-click on one of the ortho views "Browse" > Select images again (might have to de-select and re-select, ensure you select all frames if 4-D). Player controls should appear at bottom with scroll bar (click on ">" to play movie).
To inspect Right-clicking on one of the ortho views and selecting "Browse" > "Display profile" will give raw time series at coordinates of the crosshairs.
AFNI: Load EPIs > "Graph" button > "v" to play, SPACE to stop
FSL's fsleyes: Load EPIs and click on the "Movie mode" (film) button in ortho toolbar (see intro on fsleyes)
auto-detecting movement/large spikes
Artifact Detection Tools (ART) for SPM and MATLAB has art_despike (voxelwise)
AFNI's 3dDespike (also voxelwise)
Rik Henson's spm5spike
general QA reports
MRIQC: automatic quality and visual reports from Russell Poldrack's group at Stanford U, which generates FSL-style clickable/interactive HTML reports with lots of image quality metrics based on the multicenter Connectome Project and International Neuroimaging Data-sharing Initiative (INDI)'s QAP package
model problems
Artifact Detection Tools (ART) for SPM and MATLAB has a group model check art_groupcheck which bases assessment on percent signal change and looking at model estimates (SPM's con*.nii) and estimated variance of model residuals (SPM's ResMS.nii).
MRIQC has an overall group report with image quality metrics but not really model diagnostics
... many more....
Some notes that explain what you are looking at with these tools and more on what to check below.
Ss = "subjects" (i.e. who/what you tested)
Check out Natasha S. Hansen's 2013 "MRI Quality Control Workshop" slides presenting XNAT which is filled with helpful illustrations of the problems below (which is mostly a summary of her slides....) as well as Paul Mazaika's slides on "Artifact detection and repair in fMRI" for his 2015 lecture at the UCLA Advanced NeuroImaging Summerschool (filled with informative illustrations, impact of artifacts on signal, and best practices on avoiding artifacts in the first place for experimenter and MRI setup). xrayphysics.com also shows some scanner-related and movement artifacts and their impact on the MRI and k-space.
brain orientation correct?
make sure left/right not flipped
how: occipital "L is larger" rule (Toga & Thompson 2003, Nat Rev Neuro)
software viewer should scroll in the ff manner:
sagittal: anterior to posterior
axial: top anterior, bottom posterior
brain image clipped
cause: FOV position wrong
can lead to wrapping
signal loss (dark spots)
ferromagnetic metal not removed leading to artefacts
Wrapping (reconstruction gone awry)?
if brain does not overlap, okay -- if overlaps, then image unuseable
signal loss/susceptibility distortion
ringing, striping, blurring in sMRI
causes
subject motion
some types due to image reconstruction probs (e.g. "shadowed arc artifact")
low-level ghosting
image reconstruction problem
radio frequency noise/spiking
cause: probs with scanner/head coil
signal inhomogeneity
cause: probs with scanner/head coil
motion slice artifact in BOLD
cause: subject motion
mean signal intensity
signal intensity standard deviation
voxel-based signal-to-noise ratio (vSNR)
slice-based signal-to-noise ratio (sSNR)
good >= 150, bad <= 99
alignment and scaling of img values across all Ss
plot spectrogram of vx by Ss, with color = global means
plot line graph of global mean vs. Ss -- look for outliers
outliers indicate task movement, physiological noise
check covariance matrix across all imgs across all Ss
if high correlation (whether pos/neg) => poss scaling issues (Ss not on same scale) or poss non-indep of Ss (both violations of GLM)
check for outliers in regions (extreme values across the dataset)
subject motion
relative motion
absolute motion
good <= 1.49 mm, bad >= 2mm
mean motion
maximum motion
# mvmts > x mm (e.g. 0.5mm, 1 mm)
good # < 5, bad # >=5
effect of mvmt on fc-MRI: van Dijk et al. 2012 NI
b. subject motion
i. relative motion, absolute motion
ii. mean motion, max motion
iii. # mvmts > x mm
Besides checking the effects of every pre-processing step (e.g. making sure realignment actually realigned the images -- check the major structures like brain edges, ventricle-tissue contours -- you would expect some differences in gyral alignment, but be more stringent in the areas you're interested in).
check group mean and group SD image (anat and fMRI) for high variation/bright bands (i.e. high SD) around edges of brain in group
cause: high SD might indicate bad realignment across Ss (maybe need to improve normalization across Ss)
Although we would never think of not doing model diagnostics when we usually do regression modeling, the fMRI community is not in the habit of doing model diagnostics despite the urging of the fMRI methodologists (REF).
Loh et al. (2008, Statistica Sinica) propose examining the weighted sum of residuals in a sliding window to diagnose model misfitting, bias and power loss.
Bias due to mismodelling is (X^T * V^-1 * X ) ^-1(X^T * V^-1 *G* Beta )
search volume V, corr design matrix X, incorrect design matrix errX, G = discrepancy between the correct and model design matrix, errX = X + G
(Luo and Nichols (2003, NI)). That work used standard diagnostic tools for linear models (e.g., Durbin-Watson, Shapiro-Wilks) to develop new methods of performing model diagnosis in the massive univariate setting. Available as an SPM toolbox, SPMd.
Check analysis mask (what is considered "brain" and included in analyses; SPM outputs this in the subject directory as the file "mask.nii" covers the brain you are interested in
possible causes if does not cover the brain areas like you expect/is clipped
wrong FOV => remove subject or the group analysis mask will be affected (it only takes one bad subject)
signal dropout (common around brain stem and medial temporal lobe and others areas most susceptible to signal dropout like tissue/fluid edges)
in SPM, when you specify your model, you can set the masking threshold (matlabbatch.spm.stats.fmri_spec.mthresh), which effectively determines what is included as "brain" in your model (cut-off in signal considered as "real" and "interesting" signal as opposed to noise and/or outside of the brain).
Plot histograms of beta/contrast estimations for whole-brain for each subj (Tor Wager's fav)
can reveal probs and inhomogeneity across Ss
things to check
contrast values usu ctrd on zero across brain (unlikely to have overall activation or deactiv the brain across an entire experiment)
scale of values is arbitrary but should be similar across participants if preproc same
check that there is no severe skewness (altho some can be caused by true activation)
funny would be: if a lot of the brain is deactivated very strongly relative to other Ss (prob not physiological then i.e. it is most likely an artefact)
Barplot avg contrast values over all vx by tissue type (GM, WM, CSF)
Should have no strong systematic deviations from zero which would be a sign of whole brain activation or deactivation, which would be suspicious
Check the efficiency of contrast estimations (suggested by Tom Nichols who has a sample code for model information stored by the SwE Toolbox that you might adapt for your data)
Check predictors for high-leverage influential values
scatterplot of predictor vs. subjects
scatterplot of leverage vs. subjects
Leverage compounds extreme brain values so would be very problematic
Leverages = diagonals of the hat matrix H (AKA projection matrix, i.e. solution to regressor coeff)
the "hat matrix" = "projection matrix" = Covariance of predictions normalized by variance matrix of the observed values
definition: maps the vector of response values/dependent variable to the vector of fitted values/independent variable y (i.e. it puts the "hat", symbol for "estimated", on the y):
Y_hat = H * Y, where h_{ij} = Cov(Y_hat)/ Var(Y) --- that is,
weights H depend only on the predictor values, i.e. each h_{ii} quantifies the influence the the observed response y_i has on its predicted value y_hat_i (small h_{ii} => observed response y_i plays only a small role in the value of the predicted response y_hat_i (i.e. the mean)
0 <= h_{ij} <= 1
SUM[ h_{ii} ] = number of parameters (regression coefficients excluding the intercept)
for y = x*beta + e :
y_hat = X * b_hat, X = design matrix
beta_hat - beta = H * e
e = Y - Y_hat = Y - X * beta_hat
for ordinary least squares estimates
beta_hat = (X^T*X)^-1*X^T*y so H = X(X^TX)^-1X^T
Cov(e) = (I - H) * sigma^2, where H = X*(X^T*X)^-1*X^T
KEEP IN MIND: leverage only quantifies the potential for a data point to exert strong influence on the regression analysis because
leverage depends only on the predictor values BUT whether the data point is influential also depends on the observed value of the response.
identifying outliers -- common rule of thumb:
(Velleman & Welsch 1981, Am Stat): flag any observations whose leverage value h_{ii} > 3x mean leverage value, where
avg_h = h_bar = SUM( h_{ii} ) / n,
n = # observations, i.e. flag h_ii > 3 * ( SUM( h_{ii} ) / n )
h_bar = SUM( h_{ii} ) / n = rank(H) / n = rank(X) / n = tr(H) / n
OR flag any observation with a leverage > avg_h / n AND very isolated
KEEP IN MIND
You can have a high leverage point that does not change the regression estimations (as would be the case if an outlier falls along the estimated linear prediction but is not clustered around the other data points), so high leverage values do not automatically mean exclude! (This is the distinction between a leverage point and an influence point)
cases where high-leverage points are influential are when
large hat diagonal AND
large residuals (i.e. the studentized residual along with h_{ii})
You cannot always judge leverage points by looking at histograms or scatterplots between two of your variables (but good to plot anyway).
Check statistical maps
"Negative" controls (what you should not find if test is valid): Are there artefacts or whole-image shifts? If so, localized results may not be interepretable
Check for no particularly strong and pervasive activations in WM or CSF (some at the edge of the brain might be reasonable)
Look for values outside of brain/in ventricles at a liberal threshold (e.g. vx 0.05 uncorr)
"Positive" controls (what you expect/should find)
Are activations plausible/in regions where you expect?
Match previous findings?
Compare with automated meta-analysis stat map from Neurosynth of related studies (note does not distinguish activ from deactiv)
Resistance to outliers
Remove Ss that look suspicious and rerun analyses -- use ranked success scores (nonparametric, Spearman's rho against brain values) -- and see if statistical maps stable