If you are here, you have probably tried to run fmriprep through Docker and have realized that you do not have admin rights in the system that you want to use. If you have not tried that, I suggest you do it (see here how to) and only use Singularity as a plan B.
Since I cannot find a reason why you might want to run Singularity locally (other than debugging your scripts), I will describe here how to run it on a High-Performance Cluster (HPC). The first thing to do is to take a deep breath, get a cup of your favorite drink and equip yourself with a lot of patience: this can get messy.
The first thing to notice is that every HPC can be different and you will have to probably tweak here and there to get it running the first time. The examples I give here are for an HPC using SLURM and with rather closed set up, so you can think about this procedure as a brute force approach: the bright side? If you try all this, you will probably succeed in every system.
Make sure Singularity is (or can be) installed in your HPC. This part might require some time and mano izquierda to talk your support people into do it. There is not much I can do in this regard; find here a nice song to ease the process.
Once Singularity is up and running, you need to make sure that you have enough writing space for your user of HPC. Usually some temporary storage drive is allowed for this, make sure you know where it is and how to access it. NOTE: storage limits can come in size or number of files -inodes- (or both); try to discover your limits (non intended poetry here).
The next step is to download fmriprep's image. While this is a rather trivial step in Docker, in Sigularity it can be tricky. First, identify which version of fmriprep you would like to use; I recomend you always start with the latest one. Singularity does not like the "latest" tag when building, so you need to specify the exact version (e.g., poldracklab/fmriprep:20.1.1). You might want to build the image locally and transfer it to the HPC:
singularity build /my_images/fmriprep-<version>.simg docker://poldracklab/fmriprep:<version>
scp /my_images/fmriprep-<version> user@address:/path/to/writable/location/
Replacing <version> with the exact version name as in the example above and user and address with your HPC login details.
Some HPC nodes do not have internet access and fmriprep will try to go online to pull brain templates from TemplateFlow. My recommendation is that you download the templates that you are going to need (or all of them if you want to go with the sledgehammer approach) and you transfer them to the HPC. Templates can be downloaded with the Python api and using the syntax below (choose the templates that you need).
python3
from templateflow import api as tflow
tflow.get('MNI152NLin2009cAsym')
tflow.get('OASIS30ANTs')
tflow.get('MNI152NLin6Asym')
tflow.get('fsaverage')
tflow.get('NKI')
tflow.get('MNI152NLin2009cSym')
tflow.get('WHS')
tflow.get('fsLR')
tflow.get('MNIPediatricAsym')
tflow.get('MNI152Lin')
tflow.get('MNIInfant')
tflow.get('MNI152NLin6Sym')
Rather obvious step: you need to put your BIDS dataset on the HPC (duh!).
Once you have all the files needed on the HPC, you can start... preparing your call script. Singularity and fmriprep will try to write a bunch of temporary files in your home directory, so if you have limited space in home, you need to redirect them to your larger writable locations. See lines after "Re-direct some (...)" in the script below.
You need to also overwrite fmriprep's "TEMPLATEFLOW_HOME" with the path inside the container in which you are going to mount your templates outside the container. See lines after "Pass some variables into the container".
Now you are ready to call fmriprep through singularity. After the command "run", you need to mount the BIDS folder, the output folder, the templates folder and a writable folder for the home and a working directory. Use the "-B" flag for that (see script below). Do not forget to also mount the path to your freesurfer license.
Remember about the nodes not being able to access the net? You need to tell fmriprep to not attempt since it will fail anyway. That is what the "no-track" flag does.
Finally, it is important to pass the "--home" flag to the Singularity that you want to overwrite the by-default "home" directory. This one needs to be followed by "home/fmriprep".
And you are set!
Just a few more notes on the script below:
I am passing a variable into the container named "subject" so I can pass it to the --subject_label flag to fmriprep. This is pretty useful if you want to create different temporary folders for different subjects or if you want to use your job ID to index several subjects. I'll write a post on parallelizing stuff on a HPC soon; and I will link it [here].
Sometimes after crashing, FreeSurfer does not delete all of its intermediate files and that might cause some problems when trying to re-run fmriprep on the same subjects. You might want to include a line at the beginning to search for those corrupted files and deleting them in case they are found; something like this will do:
find ${FREESURFER_HOST_CACHE}/$subject/ -name "*IsRunning*" -type f -delete
Everything prior to the line "Which subject do you want to run?" is set for our HPC at the Goethe University in Frankfurt. Yours might be different in terms of available resources or requirements; adjust them accordingly.