Module Hierarchies
Module Hierarchies
We make use of module hierarchies in order to keep a consistent environment for our users. Hierarchies unify module file names, changing the module paths conveniently as we change the hierarchy.
Our module system has the following structure:
Core: The core modules are the modules that will always be available. In the list of modules we include compilers, libraries and binaries that do not require any extra dependency than the ones provided by the system.
Compiler: The Compiler modules are formed by those modules that have been compiled using a particular version of a compiler. We currently offer two compiler chains.
gcc
intel
MPI: The MPI modules are those modules that have been compiled using a particular version of MPI (like OpenMPI or MPICH, for example). We currently offer
openmpi
How do Hierarchies Work?
When you are logged into the cluster you will be presented with a default hierarchy of modules:
intel/17 + openmpi/2.0.1
To check the list of available modules for this hierarchy use module avail (check the list of Lmod Commands)
module avail
This gives the following output (the list of modules will vary as we install new software).
-------------------------------- /usr/local/share/modulefiles/MPI/intel/17/openmpi/2.0.1 -----------------------------------
amber/16-17 eigen3/3.3.4 imagemagick/7.0.4-10 namd/2.12-cuda openfoam/4.1 samtools/1.5
bcftools/1.5 fftw/3.3.6-pl2 lammps/2017-gpu namd/2.12 (D) python/3.5.1 (D) spyder/3.2.0-python2 (D)
boost/1.63 hdf5/1.10.1 lammps/2017 (D) netcdf/4.4.1.1 python2/2.7.13 vtk/8.0.1
------------------------------------ /usr/local/share/modulefiles/Compiler/intel/17 -------------------------------------
bowtie2/2.3.2 gnuplot/4.6.2 (D) gstreamer/0.10.36 kim/1.9.1 phonon/4.9.1 qt/5.8.0 (D)
cmake/3.8.2 gnuplot/5.0.6 gstreamer/1.10.3 (D) openmpi/1.8.8 poppler-qt5/0.51.0 texstudio/2.12.4
ffmpeg/3.2.4 gsl/2.3 julia/0.5.1 openmpi/2.0.1 (L,D) qt/4.8.7
----------------------------------------- /usr/local/share/modulefiles/Core --------------------------------------------
MKL/17 cuda/8.0 gaussian/g16-avx2 ghostscript/9.20 matlab/R2016b schrodinger/2017-3 vmd/1.9.3
STAR/2.5.3a cufflinks/2.2.1 gaussian/g16-sse (D) icu/52.1 pandoc/1.19.2.1 texinfo/5.1 voro++/0.4.6
base/8.0 gatk/3.6 gaussview/gv6 intel/17 (L) pdsh/2.26 texlive/2016
bin2c/git gaussian/g16-avx gcc/6.3.0 java/8u121 picard/2.11 tophat/2.1.1
------------------------------------------ /usr/local/lmod/lmod/modulefiles/Core ----------------------------------------
StdEnv (L) lmod/7.3 settarg/7.3
Where:
L: Module is loaded
D: Default Module
Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
In this output we can see the three categories of modules separated by dashed (--) lines.
Unloading the openmpi/2.0.1 module will remove the MPI modules from the available list
module unload openmpi
module avail
We can see that the MPI compilers are no longer in the output
-------------------------------------- /usr/local/share/modulefiles/Compiler/intel/17 --------------------------------------------
bowtie2/2.3.2 gnuplot/4.6.2 (D) gstreamer/0.10.36 kim/1.9.1 phonon/4.9.1 qt/5.8.0 (D)
cmake/3.8.2 gnuplot/5.0.6 gstreamer/1.10.3 (D) openmpi/1.8.8 poppler-qt5/0.51.0 texstudio/2.12.4
ffmpeg/3.2.4 gsl/2.3 julia/0.5.1 openmpi/2.0.1 (D) qt/4.8.7
---------------------------------------------- /usr/local/share/modulefiles/Core -------------------------------------------------
MKL/17 cuda/8.0 gaussian/g16-avx2 ghostscript/9.20 matlab/R2016b schrodinger/2017-3 vmd/1.9.3
STAR/2.5.3a cufflinks/2.2.1 gaussian/g16-sse (D) icu/52.1 pandoc/1.19.2.1 texinfo/5.1 voro++/0.4.6
base/8.0 gatk/3.6 gaussview/gv6 intel/17 (L) pdsh/2.26 texlive/2016
bin2c/git gaussian/g16-avx gcc/6.3.0 java/8u121 picard/2.11 tophat/2.1.1
---------------------------------------------- /usr/local/lmod/lmod/modulefiles/Core ---------------------------------------------
StdEnv (L) lmod/7.3 settarg/7.3
Changing Hierarchies
When you are logged into the cluster you will be presented with a default hierarchy of modules:
intel/17 + openmpi/2.0.1
If you need to work with GNU compilers, you can change hierarchies by switching the compiler (see the swap command in Lmod Commands).
module swap intel gcc
This will produce the following output
Due to MODULEPATH changes the following have been reloaded:
1) openmpi/2.0.1
This message indicates that the openmpi module changed from the Intel to the GNU hierarchy.
This will set your environment with the following hierarchy
gcc/6.3.0 + openmpi/2.0.1
We can list the available modules for this hierarchy:
module avail
Output:
----------------------------------- /usr/local/share/modulefiles/MPI/gcc/6.3.0/openmpi/2.0.1 ------------------------------------
R/3.2.5 boost/1.63.0 imagemagick/7.0.4-10 opencv/3.2.0 (D) rstudio/1.0.136 spyder/3.2.0-python2 (D)
R/3.3.3 (D) fftw/3.3.6-pl2 microsoft-r/3.3 python2/2.7.13 rstudio/1.0.153 (D)
----------------------------------------- /usr/local/share/modulefiles/Compiler/gcc/6.3.0 ------------------------------------------
atlas/3.10.3 ffmpeg/3.2.4 gstreamer/1.10.3 (D) openmpi/1.8.8 phonon/4.9.1 qt/4.8.7
blas/3.7.0 gsl/2.3 lapack/3.7.0 openmpi/2.0.1 (L,D) poppler-qt5/0.51.0 qt/5.8.0
cmake/3.8.2 gstreamer/0.10.36 libbsd/0.8.6 openmpi/2.1.0 qpdf/7.0.0
------------------------------------------------- /usr/local/share/modulefiles/Core ------------------------------------------------
MKL/17 cuda/8.0 gaussian/g16-avx2 ghostscript/9.20 matlab/R2016b schrodinger/2017-3 vmd/1.9.3
STAR/2.5.3a cufflinks/2.2.1 gaussian/g16-sse (D) icu/52.1 pandoc/1.19.2.1 texinfo/5.1 voro++/0.4.6
base/8.0 gatk/3.6 gaussview/gv6 intel/17 pdsh/2.26 texlive/2016
bin2c/git gaussian/g16-avx gcc/6.3.0 (L) java/8u121 picard/2.11 tophat/2.1.1
---------------------------------------------- /usr/local/lmod/lmod/modulefiles/Core ------------------------------------------------
StdEnv (L) lmod/7.3 settarg/7.3
One can observe that the core modules do not change with respect to the intel/17 + openmpi/2.0.1 hierarchy , but the Compiler and MPI modules did change.