Module System -- Advanced

Scientific software is typically organized into executable files, and support files (libraries and header files). The location in the file structure of these files must be known in the run-time environment (referred in short as the 'environment') for the software to successfully run.

LMOD provides a convenient way to dynamically change a user's environment through modulefiles. This includes easily adding or removing directory paths to the PATH and LD_LIBRARY_PATH environment variable, and some other details.

A modulefile contains the necessary information to allow a user to run a particular application or provide access to a particular library. All of this can be done dynamically without logging out and back in. Modulefiles for applications modify the user's path to make access easy. Modulefiles for Library packages provide environment variables that specify where the library and header files can be found. It is also very easy to switch between different versions of a package or remove the package. 

Paths to Modules

The MODULEPATH variable is set by default to include the baseline EasyBuild path. Custom software, specifcially that installed from source in a personal directory (e.g. /home/<caseid>/..) may have Lmod modules established. The MODULEPATH will require modification to make those modules available to use for managing the shell environment for this custom software. The directory paths to include in the MODULEPATH are described as the 'root' of a software 'tree'.

General Software Trees

Software may run on some or all of the HPC compute nodes. As a reminder, the general access compute nodes are found on the following partitions for the two clusters:

Research cluster (Pioneer):  batch, gpu, smp, amd partitions

Course-support cluster (Markov): markov_cpu, markov_gpu

The software trees are named to indicate the compatibility with the various compute nodes.

The software trees on Pioneer include::

• easybuild5_2080/2026_2  -- use broadly, not optimized for more capable servers

• easybuild5_icosa/2026_2  -- use with icosa192gb/256gb nodes

• easybuild5_2026_1  -- for use with gpul40s compute nodes

• easybuild5_2025_2  -- built with EasyBuild v5

• easybuild_2025_1   -- built with EasyBuild v4

• easybuild_allnodes  -- software built to run on all compute nodes (limited optimization)

The software trees on Markov3, the newest version of the academic use cluster, current has the trees:

• easybuild_5_2080all/2026_2 - built on 2080 gpu, use with any gpu node

• easybuild_5_icosa/2026_2  -- use with icosa192gb/256gb nodes

• easybuild_5_icosa/2026_1  -- sane as above, earlier in 2026

Optimization Trees

The CWRU HPC has nodes with a variety of capabilities, including the cpu instructions sets. The module tree name attempts to indicate what compute nodes should support the software. The current trees and their capabilities are listed:

• allnodes: (default)  should run without error on all the HPC nodes, though without the latest cpu architecture support.

• 2080:      optimized to run on gpu2080 feature nodes

• icosa:      optimized to use all cpu architecture support on the icosa192gb feature nodes.

• aisc:        optimized for use with the aisc partition nodes (limited access).

Trees are pending for the amd and gpul40s architectures.

Modify MODULEPATH variable to access alternative software trees

To switch trees, there are two approaches. Both adjust the MODULEPATH environment variable.

• Approach 1

  • export MODULEPATH=/usr/local/easybuild_<tree>/modules/all

• Approach 2

  • module unuse /usr/local/easybuild_allnodes/modules/all

  • module use /usr/local/easybuild_<tree>/modules all

The <tree> is a placeholder, with the target to be taken from the list of trees above (e.g. easybuild_allnodes)

Supplementary Software

The latest versions of licensed software and other system software supplementing that available from EasyBuild will be installed in another location, from the file system root location "/usr/local/software". For example, Matlab will be found under "/usr/local/software/matlab/<release-version>". This path is also included by default in MODULEPATH.

Possible Compatibility Issues

For compatibility reasons, be careful to load the dependency modules built with consistent toolchains and versions. For example, the following dependency modules are using the same GCCcore-12.2.0.

• module purge           # purge all the modules before loading any new modules

• module load Python/3.10.8-GCCcore-12.2.0

• module load Tk/8.6.12-GCCcore-12.2.0

• module load Tcl/8.6.12-GCCcore-12.2.0

Let's consider a scenario where you want to use PyYAML module in addition to those above. Let's say you decide to use the version "PyYAML/6.0". Then you need to check the possible dependency modules. Run 'module avail PyYAML/6' to find:

$ module avail PyYAML/6

--------- /usr/local/easybuild_allnodes/modules/all ---------

   PyYAML/6.0-GCCcore-11.3.0    

   PyYAML/6.0-GCCcore-12.2.0

   PyYAML/6.0-GCCcore-12.3.0 (D)

Note the default module is built with toolchain GCCcore-12.3.0. Use the full module name 'PyYAML/6.0-GCCcore-12.2.0' to load the consistent set of modules.

Understanding MODULEPATH environment variable

The MODULEPATH environment variable (echo $MODULEPATH) identifies the path(s) searched by Lmod.  To summarize,  'module avail' provides a listing of module names, and "module spider <name>" provides the set of modules; a description of the software; and further steps to gain more information. Following from the previous example, use "module spider gompi/' and "module spider gompic/" to better understand 'gompic':

[mrd20@hpc8 ~]$ module spider gompi/

Description:  GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support.

  Versions:    gompi/2020a, gompi/2020b, gompi/2021a, gompi/2021b, gompi/2022a

                gompi/2022b, gompi/2023a, gompi/2023b, gompi/2024a

As previously, for detailed information about a specific package (including how to load the modules) use the module's full name.   For example:

  $ module spider gompi/2021a

Note that tab-completion of module names is available in the shell.

Manipulate MODULEPATH with Lmod

Wildcard characters can be used to search all the software trees, without knowing the specific names or paths.  The 'module use'  command with * as a wild-card character prepares MODULEPATH to find all available version of the software in all trees matching the pattern 'easybuild*'.

module use /usr/local/easybuild*/modules/all

Then use either 'module spider' or 'module avail' to conduct the search:

To learn about individual modules, "module display <modulename>" shows contents of the module file.

module display NCCL/2.22.3-GCCcore-13.3.0-CUDA-12.6.0

output:

prepend_path("CPATH","/usr/local/easybuild_aisc/software/NCCL/2.22.3-GCCcore-13.3.0-CUDA-12.6.0/include") # aisc tree

Now, make sure that you request the node with appropriate partition/feature and use module use command when preparing the Slurm resource request, using:

module use /usr/local/easybuild_<tree>/modules all  # aisc

module load <Software>