HPCS Software Module Farm (SMF) Service Overview

The Software Module Farm (SMF) service is offered to all Berkeley Lab users with compatible systems, regardless of whether or not the HPCS group manages that system.  Currently we support all Red Hat Enterprise Linux hosts using version 6 (or any rebuild thereof, including Scientific Linux and CentOS).  The HPCS team manages the SMF on a centralized storage device and allows the module tree to be mounted as a read-only NFS filesystem on any authorized host.  Users of existing module farms can receive assistance in migrating to this new service if desired, and any missing software packages will be added to the new SMF service during this process.

All existing HPCS SLA customers are eligible to obtain this service free of charge; that is, it is included with all SLAs.  For systems not currently under SLA with the HPCS group, a nominal $25/month/host charge will be made to help cover maintenance and hardware costs.  A project ID will be required before any non-SLA host is permitted to mount the SMF.

If you have any questions regarding the new HPCS SMF service, please send an email to hpcshelp@lbl.gov.

Software Module Farm (SMF) Infrastructure

The Software Module Farm (SMF) is set up to use the Environment Modules software tool to set the appropriate environment variables in your shell needed to make use of the individual software packages. This system is identical to the one used on the High-Performance Computing Services (HPCS) group's LBNL Supercluster.

The following commands are available to manipulate modules in your environment:

$ module load SOFTWARE # Load the module “SOFTWARE”

$ module unload SOFTWARE # Unload the module “SOFTWARE”

$ module available   # List all modules available for loading

$ module list   # List all modules currently loaded

There are more module commands that you can use. See the manual via command:

$ man module

HPCS Software Module Farm

The HPCS group manages a variety of clusters and standalone systems across the Lab. The HPCS Software Module Farm service provides a comprehensive and well-tested suite of software modules for clusters and individual systems to utilize. It also manages the modules in a hierarchical way so that the module dependencies are well-defined and organized.

There are three types of software modules:

  1. Tool - Tool modules are built and compiled with the default system GCC compiler. They have no dependencies.
  2. Language - Language modules are compilers/interpreters for a specific language, such as GCC or Python.
  3. Submodule - Submodules include libraries, subpackages, and other software which depend on a particular Language module (e.g., a particular version of Python). Due to this dependency, submodules will only be visible once the associated Language module is loaded. For example, the numpy/1.6.1 module built against python/2.7.3 will only be visible once the python/2.7.3 module has been loaded.

Advantages for the User

  • The layout and architecture of our HPCS SMF is designed to make large suites of modules with numerous dependencies manageable for administrators and end-users alike.  Only those modules which are compatible with your environment and operating system are shown, and more modules become available as their dependencies are loaded.
  • The HPCS SMF is a replica for HPCS Supercluster SMF that is used by clusters in our supercluster environment (including Lawrencium, nano and clusters in the environment). Existing supercluster users can have identical software environment on there local systems after mounting the HPCS SMF.
  • All modules are fully tested on all supported operating systems, distributions, and versions.
  • The HPCS team handles all the compilation and maintenance tasks, so you don't have to!
  • Licenses for selected commercial compilers (currently PGI and Intel) are provided at no extra cost to you.
  • Leverage economies of scale by sharing unified maintenance efforts across the entire Laboratory.