If you want to install python packages/modules, refer to Python Module Installation Guide.
To create isolated environments for running Python., refer to Python Virtual Environment Guide
The system software library is managed through modules, including the modules to activate different Python versions. To learn more you can refer to Module System Basicis and Module System In-Depth
The current clusters runs Redhat 8 version of Linux, so environments and personal modules that were installed on older versions of the clusters may need reinstalled.
We use the EasyBuild (EB) software installation framework for Pioneer modules, allowing us to update Python versions with greater consistency. As a result of using EB, Python is provided in component modules, including Python, SciPy-bundle (numpy, scipy, pandas, ...), matplotlib, h5py, and more. The Python modules feature a 'bare' alternative that only features 'core' Python modules, when maximum flexibility in choosing modules is desired. The more complete Python is required to also load other component modules, featuring ~125 commonly used modules.
All the available versions of Python. This applies for other applications as well.
module avail python/
module spider python/
output (4 Nov 2022):
----------------------------- /usr/local/easybuild_allnodes/modules/all -----------------------------
Biopython/1.79-foss-2021a Python/3.9.6-GCCcore-11.2.0-bare
IPython/7.25.0-GCCcore-10.3.0 Python/3.9.6-GCCcore-11.2.0
IPython/7.26.0-GCCcore-11.2.0 (D) Python/3.10.4-GCCcore-11.3.0-bare
Python/2.7.18-GCCcore-9.3.0 Python/3.10.4-GCCcore-11.3.0 (L,D)
Python/2.7.18-GCCcore-10.2.0 flatbuffers-python/2.0-GCCcore-10.3.0
Python/2.7.18-GCCcore-11.2.0-bare flatbuffers-python/2.0-GCCcore-11.2.0 (D)
Python/3.8.2-GCCcore-9.3.0 protobuf-python/3.14.0-GCCcore-10.2.0
Python/3.8.6-GCCcore-10.2.0 protobuf-python/3.17.3-GCCcore-10.3.0
Python/3.9.5-GCCcore-10.3.0-bare protobuf-python/3.17.3-GCCcore-11.2.0 (D)
Python/3.9.5-GCCcore-10.3.0 spglib-python/1.16.0-fosscuda-2020b
Where:
L: Module is loaded
D: Default Module
We can see that the default version of Python is 3.10.4-GCCcore-11.3.0 (toolchain foss/2022a).
Load the appropriate python modules, for example
module load Python/3.10.4-GCCcore-11.3.0
Check python modules. Use pip3 for python version 3.
pip3 list
output: (4 Nov 2022)
appdirs 1.4.4
asn1crypto 1.5.1
atomicwrites 1.4.0
attrs 21.4.0
Babel 2.10.1
...
In the bash terminal, request a compute Node (For Markov, use the appropriate Slurm flags -A and -p):
srun --pty /bin/bash
Load the Installed version of Python module
Run Python version 3 using python3:
python3
In the Python Interactive Window, try the following. Install the numpy and scipy packages; recommended to use Python Virtual Environment.
>>> import numpy;
>>> numpy.version.version
'1.12.0.dev0+59a7b25'
>>> import scipy;
>>> scipy.version.version
'0.18.0.dev0+54c0d5a'
>>> quit()
Run from the Terminal
Copy the job file "job.slurm" and python file "test.py" from /usr/local/doc/NUMPY-SCIPY to your home/course directory
cp /usr/local/doc/NUMPY-SCIPY/* .
Run
python test.py
output:
NumPy Version:
1.12.0.dev0+59a7b25
SciPy Version:
0.18.0.dev0+54c0d5a
c:
[-3.14159265 0. 3.14159265]
Copy the job file "job.slurm" and python file "test.py" from /usr/local/doc/NUMPY-SCIPY to your home directory.
cp /usr/local/doc/NUMPY-SCIPY/* .
Submit the job. Please make sure to use the correct slurm flags in the job.slurm especially for the Markov (class)
sbatch job.slurm
Find the output as showed in the output file " slurm.o<JobID>
NumPy Version:
1.12.0.dev0+59a7b25
SciPy Version:
0.18.0.dev0+54c0d5a
c:
[-3.14159265 0. 3.14159265]