Conda environments using Miniconda3

Anaconda is not appropriate for the cluster environment in most circumstances. HPC administration team recommends to proceed with miniconda for software intended for use in conda environments. Be aware that the installer should not be allowed to initialize Miniconda3 (see detailed note below)

Though there is an installation guide to install Miniconda in your home directory, it is recommended to use the installed version . Note that you need to request a compute node to actually run the job (check Interactive/Batch job submission)

• Load Miniconda Module:

  • module load Miniconda3/22.11.1-1

• Create basic conda environment. The --prefix can just be /home/<caseID>. Choose the appropriate name for your conda enviornment. The older python library at ~/.local/lib/python* might conflict with this installation. So, you may need to delete them.

  • conda create --prefix <prefix-path>/<mycondaenv> --clone base

• Check environment info:

  • conda info -e

    • <prefix-path>/<mycondaenv>

• Source the conda profile or initialize

  • source $INIT

• Activate the environment (“conda deactivate” to deactivate the environment)

  • conda activate <prefix-path>/<mycondaenv>

    • (<prefix-path>/<mycondaenv>) [<caseID>@hpc6 ~]$

• Check the listed packages

  • conda list

    • # Name                    Version                   Build  Channel

    • _libgcc_mutex             0.1                        main

• Install python pkg (e.g. numpy)

  • conda install numpy

• Exit/Deactivate

  • conda deactivate

• To remove the Pkg/Environment

  • conda remove <prefix-path>/<mycondaenv>  --all

Creating Reproducible Python Environment

• Create a requirement text file

  • conda list -e > requirements.txt

• Install the required Packages

  • conda install --file requirements.txt

Conda - Miniconda Installation

Conda (https://docs.conda.io/projects/conda/en/latest/index.html) is an open-source package management system and environment management system that runs on Windows, macOS, and Linux.

Use the similar procedure for Anaconda Installation - https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html 

Get the shell script (for Miniconda)

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Install the script.  Make sure that you don't initialize Miniconda (by typing no) else your .bashrc file gets modified.

bash Miniconda3-latest-Linux-x86_64.sh

output:

In order to continue the installation process, please review the license

agreement.

Please, press ENTER to continue

>>>

Do you accept the license terms? [yes|no]

[no] >>> yes

Miniconda3 will now be installed into this location:

/home/<caseID>/miniconda3   # for class account, use /mnt/pan/courses

  - Press ENTER to confirm the location

installation finished.

Do you wish the installer to initialize Miniconda3

by running conda init? [yes|no]

[no] >>> no  #  avoids conda environment at ~/.bashrc which conflicts with ondemand and others

Thank you for installing Miniconda3!

Activate the conda environment using the following command where <caseID> is your actual network id:

eval "$(/home/<caseID>/miniconda3/bin/conda shell.bash hook)"

Check the conda packages installed

conda list

output:

# packages in environment at /home/<caseID>/miniconda3:

#

# Name                    Version                   Build  Channel

_libgcc_mutex             0.1                        main

ca-certificates           2020.1.1                      0

certifi                   2020.4.5.1               py38_0

...

If the package (e.g. Perl) is not installed, you can install it.

conda install perl

output:

The following NEW packages will be INSTALLED:

  perl               pkgs/main/linux-64::perl-5.26.2-h14c3975_0

...

Update Conda:

conda update conda

Deactivate Conda:

conda deactivate