Relion Installation on GoToCloud
Contents
Before proceeding with the installation, you must acquire write permissions for the EFS.
There are currently two methods:
• a. Using a ParallelCluster: Slower (needs to install all dependencies) but currently recommended.
• b. Using a standalone EC2 instance: Faster, but currently less secure.
Note: We recommend using the ParallelCluster method (a) for now. In the future, once proper security groups and access rules are defined, the EC2 method (b) should become the standard procedure.
Log in to your AWS account with administrator and create ParallelCluster.
For more information, see: https://sites.google.com/sbrc.jp/gotocloud-doc/procedure
Alternatively, you can create the ParallelCluster using the GTC WebApp:
a. Log in to a KEK901 account and create an EC2 instance in the user-vpc.
b. Follow the GTC WebApp procedures to create the ParallelCluster.
c. Launch a terminal session on the ParallelCluster head node.
[2026/07/28 Developer's Note] Due to the AWS Cloud9 end of service, an EC2 instance / ParallelCluster must now be used to write to EFS. All previous network requirements still apply.
[2024/10/02 Developer's Note] When updating EFS in the KEK system, Cloud9 should be created in the usr-vpc as per the hands-on instructions. Only kek901 is peered with the master-vpc, allowing updates to the master-EFS. Ideally, it should be named something like master-operate-vpc, rather than usr-vpc.
a. Navigate to the EC2 service in the AWS dashboard.
b. Select Instances from the left-hand menu.
c. Select the head node created in step 1.1.1.
d. In the top right corner, click Actions > Security > Modify IAM role.
e. Choose EFSWriteAccessRole and click Update IAM role.
a. Connect to the ParallelCluster head node created above.
b. Unmount the existing shared EFS (which currently has no write permissions).
c. Mount the master shared EFS with write permissions.
$ gtc_pcluster_ssh.sh
$ sudo umount /efs
$ sudo mount -t efs -o tls,iam,mounttargetip=[mount target IP address] [master of shared EFS ID] /efs
The EFS target IP and ID can be found in the AWS dashboard under the EFS service.
Example of KEK's GoToCloud platform.
$ gtc_pcluster_ssh.sh
$ sudo umount /efs
$ sudo mount -t efs -o tls,iam,mounttargetip=10.0.10.180 fs-0a1234b56789 /efs
Important: This procedure is faster than the ParallelCluster method, but it temporarily lowers security. In the future, this method will supersede the ParallelCluster method once proper security groups are designed to allow SSH access and AWS service access (for amazon-efs-utils).
a. Navigate to the EC2 service in the AWS dashboard and click Launch Instance.
b. Configure your instance with the following settings:
• Name: (Any name you can easily remember)
• Key pair name: (Select a key pair you have access to)
• Network settings (VPC): kek-gtc-master-vpc
• Auto-assign public IP: Enable
• Security groups: Select existing security group cloud9alt-ssh-efs-42fb3392.
(Future: A dedicated security group with limited, tailored access for EFS writing should be designed).
• Advanced details > IAM instance profile: Select EFSWriteAccessRole
Warning: This procedure reduces security. You must follow 1.2.4. Cleanup to delete these configurations after editing the EFS.
Part 1: Internet Gateway
a. Navigate to the VPC service and click Internet gateways.
b. Create an internet gateway named: kek-gtc-master-igw.
c. Select the new gateway, click Actions > Attach to VPC, and select kek-gtc-master-vpc.
Part 2: Route Tables
1. In the VPC service menu, click Route tables.
2. Locate and select the route table named kek-gtc-master-rtb.
3. Click Edit routes > Add route and configure it as follows:
• Destination: 0.0.0.0/0
• Target: Select Internet Gateway, then choose the newly created kek-gtc-master-igw.
4. Click Save changes.
a. Connect to the EC2 instance via SSH.
b. Run the following commands to install dependencies and mount the drive:
$ sudo yum -y install amazon-efs-utils
$ sudo mkdir /efs
$ sudo mount -t efs -o tls,iam,mounttargetip=[mount target IP address] [master of shared EFS ID] /efs
The EFS target IP and ID can be found in the AWS dashboard under the EFS service.
After editing the EFS and shutting down the EC2 instance, you must remove the temporary network access:
a. Go to the kek-gtc-master-rtb route table and remove the 0.0.0.0/0 route targeting kek-gtc-master-igw.
b. Go to Internet Gateways, detach kek-gtc-master-igw from the VPC, and then delete the internet gateway entirely. (Note: It may take a few minutes for the IP address to fully release).
The following instructions assume that anaconda are available.
Connect to head node via ssh.
$ cd /efs/em
$ git clone https://github.com/pyenv/pyenv
$ export PYENV_ROOT=/efs/em/pyenv
$ export PATH=$PYENV_ROOT/bin:$PATH
$ pyenv install --list
$ pyenv install anaconda3-2023.03
$ pyenv versions
system
* anaconda3-2023.03 (set by /efs/em/pyenv/version)
$ pyenv global anaconda3-2023.03
$ pyenv version
anaconda3-2023.03 (set by /efs/em/pyenv/version)
$ export PATH=$PYENV_ROOT/versions/anaconda3-2023.03/bin/:$PATH
How to uninstall pyenv (as necessary)
$ rm -rf pyenv
How to uninstall anaconda (as necessary)
$ pyenv uninstall anaconda3-2023.03
You can set Relion default version.
Create the version file directly under the directory where the module file exists.
$ cd /efs/em/modulefiles/relion/
$ cat > .version
##### Copy and paste from below #####
#%Module 1.0
set ModulesVersion 4.0.1-pc3.7.0
##### to above #####
$ cd 4.0.1-pc3.7.0
$ cat > .version
##### Copy and paste from below #####
#%Module 1.0
set ModulesVersion intel_amd-gcc-intelmpi-gpu
##### to above #####
Connect to head node via ssh.
$ cd /efs/em/
$ git clone https://github.com/KEK-SBRC-CryoEM/gotocloud.git
$ cp -r gotocloud/gtc_sh ./
$ cd /efs/em/
$ cp -r gotocloud/submission_script/* ./
Note: Copy files only
$ cd /efs/em/
$ cp -r gotocloud/test_schemes ./
See here how to use test schemes.
$ cd /efs/em/
$ cp -r gotocloud/schemes_creator_sh ./
$ cd /efs/em/
$ rm -rf gotocloud
If you have not installed pyenv and anaconda, perform step2. before installing crYOLO.
The following instructions assume that anaconda are available.
Connect to head node via ssh.
$ cd /efs/em/
$ export PYENV_ROOT=/efs/em/pyenv
$ export PATH=$PYENV_ROOT/bin:$PATH
$ pyenv versions
system
* anaconda3-2023.03 (set by /efs/em/pyenv/version)
$ pyenv global anaconda3-2023.03
$ pyenv version
anaconda3-2023.03 (set by /efs/em/pyenv/version)
$ export PATH=$PYENV_ROOT/versions/anaconda3-2023.03/bin/:$PATH
create conda environment and install cryolo
$ conda create -n cryolo-1.9.6 -c conda-forge -c anaconda pyqt=5 python=3 numpy=1.18.5 libtiff wxPython=4.1.1 adwaita-icon-theme 'setuptools<66'
$ source activate cryolo-1.9.6
$ pip install nvidia-pyindex
$ pip install 'cryolo[c11]==1.9.6'
$ conda deactivate
・When conda create takes too long time
Change the package solver of conda to libmamba.
cf. https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community
$ conda install -n base conda-libmamba-solver
$ conda config --set solver libmamba
<Just in case> How to remove conda environment
$ export PYENV_ROOT=/efs/em/pyenv
$ export PATH=$PYENV_ROOT/bin:$PATH
$ export PATH=$PYENV_ROOT/versions/anaconda3-2023.03/bin/:$PATH
$ conda env remove --name cryolo-1.9.6
Create module file.
$ cd /efs/em/modulefiles/crYOLO
$ cat > 1.9.6
#%Module -*- tcl -*-
set root /efs/em/pyenv/versions/anaconda3-2023.03/envs/cryolo-1.9.6
setenv CRYOLOPATH $root
prepend-path PATH $root/bin
setenv CRYOLO_SUBMIT_CMD sbatch
setenv CRYOLO_SUBMIT_SCRIPT /efs/em/aws_slurm_cryolo.sh
[CONTROL+C]
Setup default version.
$ cd /efs/em/modulefiles/crYOLO/
$ cat > .version
#%Module 1.0
set ModulesVersion 1.9.6
[CONTROL+C]
$ cd /efs/em/cryolo
$ mkdir models
Download all general models for crYOLO to head node from here. It is recommended to connect to head node via niceDCV and download from firefox via http.
Move all models for crYOLO to "/efs/em/cryolo/models" derectory
$ mv /path/to/model /efs/em/cryolo/models/
Create symbolic links for each models
$ ln -s /efs/em/cryolo/models/gmodel_phosnet_202005_N63_c17.h5 /efs/em/cryolo/gmodel_phosnet_lpf_link.h5
$ ln -s /efs/em/cryolo/models/gmodel_phosnet_202005_nn_N63_c17.h5 /efs/em/cryolo/gmodel_phosnet_denoise_link.h5
$ ln -s /efs/em/cryolo/models/gmodel_phosnet_negstain_20190226.h5 /efs/em/cryolo/gmodel_phosnet_negstain_link.h5
$ cd /efs/em/
$ git clone https://github.com/KEK-SBRC-CryoEM/gotocloud.git
$ cp -r gotocloud/cryolo ./
How to use python scripts to run crYOLO in Relion external job.
Launch Relion GUI.
Select "External" from job list on the left.
Enter the following in the "Input" tabs.
External executable: /efs/em/cryolo/gtf_relion4_run_cryolo_system.sh
Input micrographs: [path to micrographs.star file]
Enter the following in the "Params" tabs.
Param1 label, value: cryolo_repo, /efs/em/crYOLO
Param2 label, value: threshold, [threshold for cryolo]
Param2 label, value: device, [GPU ID (which GPUs to use) e.g. 0:1:2:3:4:5:6:7]
Enter the following in the "Running" tabs
Number of thread: 1
Submit to queue: Yes
Queue name: optional
Queue submit command: sbatch
Partition: [Partition name (e.g. g5-vcpu192-gpu8)]
Standard submission script: [submission script to use (e.g. /efs/em/aws_slurm_relion.sh)]
Minimum dedicated cores per node: 1
Click ”Run!"
Not required if Relion version 5 is installed, as Topaz will be installed when Relion is installed.
If you have not installed pyenv and anaconda, perform step2. before installing Topaz.
Connect to head node.
If continuing from 3.1., the following is not required.
$ cd /efs/em/
$ export PYENV_ROOT=/efs/em/pyenv
$ export PATH=$PYENV_ROOT/bin:$PATH
$ pyenv versions
system
* anaconda3-2023.03 (set by /efs/em/pyenv/version)
$ pyenv global anaconda3-2023.03
$ pyenv version
anaconda3-2023.03 (set by /efs/em/pyenv/version)
$ export PATH=$PYENV_ROOT/versions/anaconda3-2023.03/bin/:$PATH
create conda environment and install Topaz
$ conda create -n topaz-0.2.5 python=3.6
$ source activate topaz-0.2.5
$ conda install topaz -c tbepler -c pytorch
$ conda deactivate
Create module file for Topaz
$ cd /efs/em/modulefiles/Topaz
$ cat > 0.2.5
#%Module -*- tcl -*-
set root /efs/em/pyenv/versions/anaconda3-2023.03/envs/topaz-0.2.5
prepend-path PATH $root/bin
Setup default version
$ cd /efs/em/modulefiles/Topaz/
$ cat > .version
##### Copy and paste from below #####
#%Module 1.0
set ModulesVersion 0.2.5
##### to above #####
Connect to head node via niceDCV.
Open the Firefox Web Browser
Open the following web page.
https://grigoriefflab.umassmed.edu/ctf_estimation_ctffind_ctftilt
Download the latest version of ctffind-4 for linux64.
(KEK has confirmed that 'ctffind-4.1.14-linux64.tar.gz' is available)
If you cannot download installer directly to the head node from the ubuntu web browser, upload installer to the head node by referring to
method 1 here.
Open the terminal and run following commands.
$ cd /efs/em
$ mkdir ctffind-4.1.14-linux64
$ cd ctffind-4.1.14-linux64
$ cp ~/Downloads/ctffind-4.1.14-linux64.tar.gz ./
$ tar -zxf ctffind-4.1.14-linux64.tar.gz
Connect to head node via niceDCV.
Open the Firefox Web Browser
Open the following web page.
https://www.cgl.ucsf.edu/chimera/download.html
Download the latest version of Chimera installer for Linux 64-bit
(KEK has confirmed that 'chimera-1.15-linux_x86_64.bin' is available)
If you cannot download installer directly to the head node from the ubuntu web browser, upload installer to the head node by referring to
method 1 here.
Open the terminal and run command.
$ chmod +x chimera-1.15-linux_x86_64.bin
$ ./chimera-1.15-linux_x86_64.bin
Enter install location: /efs/em/UCSF-Chimera64-1.15 # Input Installation destination
Create module file for chimera
$ cd /efs/em/modulefiles/
$ mkdir chimera
$ cd chimera
$ cat > 1.15
#%Module1.0
set-alias chimera /efs/em/UCSF-Chimera64-1.15rc/bin/chimera
Setup default version
$ cd /efs/em/modulefiles/chimera/
$ cat > .version
#%Module 1.0
set ModulesVersion 1.15
$ cd /efs/em/
$ git clone https://github.com/KEK-SBRC-CryoEM/gotocloud.git
$ cp -r gotocloud/select_class3d ./
How to use python scripts to select class3d map in Relion external job.
Launch Relion GUI.
Select "External" from job list on the left.
Enter the following in the "Input" tab.
External executable: /efs/em/pyenv/versions/anaconda3-5.3.1/envs/cryolo-1.8.0/bin/python /fsx/GoToFly/SelectClass3D/gtf_relion4_run_select_class3d.py
Input particles: [Path to Class3D result file]
All fields should be blank in the "Params" tab.
Enter the following in the "Running" tab.
Number of thread: 1
Submit to queue: No
Click ”Running".