CAMM‎ > ‎Resources‎ > ‎Hardware‎ > ‎

OLCF

"The Oak Ridge Leadership Computing Facility, formerly the National Leadership Computing Facility, is a designated user facility operated by Oak Ridge National Laboratory and the Department of Energy".

CAMM users have access to the following clusters under project mat049:


Barebones profile files

These are .bash_profile and .bashrc minimal files to get you up and running after ssh to any any of the OLCF login nodes. These files can also be found under directory /ccs/proj/mat049/camm/profile/.
The preferred way to run software compiled in Titan by CAMM staff is to load the appropriate module file. The previous profile files define the appropriate environment n order to load the modules

Using compiled software in PBS scripts

To invoque a particular piece of compiled software in a PBS script, insert a call to the appropriate module file:
source $MODULESHOME/init/bash
module load desiredApplication  # loads executables and environment variables for 'desiredApplication'
Here's an example PBS script to load cp2k.

Another example:
#!/bin/bash
#PBS -A mat049
#PBS -N equil
#PBS -j oe
#PBS -l walltime=6:00:00,nodes=135
#PBS -l gres=atlas1%atlas2

# Load module for lammps
source $MODULESHOME/init/bash
module load lammps/1Feb14 # lmp_titan is added to PATH variable

# End PBS directives and begin shell commands
prefix='equil.1' # Either of 1, 2, or 3
wd=$MEMBERWORK/mat049/pStyrene_Toluene/1styrene32/1styrene32_4017toluene
cd $wd
for T in 30 50 70 90 110 130 150 170 190 210 230 250 270 290 310;do
aprun -n 144 lmp_titan -in ${prefix}_T${T}.in &
sleep 1s #wait a bit in between submissions
done
wait
# Copy output and input to project directory
projd=$PROJWORK/mat049/jbq/pStyrene_Toluene/1styrene32/1styrene32_4017toluene
mkdir -p $projd
/bin/cp ${prefix}* $projd/

Submit a job dependent on another

ID=`qsub my_first_job.pbs`
qsub -W depend=afterok:$ID my_dependent_job.pbs

Show remanining time in projects

 $> showusage
                                  Project Totals           jmborr
 Project      Allocation        Usage    Remaining          Usage
__________________________|____________________________|_____________
 bip100         8000000   |    3338464      4661536  |          0
 mat049         8000000   |   13055321     -5055321  |    1136991

Command to show unallocated nodes

This is Useful if you can run your job accommodating with available resources
$> showbf
Partition     Tasks  Nodes      Duration   StartOffset       StartDate
---------     -----  -----  ------------  ------------  --------------
ALL           45704   2628      00:56:32      00:00:00  19:20:59_08/19
ALL            4728     67      INFINITY      00:00:00  19:20:59_08/19
titan         43768   2612     00:56:32      00:00:00  19:20:59_08/19
titan          2792     51      INFINITY      00:00:00  19:20:59_08/19
login          3928     17      INFINITY      00:00:00  19:20:59_08/19


Storage Lifetime Policy

Area Path Type Permissions Quota Backups Purged Retention
Project Home /ccs/proj/[projid] NFS 770 50 GB Yes No 90 days
Member Work $MEMBERWORK/[projid] Lustre® 700 [1] 10 TB No 14 days 14 days
Project Work $PROJWORK/[projid] Lustre® 770 100 TB No 90 days 90 days
World Work $WORLDWORK/[projid] Lustre® 775 10 TB No 14 days 14 days
Project Archive /proj/[projid] HPSS 770 100 TB [2] No No 90 days




Subpages (3): Eos Rhea Titan