• If needed: Install an X11 system on your Windows laptop like MobaXterm: https://mobaxterm.mobatek.net 

• To ssh onto the server: `ssh <username>@casper.hpc.ucar.edu -Y` (-Y to do X-11 forwarding on Mac, -X on non-Mac)

• More NCAR/CISL Casper Resources

  • Description of the Data Analysis and Visualization server Casper
    https://ncar-hpc-docs.readthedocs.io/en/latest/compute-systems/casper/#casper-hardware
    https://ncar-hpc-docs.readthedocs.io/en/latest/compute-systems/casper/

  • In-depth information on Derecho (summer 2023):
    https://www.cisl.ucar.edu/sites/default/files/2023-08/2023%20Derecho%20Overview%20for%20NCAR%20Labs.pdf 

  • Description of the charges for Derecho jobs:
    https://ncar-hpc-docs.readthedocs.io/en/latest/pbs/charging/#exclusive-nodes 

  • Description of the High-Performance Computing (HPC) system Derecho and its batch queues
    https://ncar-hpc-docs.readthedocs.io/en/latest/compute-systems/derecho/
    https://ncar-hpc-docs.readthedocs.io/en/latest/pbs/

  • New user orientation
    https://ncar-hpc-docs.readthedocs.io/en/latest/getting-started/
    https://ncar-hpc-docs.readthedocs.io/en/latest/tutorials/new-user-training/

• DCMIP-2025 project account: UMIC0107

• NCAR provides a Systems Accounting Manager (called SAM) that lets the user tailor the Unix environment (e.g. the choice of the login shell) and provides the accounting statistics for all batch jobs. SAM can be opened via the web page:
  https://sam.ucar.edu/app/login

Logon with your account name, CIT password and DUO

The command 

    gladequota

gives you an overview of the available space in your directories.

The file server attached to cheyenne is named 'glade'. It provides 50 GBs of disk space in your $HOME directory

    /glade/u/home/$USER

In addition, you can utilize up to 2 TBs in the project directory (permanent disk space):

    /glade/work/$USER

Typically, the CESM model is compiled and run in the temporary disk space (files will be kept for a few months and will then be automatically deleted):

    /glade/derecho/scratch/$USER

The scratch space provides up to 30 TBs.

More information on NCAR's 'glade' file system can be found here:

https://ncar-hpc-docs.readthedocs.io/en/latest/storage-systems/

Information on data transfers to other machines and data sharing on the NCAR machines is provided here:
https://ncar-hpc-docs.readthedocs.io/en/latest/storage-systems/data-transfer/

Information on the PBS batch queuing system can be found on the web page
https://ncar-hpc-docs.readthedocs.io/en/latest/pbs/

Jobs can be submitted to the PBS queues via the command

    qsub  job-script

where the file 'job-script' includes a series of PBS commands, like the specification of the number of processors, selected queue, wallclock limit and the job account number.

Our account number for the class is:

#PBS -A UMIC0107             # DCMIP-2025 account number 

Special batch queue for DCMIP on casper

workshop  (maximum run time is 12 hours)

Helpful commands to monitor or control your jobs are:

List your unfinished jobs or list all of your pending, running and finished jobs

    qstat

    qstat -u $USER

    qstat -xwu $USER

List all jobs in the queue 'workshop'

    qstat -q workshop

Kill a running job identified by its job_ID, or kill all of your pending and running jobs (use backticks as shown):

    qdel job_ID
    qdel `qselect`

NetCDF is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data:

http://www.unidata.ucar.edu/software/netcdf/

In computational science it is crucial to stick to standards to ease the exchange of data and metadata. In atmospheric science, a common netCDF standard is the netCDF Climate and Forecast (CF) metadata convention. The model output of NCAR's Community Atmosphere Model (CAM) is CF-compliant (with one minor exception).

NetCDF output can be converted to ascii via the 

ncdump netcdf-file | more

ncdump netcdf-file > netcdf-file.txt

command. These commands write the content of the netcdf file either to the screen or to the txt file. Just looking at the header information of the netcdf file can be done via

ncdump -h netcdf-file

ncdump -c netcdf-file

Sometimes it can be useful to manipulate a netCDF file, e.g. in order to extract a single variable or connect two netCDF files. This can be conveniently done via the netCDF operators nco. The complete documentation is provided on the page:

http://nco.sourceforge.net/nco.html

Examples of their use are:

• Rename a variable name via ncrename
  Example: rename the variable (-v) 'temp' to the new name 'T' in file file.nc, the NCO command will not (-h) be appended as a global 'history' attribute:
  ncrename -h -v temp,T file.nc

• Rename a coordinate variable via ncrename
  Example: In order to rename a coordinate variable 'level' to 'lev' so that it remains a coordinate variable, you must separately rename both the dimension (-d) and the variable (-v):
  ncrename -h -d level,lev -v level,lev in.nc 

• Changing or adding global attributes via ncatted
  Example: Add the global attributes 'horizontal_resolution' and 'grid' and set them to 'medium' and 'latlon'. If a global attribute with the same name is present is will be overwritten (o), if it is not present it will be created, the information 'medium' or 'latlon' is a character string (c)
  ncatted -h -a horizontal_resolution,global,o,c,"medium" -a grid,global,o,c,"latlon" file.nc
  Even more attributes can be added or changed at once.

• Delete all existing global attributes via ncatted
  Example: Delete (d) all global attributes in file in.nc
  ncatted -h -a ,global,d,, in.nc

• Changing or adding attributes of a variable via ncatted
  Example: Change or add the 'long_name' attribute of the 'T' variable to 'Temperature' in file file.nc
  ncatted -h -a long_name,T,o,c,Temperature file.nc
  
  Example: Change or add the 'units' attribute of the coordinate variable 'lev' and set it to 'level', set the 'positive' indicator of the 'lev' variable 'lev' to 'down' which indicates that the 'lev' fields increases in the downward direction in file file.nc
  ncatted -h -a units,lev,o,c,level -a positive,lev,o,c,down file.nc

• Exclude variables from the netCDF file and create a new output file via ncks
  Example: Create the netCDF file out.nc containing all variables and any associated coordinates, except (-x) the listed variables (-v), from the netCDF in.nc
  ncks -h -x -v co2vmr,n2ovmr,isccp_prs,isccp_prstau,isccp_tau in.nc out.nc

• Extract variable from a netCDF file via ncks
  Example: Extract variables time and PS from netCDF in.nc. If out.nc does not exist it will be created. Otherwise you will be prompted whether to append to or to overwrite out.nc. All relevant coordinate information will be kept.
  ncks -h -v time,PS in.nc out.nc