CESM/CAM Model Documentation

Basic Configuration and Installation Steps for CAM

The basic steps to building a model are as follows:

1. Create a case: for DCMIP-2025 you will clone existing configurations via the create_clone command as described on the Workflow page

   • For your information: This differs from the typical procedure to create a new case which we do not use at DCMIP-2025. We just show the regular way here for information purposes (all XXX need to be replaced, see below):

~/CAM_XXX/cime/scripts/create_newcase --compset XXX --res XXX --case ~XXX/your_directory --run-unsupported --pecount XXX --project UMIC0107

2. Set simulation settings via ./xmlchange

3. Modify namelist user_nl_cam

4. Source Code Change (to force analytic initializations)

5. Build the model

   • qcmd -A UMIC0107 -- ./case.build

6. Submit Case

   • ./case.submit

7. FV3/MPAS need postprocessing to fit on lat-lon grid

8.  Play with output file and transfer!

1. create_newcase:

Create a new case in a self explanatory directory with relevant info (example below) with the CAM version identifier XXX

 ~/CAM_XXX/cime/scripts/create_newcase --compset FADIAB --res f09_f09_mg17 --case ~/CAM_XXX_cases/self.explanatory.name --run-unsupported --project UMIC0107 --pecount 128

2. Settings via xmlchange (after create_newcase):

Navigate to ~/CAM_XXX_cases/your_new_case and configure the runtime settings (example below)

./xmlchange DOUT_S=FALSE,STOP_OPTION=ndays,STOP_N=NN

./xmlchange ATM_NCPL=XX

./xmlquery CAM_CONFIG_OPTS

./xmlchange --file env_build.xml --id CAM_CONFIG_OPTS --val "--phys adiabatic/kessler/held_suarez --analytic_ic --nlev=YY"

./xmlchange JOB_WALLCLOCK_TIME=01:00:00

./case.setup

3. namelist change (after case.setup):

Modify/add namelist entries in your case directory which determine the input variables and files: file user_nl_cam
Namelist examples will be made available.

Check whether the namelist settinsg are all correct: 

./preview_namelists #if this hangs then something wrong with user_nl_cam check atm_in and drv_in in /glade/scratch/$USER/case_name/run

Check the settings for the actual simulations before submitting the run:

./preview_run

4. Source Code Changes:

This forces analytic initializations, only need if you want to override the defaults. Copy the appropriate file from a pre-existing directory and change it accordingly.

• During DCMIP-2025: after cloning one of the DCMIP test case configurations go to the SourceMods/src.cam directory (linked from your case directory). In case you would like to make changes to the analytic initial conditions change the ic_* file in the SourceMods/src.cam directory.

5. Build the model:

Navigate to ~/CAM_XXX_cases/your_new_case and build the model (in a batch job). The build process takes a few minutes. The compilation is executed on a compute node. Wait until the compilation finishes. You should see the message MODEL BUILD HAS FINISHED SUCCESSFULLY

qcmd -A UMIC0107 -- ./case.build

If  the build step failed, check the atm log file in /glade/scratch/$USER/CAM_XXX.self_explanatory_name/bld/ for the details (at the end of the log file), debug your code and build again until successful.

6. Submit Case via ./case.submit:

Navigate to ~/CAM_XXX_cases/your_new_case and submit the model

./case.submit

To check the status of your batch job. Q indicates that the job is in the queue with name 'regular', an R appears for a running job:

qstat -u $USER

Output file should be in /glade/derecho/scratch/$USER/XXX/model_name/run/ with the 'h0' in it, where XXX was defined in your output-root at the beginning. Default is to be in the scratch folder with the model name. Remember to change the name of this file to another name, because when you run again it will overwrite this file name.

mv /glade/derecho/scratch/$USER/XXX/model_name/run/old_name.nc /glade/derecho/scratch/$USER/XXX/model_name/run/new_name.nc

7. Transfer Files:

• In case you want to transfer files from Derecho or Casper to your local laptop or to a different server, this can be done in various ways. For the transfer to your laptop use the scp Unix command (secure copy). Open an Xwindow on your laptop, change into a local directory of your choice where the file(s) should be copied to (via the cd command) and enter:
  scp user_name@derecho.hpc.ucar.edu:your_glade_directory/file_name .
  or
  scp user_name@casper.ucar.edu:your_glade_directory/file_name .
  where user_name is your name on Derecho/Casper, your_glade_directory is the directory, file_name is the desired Derecho/Casper file, and the dot at the end states that the file will be placed into your local laptop directory.  You will be asked for the CIT password and the DUO authentication.
  The scp command is good if you only want to transfer a few files. Note that multiple files can also be bundled into a so-called tar archive file before the data transfer, like
  tar cvf my_ncl_scripts.tar *.ncl
  in the selected Derecho directory. This bundles all files with extension .ncl and creates the new, single file my_ncl_scripts.tar which can then by transferred. Typically the tar file can be unpacked on the local Mac or Windows machine when double clicking it, or via the submission of the following command
  tar xvf my_ncl_scripts.tar

• If you need to transfer very big files or many files to a server (not your laptop), e.g. let us assume you have a Globus account on your server
  https://www.globus.org/data-transfer

Sign into Globus with your credentials, and a GUI enables you to select the /glade/* directory on Derecho and a directory on your server. The file transfer can then start from Globus after the authentication on both machines.

Example CAM-FV Model Configuration and Installation Steps (for informational purposes, not followed step-by-step during DCMIP-2025)

Instructions for an example installation. Symbols and commands in red point to Unix or CESM configuration commands which need to be executed. This installs and runs an adiabatic baroclinic wave configuration.

1. Familiarize yourself with the NCAR CESM Tutorial notes (see links above). As a starting point, the 'Practical' lecture by Alice Bertini (from 2018) or the lecture from the 2021 tutorial can serve as a guide:
   https://www.cesm.ucar.edu/events/tutorials/2018/files/Practical1-bertini.pdf
   https://www2.cesm.ucar.edu/events/tutorials/2021/coursework.html, select the 'Practials' by Kate Thayer-Calder, Christine Shields and Cecile Hannay (see also the direct links in the previous CESM Tutorials block)

2. Logon to Derecho (with either -Y on Mac laptops or -X on WIndows laptops)
     ssh -Y -l user_name derecho.hpc.ucar.edu
   Enter the CIT password when asked for the Token and authenticate via DUO.
   You will be in your home directory once you are on the machine, check via the command pwd which should show
     /glade/u/home/user_name
   The symbol ~ also stands for your home directory. You can always return to the home directory from any subdirectory when typing cd

3. Open 2-3 xterm windows via
     xterm &

in the login window.

4. Copy a configuration file for NCL graphics application into your home directory (only needs to be done once):
       cp /glade/u/home/cjablono/.hluresfile ~/.hluresfile

5. Download instructions 

Get a recent cam_development branch, see also the instruction on the page https://github.com/ESCOMP/CAM. We slightly modify these instructions to download a specific tag (cam6_04_093 as an example below) of the development branch instead of a released branch, see the tagged versions here:
https://github.com/ESCOMP/CAM/tags

git clone https://github.com/ESCOMP/CAM CAM_6_4_093_20250519
cd CAM_6_4_093_20250519
git checkout cam6_4_093
./bin/git-fleximod update

Note that this is a download of the atmospheric component CAM, and not a full download of the coupled CESM model (which we save for later).

7. We are now ready to configure the CAM model. Start by creating a new case. We get some guidance from the CESM 'Simpler Model Configurations' web page https://www.cesm.ucar.edu/models/simpler-models/fkessler/index.html.
   However, instead of configuring a moist baroclinic wave with Kessler microphysics (as explained this web page, the FKESSLER compset) we select a dry adiabatic configuration (the FADIAB compset) which we start from the dry idealized baroclinic wave initial condition by Ullrich et al. (2014).
   
   The first step is to create a new case (note that the double dashes are important, this is one line):
   ~/CAM_6_4_093_20250519/cime/scripts/create_newcase --compset FADIAB --res f09_f09_mg17 --pecount 256 --case ~/CAM_6_4_093_20250519_cases/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016 --run-unsupported --project UMIC0107
   This creates a new case which we call 'CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016. It uses the compset FADIAB which describes an atmosphere-only simulation without any physical parameterizations. The resolution setting --res determines the choice of the dynamical core ('f' stands for a finite-volume dynamical core on a latitude-longitude grid) with the horizontal grid spacing of 0.9x1.25 (lat x lon) degrees. 30 vertical levels are used by default (can be changed). The project number is our DCMIP account code for the computing resources. You can check out the links above that point to the CESM compsets and resolutions. The option --run-unsupported is always useful, even needed. This configuration sets the number of processors to 256 (or 2 Derecho nodes with 128 processors each) for the f09_f09_mg17 setting. A 15-day simulation will then only take 1min:33sec.

8. Change into the new case directory via
     cd ~/CAM_6_4_093_20250519_cases/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016
   Configure and check the runtime settings via xmlchange commands. Specify:
     ./xmlchange DEBUG=FALSE,DOUT_S=FALSE,STOP_OPTION=ndays,STOP_N=15
     ./xmlchange --file env_build.xml --id CAM_CONFIG_OPTS --val "--phys adiabatic --analytic_ic"
     ./xmlquery CAM_CONFIG_OPTS
     ./xmlchange JOB_WALLCLOCK_TIME=00:10:00
   The first command sets the simulation period of the model to 15 days and we deactivated short-term archiving of the results (DOUT_S setting). The second command activates analytic initial conditions for the Ullrich et al. (2014) dry baroclinic wave (in combination with a namelist setting specified below).
   The xmlquery command provides information about the configuration choices. We limit the maximum job execution time to 10 minutes.

9. Activate the simulation setting via
     ./case.setup

10. Edit the input namelist file 'user_nl_cam' in your case directory. Add the user settings:
    analytic_ic_type = 'dry_baroclinic_wave_dcmip2016'
    empty_htapes     = .TRUE.
    avgflag_pertape  = 'I'
    fincl1           = 'PS:I','T:I','U:I','V:I','OMEGA:I','T850:I','U850:I','V850:I','OMEGA850:I','PHIS:I','PSL:I'
    MFILT            = 90
    NHTFRQ           = -24
    NDENS            = 2
    fv_nsplit        = 10
    fv_div24del2flag = 4

These settings (1) define the analytically prescribed initial conditions (analytic_ic_type), (2) state that no default output should be produced (empty_htapes), (3) select instantaneous output (avgflag_pertape), (4) select the output fields for the history file h0 (fincl1), (5) allow 90 time samples before the h0 output file is closed and a new one is opened, (5) determine that the output frequency is 24 hours (NHTFRQ, negative is hours), (6) select single-precision output (NDENS), (7) determine that the physics time step is subcycled 10 times in the FV dynamical core (fv_nsplit), and (8) select a 4th-order divergence damping machanism (fv_div24del2flag).

11. Preview the namelists (input parameters for the model run) via
    ./preview_namelists
    and take a look at the namelist files in the scratch directory
    /glade/derecho/scratch/$USER/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016/run
    for example
    more atm_in
    more drv_in

You can also preview the jobscript settings:

./preview_run

12. Build the model (in a batch job) via
      qcmd -A UMIC0107 -- ./case.build
    The build process takes a few minutes. The compilation is executed on a compute node. Wait until the compilation finishes (about 5-10 minutes). You should see the message
    MODEL BUILD HAS FINISHED SUCCESSFULLY

13. Submit the job to the batch queuing system via
      ./case.submit
    This is a short simulation that should only take a few minutes to run (once the job started).

14. Check the status of your batch job via
      qstat  -u $USER
    
      Output like:
    Req'd  Req'd   Elap

Job ID          Username Queue    Jobname    SessID NDS TSK Memory Time  S Time

--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----

2379531.dadmi* cjablono regular  run.CAM_J*  19382   3 108    --  12:00 R 00:01

Q indicates that the job is in the queue with name 'regular', an R appears for a running job.

The model output will be generated in the
/glade/derecho/scratch/user_name/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016/run
directory. Look for the netcdf files (ending with .nc) that are named 'h0i', 'h1i', .... (you will only have h0i). These are so-called history files and contain the output data. Go to a different xterm window and change into your scratch directory via

   cd /glade/derecho/scratch/$USER/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016/run

15. Look at the history output files with the visualization tools ncview or ncvis, e.g.
    ncview your_file_name &
    ncvis your_file_name &
    and make yourself familiar with the ncview and ncvis functionality (play with the GUIs).

16. Run an example Python or NCL script to visualize the simulation.
    When using NCL create an ncl directory in your home directory
      mkdir ~/ncl
    and copy the NCL scripts
      cp /glade/u/home/cjablono/climate_589_WN2024/ncl/dry_lat_lon_single_file*.ncl ~/ncl
    Go into the ncl directory via
      cd ~/ncl
    specify the output location of your data in the script (edit), and run the scripts (best in different xterm windows to have them side-by-side)
    ncl dry_lat_lon_single_file.ncl
    ncl dry_lat_lon_single_file_zoom.ncl
    ncl dry_lat_lon_single_file_ps_t850.ncl
    On some Mac laptops the graphics in the X11 window are unfortunately corrupt which is a problem with the Mac's XQuartz window application (version 2.8.5). This problem apparently does not exist in older XQuartz versions. The problem went away when switching to the older XQuartz version 2.8.4.
    By default, the scripts plot some selected fields (mostly at 850 hPa, day 9) to the screen. The first script plots the whole globe in an equidistant cylindrical projection, the second script zooms into a portion of the domain.
    Create png files via
    ncl level=850 day=9 'pfmt="png"' dry_lat_lon_single_file.ncl 
    ncl level=850 day=9 'pfmt="png"' dry_lat_lon_single_file_zoom.ncl
    ncl  day=9 'pfmt="png"' dry_lat_lon_single_file_ps_t850.ncl
    The png file can be viewed via
      display your_filename.png &
    Other possible formats for the pfmt option are "pdf" or "eps". These files can be viewed with the viewer evince (on Casper), encapsulated postscript files with either gv (ghostview, Casper) or gs (ghostscript, Derecho or Casper). The ncl example plot is posted below. The ncl script can be customized or called with varying input parameters like day=7 (for the visualization of day 7), a different pressure level like level=700 (for 700 hPa), or a different output format like 'pfmt="eps"' or 'pfmt="pdf"' as mentioned above. The output format png is best for the posting to the Wiki webpage.

17. Consider installing the visualization tool Panoply on your laptop and become familar with the GUI. This does not work on the NCAR machines.

18. In case you want to modify the initial conditions (via a code change):
    Change the source code (the initial conditions), recompile the code, rename the first output file (otherwise it will be overwritten), re-run the simulation.
    Copy the original initial data for the Ullrich et al. (2014) baroclinic wave routine into the SourceMods/src.cam directory
    cp ~/CAM_6_4_093_20250519/src/dynamics/tests/initial_conditions/ic_baroclinic.F90 ~/CAM_6_4_093_20250519_cases/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016/SourceMods/src.cam

19. Edit the file with an editor like nano or vi (this requires you to learn a few vi basics. Here is a webpage that explains a few essential vi commands: https://www.cs.colostate.edu/helpdocs/vi.html
    vi ~/CAM_6_4_025_20240829_cases/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016/SourceMods/src.cam/ic_baroclinic.F90
    Change the initial conditions, e.g. change the temperature (in Kelvin) at the equator to T0E = 290. (default is T0E = 310.). This will impact the baroclinicity of the flow and the evolution of the baroclinic wave (slower evolution).

20. Rename the first output file (this is one line)
    mv /glade/derecho/scratch/$USER/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016/run/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016.cam.h0i.0001-12-27-00000.nc  /glade/derecho/scratch/$USER/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016/run/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016.cam.h0i.0001-12-27-00000.T0E_310.nc 

21. Recompile the model (from ~/CAM_6_4_093_20250519_cases/CAM_6_4_093_20250519_fv09L30_dry_bw_dcmip2016)
      qcmd -A UMIC0107 -- ./case.build

22. Rerun the model via
      ./case.submit
    and analyze.

23. In case you want to transfer files from Derecho/Casper to your local laptop or to a different server, this can be done in various ways. For the transfer to your laptop use the scp Unix command (secure copy). Open an Xwindow on your laptop, change into a local directory of your choice where the file(s) should be copied to (via the cd command) and enter:
    scp user_name@derecho.hpc.ucar.edu:your_glade_directory/file_name .
    or
    scp user_name@casper.ucar.edu:your_glade_directory/file_name .
    where user_name is your name on Derecho/Casper, your_glade_directory is the directory, file_name is the desired Derecho/Casper file, and the dot at the end states that the file will be placed into your local laptop directory.  You will be asked for the CIT password and the DUO authentication.
    The scp command is good if you only want to transfer a few files. Note that multiple files can also be bundled into a so-called tar archive file before the data transfer, like
    tar cvf my_ncl_scripts.tar *.ncl
    in the selected Derecho directory. This bundles all files with extension .ncl and creates the new, single file my_ncl_scripts.tar which can then by transferred. Typically the tar file can be unpacked on the local Mac or Windows machine when double clicking it, or via the submission of the following command
    tar xvf my_ncl_scripts.tar

24. If you need to transfer very big files or many files to a server (not your laptop), e.g. let us assume you have a Globus account on a specific server, it is better to use the 'Globus' application
    https://www.globus.org/data-transfer

Sign into Globus with your credentials, and a GUI enables you to select the /glade/* directory on Derecho and a directory on your server. The file transfer can then start from Globus after the authentication on both machines.

25. Example ncl plot of the control simulation with original parameter settings (figure will be posted soon)