Running models at GFDL is generally done within the Flexible Runtime Environment (FRE). A model configuration, including all of the necessary bits and pieces, is described within an .xml file. Then, a number of FRE scripts are used to check the source code out of the repository, generate the make files for compilation, and generate run scripts.
Quickstart Guide
First, you need to be logged into the supercomputer (the ic).
Next, you need to have the xml for the ‘experiment’ you want to run. ‘Experiment’ simply means the overall configuration of the model as you wish to run it, including all of the nitty gritty details. An xml may contain multiple experiments, which can point to each other through ‘inheritance’, but that gets complicated and will be ignored here.
An xml contains a number of parts.
Setup: points to input and output directories, compilation templates, and to software packages that will be loaded when a script is run.
Within an experiment
description: Simply a text description of the experiment.
cvs: The commands to check out code from the repository
compile: Instructions for the compiler, including the distinction between dynamic memory allocation and static memory allocation (the latter of which is faster at this time)
gridspec: The coarse model uses the new mosaic gridspec format. The gridspec pointer goes to the grid descriptor file, and an additional cpio? file containing the actual gridspecs is included in the fmsData Sets?.
initcond: This points to the initial conditions cpio? file. It needs to contain a number of restart files for the ocean, atmosphere, land and biogeochemistry, that must be on the appropriate grids.
dataTables: These can be separate files, or can be typed directly into the xml. They generally include boundary conditions, mostly (but not exclusively) for the ocean-atmosphere boundary.
fieldTables: These are required to switch on the ocean tracer advection packages, including temperature, salt, and any biogeochemical tracers. They can also include parameter settings, similar to namelists.
fmsData Sets?: This is a list of files, and packages of files, to be loaded at the start of a run. They are generally boundary conditions for the model, including 1d to 4d fields. Some of the files need to be renamed here, to the name the model will be looking for.
namelists: There are a huge number of namelists, each one for a specific module.
The new xmls do not need to be configured for a particular user. Once you have an xml called model.xml, that contains an experiment called experimentname, place it somewhere logical in your home directory, such as /home/user/fms. You can check out the code and generate the make file by typing:
fremake -x model.xml experimentname
This will result in all of the source code being dumped into the directory
model/src
and the creation of a script,
script/mk_experimentname
Simply type in the name of this script, and the model will compile (as long as you’re on the ic). This might take an hour or so.
When it’s done, you can generate a runscript. The settings for this are located in the portion of the xml. There are two types of run, regression and production. Regression runs are for testing the model. Production runs are for research purposes.
For production runs:
simTime, units = the total length of the run desired
npes = the total number of processors used (needs to match the settings in the coupler namelist)
ocean_layout = the processor layout for the ocean (needs to match the settings in the coupler and ocean_model namelists)
segment simTime, units = the length of each individual segment of a run, usually 1 year
runTime = the length 1 year should take to run, used to calculate how many runs can fit in a single batch job submission (currently limited to 10 hours per submission). Be careful – if your job goes over time, due to an unrealistically optimistic runTime, it will be killed and not resubmitted, which then requires a bit of work to fix. Best to err on the side of pessimism by ~20%.
peLayout ocean = this needs to be the same as ocean_layout above and in ocean_model_nml. Not sure why it’s specified three times.
Troubleshooting?