Application Benchmarks

Answering the Question: Where should I run my compute job?

Every compute job is different, the best way to determine where your compute job will run best is to perform benchmarking with your specific application. This page was created to give users a general idea of performance on various systems. In addition the performance of jobs can be influenced by overall system usage.

CPU Performance

File Write Performance

This benchmark is a simple C program which writes to new files. A file is created and written to until it reaches a specified size, then the next file is created until a specified number of files have been created all of the specified size. This benchmark does not represent random writes, as files are only appended to. The following plots were generated with a python script which runs the program at all combinations of 1-201 files created and 1-201kB memory size at intervals of 10. This was run on various systems writing files to various volumes. Currently each individual point represents a single run of the C program (no replicates).

Compute Node Coeus

Write intensive jobs on coeus should write to either scratch or local-scatch, data that needs to persist or be backed up should be moved to an appropriate location after job completion.

Scratch is an nfs volume, local-scratch is local to each compute node. Because of this local-scratch has lower latency than scratch, and is not impacted by other users as scratch might be.

Phi Nodes Coeus

It seems like there might be something wrong with local-scratch on the phi nodes, or even the phi nodes in general, performance is very bad. File write performance on phi nodes in general seems very bad, this might be why users are avoiding using them, as many of our users have write heavy jobs.

Standard Linux Compute - Hecate

Performance on the standard linux compute servers is impacted by other users on the system. You can determine how heavily used the system is by running the command 'htop' ('q' to quit). There was a load average between 2 and 3 when these benchmarks were run.

TODO: description of figures. Also, do we want to show writes to home? Or should we not show that to dissuade users from creating jobs that write to home. Can/should we create a local scratch loop back for standard compute like we did for compute nodes?

Note this is comparable to writing to NSF volumes from a compute node on coeus.