Storage

Storage is a very important component of the High Performance Computing Cluster at Case Western Reserve University.  Without it, data cannot be stored or processed or distributed.  The storage also needs to support running thousands of jobs on the cluster concurrently. Check Guide to Storage Quota for monitoring and managing Storage.

This is the main storage information:

HPC/Research Storage

High Performance/General Purpose storage that is mounted on the cluster. The current capacity is ~2.2PB and it has the 7-day snapshot policy and second-site replication for the data protection. This storage houses several important filesystems that form the backbone of the cluster:

1. • /home: location of the default user workspace.
     /home has limited capacity based on the group quota, which is dependent on the member/guest group status.
     Member has 1.2TB hard quota, while guest has 300GB hard quota.
     Group that has hit the hard quota, will no longer be able to write into the workspace, until the quota warning is cleared.
     The best command to check the group quota (and the usage total by the user) is "quotagrp <PI's account>" or "quotagrp2 <PI's account>" for more detailed information for Pioneer.

   • /usr/local:
     All installed software/applications are located here.

   • /scratch: location of the job workspace.
     Typical job has a defined $PFSDIR which resides on the /scratch/<clustername>/jobs/job.<JOBID>.hpc
     All groups have a starting 2TB group quota limit on this workspace.
     Members who need more than 2TB group quota may request additional quota by contacting us.

   • /mnt/vstor: location of leased project workspace
     Any research group may pay for a leased storage volume on the HPC Storage at the prevailing rate (Currently $180/TB/year).
     Having this volume would benefit the research group that needs a large processing/analysis space for their workflow.

Additional Storage Options

Research Dedicated Storage

For research groups that require more than 100TB of storage, an inexpensive option is to acquire 2 storage servers (that replicate each other). For example, 2 storage systems would cost maybe around $60k and can provide ~400TB of storage.

The RDS servers are mounted to login and data transfer servers, and some compute nodes.  Copying data for use in workflows is described at the following link: https://sites.google.com/a/case.edu/hpcc/data-transfer#h.nltp3jo2szbu

Research Archival Storage

If the data is no longer analyzed, it can be placed in a cold/archival storage, to be used rarely. For such cases, the Research Computing group provides Research Archival Storage service.

We use Ohio Supercomputing Project Storage/Tape system for the Archival service, and Globus Data Transfer tool to move the data into the Archive.  The Archive service would force for an encrypted transfer, making the transfer more secure. 

The data from both HPC Storage and Research Storage can be archived using Globus where users can manage their archival process mostly by themselves. 

Visit HPC Guide to Archival Storage for detail information.