Other HPC Resources

Other High Performance Computing Resources

The HPC cluster maintained at CWRU is of a modest size in comparison with other HPC / supercomputer resources and commercial providers.  While our resource serves well for many research uses, it is not of adequate size to handle larger computational tasks that may require several hundred or several thousand simultaneous processors.  For those larger jobs, there are other resources available to CWRU researchers that are both free and commercially available

Feel free to send any inquiry about the following resources to . Our team can help provide a solution that fits your needs.

Important Notes

  • We have use cases of users requiring the max limit of wall clock time to run their jobs. The alternative resources can be handy when HPC is undergoing maintenance and is not able to provide the max wall time.
  • Initially, the short transition phase from HPC to other computing resources can be challenging as other environments may be little different than HPC. Contact us   if you encounter any problem during that process.

State and National-funded computational resources:

OSC (Ohio Supercomputing Center)

The Ohio Supercomputer Center is a statewide resource that provides supercomputing services and computational science expertise to Ohio university researchers as well as Ohio industries.

Any faculty member or research scientist at an academic institution in Ohio is eligible for an academic account at OSC. These researchers/educators may request accounts for their students and collaborators. Commercial accounts are also available. More information about applying for both academic and commercial accounts at OSC can be found at https://www.osc.edu/supercomputing/support/account.

ITS Research Computing can provide free consultation and contacts to request for the OSC access. A start-up allocation is typically provided at 50,000 SUs, with minimal information verification. Further allocation would require a usage proposal and would be granted based on previous successful allocation utilization in the form of publications and peer recognition.

XSEDE (eXtreme Science and Engineering Discovery Environment)

XSEDE integrates resources and services - supercomputers, collections of data, and new tools, makes them easier to use, and helps more people use them.
Submit an allocation request via the XSEDE portal. XSEDE resources would vary from time to time, and currently consists of the following clusters:
  • San Diego Supercomputing Center: Gordon, Trestles
  • Texas Advance Computing Center: Lonestar, Stampede (Xeon Phi)
  • NICS Oak Ridge: Kraken
  • Pittsburgh Supercomputer Center: Blacklight
  • Indiana University: Mason, Quarry
  • Georgia Tech: Keeneland (Nvidia)
To apply for an account, you will need the following information:
  • An estimate of the computing time needed in SUs, or an estimate of the amount of storage needed (in Terabytes, or TB).
  • A short abstract of your computational project
  • The Principal Investigator's CV
Similar to OSC resources, a start-up allocation can be assigned quickly and then further allocation would require annual renewal.

Commercial on-demand computational resources

POD (Penguin on Demand)

Penguin Computing on Demand (POD) provides you HPC resources for which you pay-as-you-go. You don't need to own a powerful cluster to run your jobs even at large scale. POD's compute environment was designed specifically for high-performance computing and features typical HPC components such as low-latency interconnects and GPUs. For optimum performance all jobs are managed by industry leading HPC schedulers that support the job submission semantics of the open source scheduler TORQUE. All jobs are executed directly on POD HPC servers without a virtualization layer in the middle which provides similar environment as CWRU High Performance Computing (HPC) Cluster. Visit POD website for details.

POD Portal

Beyond offering an easy-to-use HPC compute cloud environment, POD provides centralized management capabilities through the POD portal. It also allows users to complete job submission process through a web interface. Refer to POD User Documentation for details.

POD Registration

POD offers easy registration. It requires a valid credit card but it is not charged when selecting free usage tier by submitting the job in a FREE queue (#PBS -q FREE) that includes resources as Intel 2.9GHz Westmere 1248GB 24 Cores for 5 Minutes. For more pricing and queue information, visit POD Cloud Rates and Services and for POD Job Queue

Available Applications 

Pre-installed applications are available via module commands as in CWRU HPC. Find the details for accessing applications here.

Job Submission Options

POD Tools: It enables job submission and queries without first logging into a POD login node.
ScyId Insight: It provides option to submit jobs through web interface from POD Portal. While in POD portal, (i) Launch ScyId Insight button on the left pane, (ii) click on new job, (iii) transfer/upload the required files from your PC, (iv) create and submit a job using job submitter GUI, and (v) download output files.
POD CLI: Submitting the job through bash script from the command line as in CWRU HPC.

POD User Documentation

Refer to this documentation for details.

Amazon AWS (Amazon Web Services, Cloud Computing)

Amazon AWS provides a stack of cloud computing web services that provide customizable, on-demand computational resources in the cloud. It is designed to accommodate bursty, on-demand computing by eliminating the complexity of building a significant hardware investment upfront.

High Performance Computing Service

Amazon AWS provides Elastic Cloud Computing (EC2) cluster resources as their generic offering. Customers with complex computational workloads such as tightly coupled parallel processes, or with applications sensitive to network performance, can achieve the same high compute and network performance while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2. Cluster Compute, Cluster GPU, and High Memory Cluster instances have been specifically engineered to provide high-performance network capability and can be programmatically launched into clusters – allowing applications to get the low-latency network performance required for tightly coupled, node-to-node communication. Please note that Amazon EC2 runs on virtualized environment.

Besides the generic EC2, Amazon AWS also provide many other types of specialized computational resources such as Relational Database Service (RDS), Redshift, Elastic MapReduce, and so on.

To use Amazon AWS services, a potential user needs to create a free Amazon Web Service (AWS) account and also has to submit the payment method (a credit card for any possible incurred fees. This AWS account can be used to access any AWS services, such as Amazon Elastic Compute Cloud (EC2),  Elastic MapReduce, Relational Database Service (RDS), Simple Storage Service (S3), and Glacier. AWS also provides an identity and access management (IAM) console that can let you add groups of existing AWS users with certain administrative and executive privileges. This control access mechanism allows a research group at CWRU to have multiple AWS users of group members without worrying about limited or compromised access.

Storage and Archival Service

Amazon has a couple of tightly-coupled storage systems that work well with EC2 resources, i.e. the EBS (Elastic Block Storage) and the S3 (Simple Storage Service). The EBS is typically bundled with the EC2 purchase, while S3 can also be an independent storage reservation.

The lowest-priced archival storage provide by Amazon is the Amazon Glacier, priced at $120/TB/year. Please note that this price would increase significantly each time the users need to retrieve the data. A user also needs to maintain a permanent client that reflects the Glacier file structure, because retrieving the storage file structure for the latest snapshot would take several hours. The very nice thing about this service is the natural contingency protection against data lost. This provision was generated when the data got spread out to different Amazon data warehouses.

Conceptually, Amazon Glacier serves as the lower tier for the Amazon S3 storage. AWS provides a management console as a web tool that can easily help create vaults and assign sharing limitations for access. Vaults, more familiarly recognized as folders or directories, consist of archives, another name for the uploaded files. Each archive is assigned a unique archive ID that can later be used to retrieve the data. An archive can represent a single file or a zipped combination of several files to be uploaded as a single archive. A single file can be uploaded as an archive, but the overall costs will be lower if files are aggregated in TAR or ZIP formats before uploading to Amazon Glacier. Individual Amazon Glacier archives can range in size from 1 byte to 40 terabytes. The largest archive that can be uploaded in a single Upload request is 4 gigabytes. For items larger than 100 megabytes, customers should consider using the multipart upload capability. Archives stored in Amazon Glacier are immutable, i.e. archives can be uploaded and deleted but cannot be edited or overwritten.An individual account can have up to 1000 vaults and there is no maximum limit to the total amount of data that can be stored in Amazon Glacier.




Subpages (1): Big Data Management
Comments