Memex Cluster

Welcome to the Memex cluster!

Memex is a high performance computing (HPC) cluster administered by Carnegie Information Systems staff. It is available to all Carnegie staff for their research. There are 102 shared compute nodes available to all researchers. The cluster was purchased from Dell and installed in early January of 2016. It has been in production since late January 2016.

The origin of Memex

From Wikipedia (https://en.wikipedia.org/wiki/Memex)

The memex (a portmanteau of "memory" and "index") is the name of the hypothetical proto-hypertext system that Vannevar Bush described in his 1945 The Atlantic Monthly article "As We May Think". Bush envisioned the memex as a device in which individuals would compress and store all of their books, records, and communications, "mechanized so that it may be consulted with exceeding speed and flexibility." The memex would provide an "enlarged intimate supplement to one's memory". The concept of the memex influenced the development of early hypertext systems (eventually leading to the creation of the World Wide Web) and personal knowledge base software. The hypothetical implementation depicted by Bush for the purpose of concrete illustration was based upon a document bookmark list of static microfilm pages, and lacked a true hypertext system where parts of pages would have internal structure beyond the common textual format. Early electronic hypertext systems were thus inspired by memex rather than modeled directly upon it...

Hardware, Storage, Networking

Quick facts about Memex:

  • 107 servers
  • 214 CPUs
  • 2568 cores
  • 13,696 Gigabytes of memory
  • 108 Teraflops theoretical CPU performance
  • 5.82 Teraflops theoretical GPU performance
  • More than 1 Petabyte raw storage
  • Lightning fast non-blocking FDR Infiniband network
  • 40 Gigabit Internet connectivity
  • Access to Internet2 and other research networks
  • $1.5 million list price

Please, visit our Infrastructure pages for detailed information.

Software

Memex is running the open source Rocks Cluster Distribution, a Linux distribution intended for high-performance computing clusters. Rocks includes many tools (such as MPI) which are not part of CentOS but are integral components that make a group of computers into a cluster.

The base installation is customized with additional software packages by using "Rolls". These extend the system by integrating seamlessly and automatically into the management and packaging mechanisms used by base software, greatly simplifying installation and configuration of large numbers of computers. The following rolls are installed on Memex:

  • area51
  • base
  • bio
  • ganglia
  • hpc
  • kernel
  • kvm
  • perl
  • python
  • Red_Hat_Enterprise_Linux_6.6
  • slurm
  • web-server

A detailed list of installed software packages can be found here.

Resource Partitioning and Sharing

Memex uses a traditional compute cluster condominium model, where participating owners (departments, staff and their research teams) have priority use of the computing resources they purchased. When those resources are not in use, others can use them. When the purchasing owner wants to use his/her computing resources, jobs run by others on their shares will be killed. Participating owners also have access to shared Memex nodes, along with everyone else. Dedicated storage purchased by departments and staff will not be shared by others. The cluster currently has the following partitions.

CLUSTER - scavanger partition with 102 nodes, maximum 2 hr long jobs

DGE - 10 nodes

DPB - 10 nodes

DTM - 20 nodes

EMB - 5 nodes

GL - 10 nodes

HQ - 5 nodes

OBS - 40 nodes

GPU - 2 nodes

How to Request an Account

To request an account you should contact your departmental representatives:

DGE - Anna Michalak

DPB - Sue Rhee or David Erhardt

DTM - Peter Driscoll or Peter van Keken

EMB - Fred Tan

GL - Ron Cohen or Andrew Steele

OBS - Juna Kollmeier or Andrew Benson

HQ - Floyd Fayton

Once approved, have your departmental representative inform your department’s IT staff, they should be able to take care of the rest for you.

Next steps, Logging in, Submitting Jobs

More information about using the cluster, such as logging in, backing up your files, transferring files to/from the cluster, compiling your code, submitting a job with SLURM, can be found under the Using the Cluster section.