Extending Science Through Enhanced National CyberInfrastructure (ExTENCI) is a joint project of the Open Science Grid (OSG) and TeraGrid/XSEDE, funded by the National Science Foundation.

Project Definition            The Team            Documents

This project has completed.  The following report summarizes the project and results

ExTENCI: Extending Science Through Enhanced National Cyberinfrastructure Project Outcomes Report 7/31/2013


This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.


The national Cyberinfrastructure (CI) consists of a nationwide network of high performance supercomputers originally provided by TeraGrid (TG) and now provided by the Extreme Science and Engineering Discovery Environment (XSEDE) and high throughput (large numbers of powerful microcomputer resources) provided by the Open Science Grid (OSG). The CI computers are located at universities and national laboratories. There are also experimental and commercial “cloud” computers that run applications embedded in Virtual Machines that contain the entire software system (e.g. Microsoft Windows or Linux). The CI is funded by the NSF, Department of Energy, and universities. Scientists, researchers, and engineers use the CI to do research to solve scientific and engineering problems using simulations. They also study and analyze data from experiments (such as from the Large Hadron Collider -­‐ LHC) and satellites (such as weather, ocean, land and climate data). ExTENCI’s primary goal was to develop and provide production quality enhancements to the CI to enable specific science applications to more easily use both the OSG and TG/XSEDE or broaden access to a capability to both TG/XSEDE and OSG users. ExTENCI made enhancements in four areas: 1) Workflow and client tools that permit a software application to use OSG, XSEDE and cloud resources; 2) Distributed file systems that provide easy and fast access to scientific data across a wide area network connecting universities and national laboratories; 3) Virtual Machine technologies that hide complexity of applications and allow them to run on different cloud environments; and 4) Job submission paradigms that distribute work to computing resources. Eight institutions provided the ExTENCI project staff.

Intellectual Merit

ExTENCI enhanced the capability and expanded use in four areas and five tools in the CI:

  1. 1)  Workflow and Client Tools

    Swift – a parallel scripting language for running applications on the CI – was enhanced to: 1) dispatch work to either XSEDE or OSG; 2) increase the capacity to handle a larger number of simultaneous jobs; 3) take advantage of OSG’s glidein pilot job system, and 4) make use of OSG’s data services.

  2. 2)  Distributed File Systems

    Lustre/WAN – a filesystem that provides transparent access to data located at another location – was enhanced by adding Kerberos computer network user authentication security. ExTENCI set up hardware and software to support Lustre/WAN services at five sites and installed Lustre servers at two sites. Performance analysis/tuning was

ExTENCI: Extending Science Through Enhanced National Cyberinfrastructure Project Outcomes Report 7/31/2013

completed to enable CMS (LHC high energy physics experiment) science applications at two Florida Universities (FIU and FSU) to run on data from the University of Florida and Fermilab. Three major science applications were tested in this environment.

  1. 3)  Virtual Machines (VMs) and Cloud Technologies

    HTCondor – a specialized workload management system for compute-­‐intensive jobs and the basis of the OSG -­‐ was improved to better support cloud services. ExTENCI created new tools for creating, managing, distributing VMs to clouds; invented a Cloud Dashboard that enables users to start, interact with, and stop VMs via a browser; and created two new cloud services.

  2. 4)  Job Submission Paradigms

    SAGA/BigJob – a standardized language for defining and running applications on the CI – was enhanced by designing and implementing an HTCondor adaptor to provide an interface to OSG thereby enabling SAGA and BigJob users to send application jobs to
    either XSEDE or OSG resources. BigJob was made more reliable and provided with a Gateway to enable scientists to set up pilot-­‐based application runs via a browser interface.

Broad Impact

ExTENCI produced enhancements to the CI that were used by researchers in the science areas of glass material modeling, protein modeling, theoretical chemistry, earth systems science, molecular biophysics, high energy physics, and sociology of science. This science work, submitted using these enhanced tools, consumed about 7 million hours of computer time on the CI. Eight scientific papers were published and over thirty presentations and papers were delivered at conferences.

A PhD student used these tools to do the science behind his successful thesis. A female PhD student and a female MS student are working on their theses. User guides were enhanced or written and tutorials were delivered to train people on these technologies. Ten graduate students supplemented their training by working on ExTENCI. The new software and facilities provided by ExTENCI will remain to serve scientists in the future.