Swift/T - Swift for high-performance systems

Swift/T is a completely new implementation of the Swift language for high-performance computing.  In this implementation, the Swift code is translated into an MPI program that uses the Turbine (hence, /T) and ADLB runtime libraries for highly scalable dataflow processing over MPI, without single-node bottlenecks. 

This page is obsolete- go to:
http://www.swift-lang.org/Swift-T

Documentation and Downloads

Language overview

Swift is a naturally concurrent language with C-like syntax.  It is primarily used to manage calls to leaf tasks- external functions written in C, C++, Fortran, Python, R, Tcl, or executable programs.  The Swift language coordinates the distribution of data to these tasks and schedules them for concurrent execution across the nodes of a large system.  

Swift has conventional programming structures- functions, if, for, arrays, etc.  Some functions are connected to leaf tasks, which are expected to do the bulk of the computationally intensive work.  Tasks run when their input data is available.  Data produced by a task may be fed into the next using syntax like: 
y = f(x);
z = g(y); 

If tasks are able to run concurrently, they do: in this, case, two executions of g() run concurrently.

y = f(x); 
z1 = g(y, 1);
z2 = g(y, 2);

Swift loops and other features provide additional concurrency constructs. Data movement is implicitly performed over MPI. 

The Swift compiler, STC, breaks a Swift script into leaf tasks and control tasks.  These are managed at runtime by the scalable, distributed-memory runtime consisting of Turbine and ADLB.  For example, the following code is broken up into the task diagram: 

 
int X = 100, Y = 100;
int A[][];
int B[];
foreach x in [0:X-1] {
  foreach y in [0:Y-1] {
    if (check(x, y)) {
      A[x][y] = g(f(x), f(y));
    } else {
      A[x][y] = 0;
    }
  }
  B[x] = sum(A[x]);
}
 

Differences from Swift/K 

The previous implementation of the Swift language is Swift/K, which runs on the Karajan grid workflow engine (hence, /K). Karajan and its libraries excel at wide-area computation, exploiting diverse schedulers (PBS, Condor, etc.) and data transfer technologies.  
  • Swift/K is designed to execute a workflow of program executions across wide area resources.
  • Swift/T is designed to translate a Swift script into an MPI program composing native code libraries for execution on a single large system.

Migration to Swift/T

Swift/T is able to execute Swift/K-style app functions and is a natural migration from Swift/K.  Key differences include: 
  1. Enhanced performance: 1.5 billion tasks/s 
  2. Ability to call native code functions (C, C++, Fortran)
  3. Ability to execute scripts in embedded interpreters (Python, R, Tcl)
  4. Enhanced builtin libraries (string, math, system, etc.)
  5. Slight syntax differences (a main() function, simplified mappers, etc.)

Performance 

Swift/T has been optimized for performance on the largest available systems, including the NCSA/Cray Blue Waters and the ALCF/IBM Blue Gene/Q.  It has been run at 512K cores producing 1.5 billion tasks/second, where each task is transmitted to a worker over MPI.  

The high performance demonstrated by Swift/T is due to: 
  • A high-performance load-balancer used internally called ADLB, which features a scalable work-stealing network
  • Fast distributed data management and distributed control of data-dependent operations (futures)
  • A host of compiler optimizations in STC 

Language tools

The Swift/T user guide and downloads can be found here: 
http://www.mcs.anl.gov/exm/local/guides/swift.html

Tutorial

There materials from a Swift/T tutorial at the Swift/T CCGrid tutorial site.

Contributing to Swift/T

To contribute code to the project: see HowToContribute.txt in the downloaded bundle.


Comments