Master's Project

High Performance Computing with Microsoft Project

Abstract:

It is difficult for many scientific researchers to perform large scale analysis. One major concern for these end users is their productivity in designing and executing applications to perform analysis. These applications benefit greatly from concurrent execution in a distributed computing environment. The productivity of some of these researchers will be hindered if they are to learn how to use a command line tool or text based programming language. They simply wish to focus on science rather than learn another tool. Recently, some frameworks and systems are being used to create and run these applications. However, these tools are still difficult to learn. We present Cyberaide MSProject. We will show how Cyberaide MSProject leverages the Microsoft Project software package to create and run workflows on the Grid. We present an evaluation to determine its overall usability.

Project Committee:

Chair: Hans-Peter Bischof

Reader: James Heliotis

Observer: Walter A. Wolf

Status:

Report, Presentation

Progress:

  • Week 1
    • Started working with hierarchical Workflows in Microsoft Project. I have added support for sub-workflows in Microsoft Project.
    • Created a new type of task for data transfers.
    • I was able to simplify execution of Gram jobs on a cluster. New commands hide the complexity of a cluster.
    • I was able to tap into mediator web services using WCF. I am currently working on some issues regarding security headers. How to use WSE or WCF in VS2008 to solve this issue?
  • Week 2
    • Did some documentation.
    • Added some support for managing input/output files.
    • Also - started working with RSL language.
  • Week 3
    • Will add support for several data structures found in the Karajan language.
  • Week 4
    • Will create a real e-Science workflow for Cyberaide Project with the other team members. Topic: Distributed Water Threat Management.
    • Worked with the team to compile and run Water Threat Management/Epanet on OAK. Attempted to compile and run on TeraGrid resources BigRed and Lonestar. I am having trouble compiling against their versions of MPI. I did not find any batch job submission commands for Torque (qsub) on lonestar, but I did find some commands for LSF job submission (bsub).
    • Successfully did job submission of WTM on BigRed. Working on MPI compilation issues on BigRed.
    • Successful job submission of WTM on OAK using lam.
    • Received response from helpdesk on paralleljob command on BigRed. Successfully did MPI job submission on BigRed using LoadLeveler.
    • Working is issue: scripting softenv commands
  • Week 5
    • Re-wrote makefile and script for distributed water threat management.
    • Started taking performance measurements.
  • Week 6
    • Debugging MGA-Pepanet.
    • Built and ran DWTM on loadleveler on BigRed.
    • Started using the Network Weather Service to make queue predictions.
  • Week 7
    • Ran mga-pepanet on normal queue on BigRed.
    • The NWS appears to be highly inaccurate. Started gathering performance results.
    • resolved some issues related to MPI2 portability on the TeraGrid.
  • Week 8
    • Started investigating deployment of WTM on Abe, Queen Bee, and Steele.
    • Started looking at MPI flavors.
    • Started looking into architecture or MGA-Pepanet.
  • Week 9
    • Single sign on, SSH certificates, Keychain, related technologies.
  • Week 10
    • Installation Testing & Documentation


March 19, 2009