*  a virtual private network, with your own private IP namespace, and

  *  a private global file namespace, for transparently accessing files across wide-area systems.

Unlike other equivalent user-space systems, MyRT can be deployed with no administrator permission, pre-installation of kernel modules (e.g. FUSE), virtual machine monitors/players (e.g. In-Virgo), or TUN/TAP network devices (e.g. OpenVPN).  The goal is to allow any user, with accounts on multiple systems on the wide-area, with possibly different host OS versions/configurations, to overlay novel functionalities easily.  

The ability to deploy in user-space, without administrator permission, allows easy experimentation of novel network OS features in an IT landscape dominated by a few slowly evolving operating systems. The advantage of this approach is particularly evident in cloud infrastructures where only virtual machine instances are provisioned for the user. In these infrastructures, the user does not have "real" administrator permission, disallowing some kernel devices to be installed for example.  

In the current implementation, MyRT uses the shared library preloading mechanism (where available), and the ptrace process-tracing system call, to interpose overlay functionality into the native LIBC library and system call interface.  In particular, MyRT currently packages the systems XUFS, for providing a personal global file name space emulation, and UVPN, for providing a private class-B network emulation. 

A subset of the features in the run-time overlay is also used to support the NSF-funded MyCluster project.

XUFS: eXtended User-space File System

XUFS allows users to mount their HOME directory on a personal Linux/Cygwin workstation/laptop onto a remote computer. 

XUFS is particularly suitable for use on high bandwidth wide-area networks (WANs) because it makes use of aggressive whole-file caching on the remote system, multiple TCP connections for striped data transfers of large files, concurrent pre-caching of small files, and an optimistic cache-coherency protocol for synchronizing multiple cached file on the WAN. Micro and macro benchmarks have shown superior performance when compared with wide-area network file systems like GPFS-WAN, OpenAFS and NFSv4.

Please see [1] for more information on our experimental implementation on the NSF TeraGrid.  Related projects with similar motivation (albeit with different fundamental design choices) include the Jade [3] and Chirp [4] file system.

UVPN: User-space Virtual Private Network

UVPN allows users to create a private class B network.  The system does this by interposing the user customized overlay behaviour into the standard BSD network socket interface, allowing compute nodes to acquire private IP addresses in the network, as well as advertise and exchange message with each other using them. 

The system is particularly suitable for enabling bidirectional network connectivity between compute nodes within multiple disjoint private local networks.  This is because it allows the transparent tunneling of all TCP/UDP traffic through a series of strategically located user-space network routers, where these strategic locations could be gateway hosts into each private network.  Furthermore, the system has been shown to incur little overhead when compared to native in-kernel networking.   

Please see [2] for more information on our private network overlay implementation.

Installation

To try the software, you may download the package (version 0.5.14 for Linux - ALPHA)

      1.  unzip/untar package on host_A and host_B

      2.  Change to the unpacked directory

      3.  Run ./configure

      4.  Run "make" and "make install"

      5.  Add $HOME/myrt/bin to your PATH

All the binaries are installed in $HOME/myrt/ by default.  This can be changed by specifying a different location with the “—prefix” flag when configure is invoked.

XUFS Usage

For illustrative purposes, we will show how the XUFS SSH client (ussh) is used.  This client is just one example of the types of command-line tools that can be created with the MyRT library.

From local host_A:

    A_dir/> ls       
    file1 file2 subdir1
    A_dir/> ussh host_B
    Password:
    XUFS mounting xufs-rhome …
    B_dir/xufs-rhome> ls  
    file1 file2 subdir1

By default, USSH will tunnel all TCP connections through an encrypted port-forwarded SSH channel.  This allows you to access remote systems through firewalls.

However, if there is a range of ports that are by opened at the remote computer, you can improve performance considerably by use raw TCP connections through those open ports.

To do this, you need to do the following:

Set the environment variable XUFS_PORT_RANGE in your .cshrc/.bashrc startup file on the computer with the open port range.  E.g.

export XUFS_PORT_RANGE=50000,51000         

Use the --notun USSH flag when logging into the remote computer.  E.g.

A_dir/> ussh --notun host_B 

UVPN Usage

In the current implementation, UVPN is enabled when the LD_PRELOAD environment variable is defined with the installed UVPN shared object (i.e. $HOME/myrt/bin/libuvpn.so). In addition to this, two setup steps need to be performed:

1.  A UVPN router (i.e. $HOME/myrt/bin/uvpn_proxy) needs to be started for each subnet in your private class B network.  For example, UVPN allows a private network to be created composed of two subnets with prefixes 189.1.x.x and 189.2.x.x.  (You may need to do this in order to bridge between two host private networks)  In this case, a UVPN router needs to be started in each subnet to allow compute nodes within each private network to connect to their respective routers, and the peer routers to exchange application messages destined across subnets.   The figure below illustrates this for two subnets “Network A” and “Network B”.  In this case, two UVPN subnet routers are started on “Host_A” and “Host_B” respectively, allowing the internal compute nodes “Host_A1” and “Host_B1” to exchange network messages between each other, with UVPN tunneling the network traffic through the subnet routers.   

Additionally, in order for the UVPN subnet routers to know how to forward packets between subnets, the system requires a route table file (default: $HOME/.uvpn_route) to be defined with the contact (host:port) information for each participating UVPN subnet router.  An example of this route table file is        shown below:

        189.1.0.0     <host_A:port_A>
        189.2.0.0     <host_B:port_B>

2.  The application also needs to be wrapped in a script with a prologue that defines (a) the contact information of its responsible UVPN subnet router, and (b) its private subnet address.  An example wrapper script, using the client command uvpn_register to get its unique IP address, is shown below:

         #!/bin/sh
         export UVPN_HOME_PROXY=host_A
         export UVPN_HOME_PROXY_PORT=port_A
         SRC_ADDR=`$HOME/myrt/bin/uvpn_register –s –r`
         export UVPN_SRC_ADDR=$SRC_ADDR
         exec $@

Related Publications

1. Edward Walker, “A Distributed File System for a Wide-Area High Performance Computing Infrastructure”, Proceedings of USENIX Workshop on Real, Large Distributed Systems (WORLDS’06), Seattle, WA, Nov. 2006. (pdf)

2. Edward Walker, “Creating Private Network Overlays for High Performance Scientific Computing”, Proceedings of ACM/IFIP/USENIX International Conference on Middleware, Newport Beach, CA, Nov 2007. (pdf)

3. Herman Rao, and Larry Peterson, “Accessing Files in an Internet: The Jade File System”, IEEE Trans. Software Engineering, vol. 19, 1993.

4. Chirp, The Cooperative Computing Lab, University of Notre Dame