2.6) Experiment Access Workflow

Preliminaries - Accessing Development Session

As previously described in Section 1.8 and preceding sections, access to the AERPAW facility is batch-mode for all Canonical (Program-it-Yourself) Experiments, and Experimenters are expected to develop their Experiment completely in the Virtual Environment.

After initial development, you can submit your Experiment for Testbed Execution, and AERPAW Ops will schedule a time when your Experiment will execute in the Testbed. Your AVNs will be moved into the AFRNs and APRNs you identified as target Resources, and the code in your AVN will auto-execute exactly as you have programmed in the Development Environment. Typically, this will result in gathering R/F and/or other data, and saving such data in one or more of your AVNs.

After that execution is complete, your AVNs will be retrieved from the AFRNs and APRNs, and you will be notified by an email message. At this point, you should again request a Development Session. Your AVNs retrieved from the testbed will be used to instantiate your Development session this time, so after the Development Session is established, you can access it to see the saved data. You can also iteratively further develop the Experiment (modify or augment the programs), and "Submit to Testbed" again.

Before executing your Experiment in the Testbed, AERPAW Ops may choose to run it in an Emulation Session, and if any problems surface, return it to you without attempting a Testbed Execution. In this case as well, you will receive a mail, and should again request a Development Session to troubleshoot your Experiment.

Access versus Development

The Development actions are specific to specific Sample Experiments (and specific Experiment functionality such as vehicle control, SDR programming, test traffic generation, data collection), and are described in Sections 3 and 4.

In this section, we provide instructions as to the common procedures to enable you to log into the AVNs corresponding to your target AFRN and APRNs, after you have clicked "Initiate Development".

Retrieving Access Information

After the Development Session has been created by the AERPAW Platform, and it is ready for the Experimenter to log in, the Experimenter is notified by an email message. Specific access information to this Session is also attached to the Experiment as viewed through the Portal to enable the Experimenter to access the Experiment.

When you receive the email informing you that your Development Session is ready, log back into the Portal, and navigate to the Experiment Details screen. You will find links to two files. One is an "Experiment Manifest", and the other is an "OVPN Profile":

The Development Session of the Experiment consists of a number of AVNs, one for each AFRN and one for each APRN that your Experiment targeted, that you can separately login to. All the AVNs have an extra network interface, connected by an "Experimenter's Management (XM) Network" - a virtual Layer-2 network individualized to your Development Session. In addition, one additional Virtual Node is created to support your use of the Development Session - the OEO Console. The use of this console is explained briefly below, and in more detail in later sections.

The "Experiment Manifest" simply provides the XM interface IP address of each of these AVNs. The "OVPN Profile" allows you to create an interface on your own work computer that is in the same virtual Layer-2 network, and thus allow you to ssh from your work computer directly into each AVN. You must install appropriate freely available client software (OpenVPN V2 for Linux and Windows, Tunnelblick for MacOS) on your work computer in order to be able to utilize the "OVPN Profile".

Note: In any subsequent Development Sessions, the same Manifest and OVPN Profile will continue to work for you, until the Retirement of the Experiment.

Accessing Development Mode Session

The User's access to the Development Session AVNs and OEO Console consists of two steps:

Establish a VPN connection to the AERPAW Development Environment
Use Secure SHell session (ssh) to log in to each AERPAW Virtual Node (AVN), and the OEO Console.

Establishing VPN Connection

First, save the OVPN Profile locally in your work computer as a plain ASCII text file, name this file logically, say aerpaw_exp.ovpn . Please make sure no extraneous character gets into this file, just the text information as provided. (If you view this file, use an ASCII text editor rather than a word processor of some kind; avoid editing this file at all.)

Next, check that the IP address space the VPN will attempt to commandeer are not already in use in your computer. Refer to the Manifest; the address block of interest is a /26 IP address space that includes the addresses provided for the AVNs and the OEO Console. For example, an address block of interest might be 192.168.108.0/26 . Examine your routing table on your work computer, and ensure that this specific block is not already being utilized. (To examine routing table, refer to the documentation for your networking tools; typical ones are - Linux: route -n , Mac OS : netstat -rn -f inet , Windows: ipconfig). If this block is already being utilized, for another VPN or NAT, you cannot use that work computer to access the AERPAW AVNs.

Next, connect to the AERPAW VPN server specific to your Experiment Development Session. AERPAW uses the Linux openvpn software to provide Layer-2 VPN service directly into the Experiment Development Session resources for the user. You may connect to the custom server created for your Experiment by running an OS-appropriate VPN client with the appropriate functionality on your work computer; provide the "OVPN Profile" downloaded from the Portal to your VPN client.

(Need more info for VPN Client? Expand this section)

On Linux:

Use openvpn. Linux is by far the recommended platform from which to access AERPAW. In our experience, the VPN connection from Linux is stable. Most of our testing was with openvpn 2.4.4 running on Ubuntu 18.04 . The further instructions below assume this combination.

NOTE: openvpn Version 3 will NOT work - it drops support for Layer-3 VPNs. Please make sure to use openvpn Version 2.

Use sudo apt-get install openvpn if you do not already have it installed.

Running the software: Provide the OVPN profile file you have saved above, by running the following at a command prompt (e.g. terminal window):

sudo openvpn --config aerpaw_exp.ovpn

This will create a new logical network interface, likely named tap0 or similar, in your OS. You can check on this with:

ifconfig -a

The interface is created in a DOWN state, and Linux does not automatically request a Layer-3 address on it. This requires two further commands to be executed:

sudo ifconfig tap0 up

sudo dhclient tap0 -v

On Mac OS:

Use Tunnelblick. This free software can be downloaded from https://tunnelblick.net/downloads.html ; we have tested connectivity using the 3.8.6a stable build, on MacOS 10.15.7 . Generally the connection provided by Tunnelblick appears to consistently work correctly with AERPAW's openvpn servers, and has also been stable in our testing. Follow the instructions provided in the Tunnelblick documentation to provide Tunnelblick with the aerpaw_exp.ovpn file. Note: in our testing, we find that after Tunnelblick establishes the connection, and it turns green, there is nevertheless a delay of several seconds before an IP address is assigned to the tap interface, and the interface becomes usable.

Note: The OpenVPN Connect software on MacOS will NOT work - it does not support Layer-2 VPNs.

On Windows:

OpenVPN client for Windows should work fine. We are aware of different kinds of OpenVPN clients and some of them appear not to work well with the standard Linux OpenVPN server that AERPAW uses. The one that has worked without problems is the community edition at https://openvpn.net/community-downloads/ .

Download OpenVPN here and install it. This should create an icon in the system tray for OpenVPN. The openvpn profile file you received as part of the Manifest can be imported by selecting the menu item "Import -> Import file". After the file is imported click "connect" and this should create a new logical network interface in your OS with IP address 192.168.108.xxx for the example provided above.

If OpenVPN does not work, you can also try Windows Subsystem for Linux (WSL), a compatibility layer that enables Linux binary executables to be run on a Windows OS. You must install WSL 2 (WSL 1 provides a thinner compatibility, which will not suffice).

The instructions for manual install of WSL are located at https://docs.microsoft.com/en-us/windows/wsl/install-manual . Some of the steps require that you have System Administrator privilege on your Windows system. Also, at one point your system needs to be rebooted before proceeding.

After you have installed WSL, you need to install Ubuntu Linux on it. Ubuntu 20.04 has been tested for this purpose and appears stable; Ubuntu 18.04 should also work but has not been tested. Install by going to the Windows Store app, searching for Ubuntu, selecting the version you want, and then clicking install. Once installed, launch Ubuntu and you should be asked to setup a username (such as "myapuser") and a corresponding password. Once you have done so, Ubuntu should be all set up. You can use this userid to login to the Ubuntu. When you temporarily need root access, you can use sudo to execute commands as root inside this Ubuntu installation, using the password.

Move the openvpn profile file you received as part of the Manifest inside your Ubuntu file system. From this point on, you should be able to follow the instructions above for Linux above.

Using SSH sessions - Log into each AVN

Once your VPN session has been successfully established, you can ssh in to each of the AVNs of your Experiment Development Session, from separate terminals/windows, or other SSH client applications, at the respective IP addresses provided in the Experiment Manifest file.

In each case, make sure to provide the ssh client with the private key corresponding to the public key that you uploaded into the Portal in a previous step ("Account Authentication and Authorization"). SSH access to AERPAW Development Session nodes is strictly by SSH key, and not by password. (If you provided multiple keys in that previous step, they have all been installed in the AVNs, so you would be able to log in with the private key corresponding to any of them.)

When you ssh in, you will have the effective AND real User ID of "root". As such, please use caution in your actions; you would have the ability to disrupt anything and everything inside your AVNs as well as your OEO Console.

(Need more info for SSH? Expand this section)

On Linux, MacOS:

Typically, ssh is part of the OS distribution, and needs to be executed on the command line. The option to use a specific private key is invoked by the option -i, and a userid can be specified before the target host. For example, assume that on your laptop you have saved your public and private keys in the .ssh directory under your home directory (this is the normal place to save them) with the names my_aerpaw_key.pub and my_aerpaw_key, respectively. (Of these, you would have uploaded the former, i.e. the public key, as "Credentials" to the AERPAW Portal, in an earlier step.) Now, to log in to the first AVN provided in the example JSON above, open a command shell (Terminal window, for Mac OS), and type:

ssh -i ~/.ssh/my_aerpaw_key root@192.168.108.1

(specifies the right key to use) (specifies user) (and target host - first AVN)

This should log you into the AERPAW Virtual Node without any need of using a password. Inside the AVN, you should have logged in as the User "root".

In separate command shells (windows), you can login to each of your AVNs, and the OEO Console.

Note: A common pitfall is that most ssh clients require the private key and the directory it is in to have restrictive file permissions, so that they are truly private to the user. In other words, the .ssh directory, and the my_aerpaw_key file, in the above example, need to have read, write, and (for the directory) scan access only for the user. To ensure this, if you have ssh problems, with your ssh client complaining about your key being insecure, execute the following commands:

chmod 700 ~/.ssh; chmod 400 ~/.ssh/my_aerpaw_key

Note: When you originally created your ssh key-pair, the key generation program you used may have given you an option to locally encrypt your private key file. If you took that option, you entered a passphrase to encrypt your private key, and at this point, you will need to supply that passphrase again, so as to be able to use your private key.

More information and background on ssh can be found in the manual pages of your computer's OS, or at various places on the web, such as here.

On Windows:

There are many SSH clients for Windows such as MobaXterm, XShell, which are known to run properly with the Openssh servers that AERPAW uses. Please refer to the respective documentation or help material for such applications to know how to use them, how to specify the key to use, and how to specify to log in as the user "root". Make sure you point your SSH client application to

(i) the right ssh private key for the public key you uploaded on AERPAW portal

(ii) the right destination IP address for your experiment

(iii) make it use the username “root”)

Alternatively, assuming you are using the Windows Subsystem for Linux (WSL) solution to be able to establish the VPN connection to your experiment in AERPAW, you can also use ssh from inside the WSL environment. If you do, you should follow the instructions above for Linux, including placing the ssh keys in appropriate locations inside WSL, such as the home directory "~".

Log into the OEO-CONSOLE

In addition to the AVNs corresponding to your target Resources, you must also login to the OEO Console. There are two minor differences between accessing the AVNs and the OEO-CONSOLE.

The first is that the IP address of the OEO-CONSOLE always ends with the byte of 62.

The second, optional, step is for forwarding data from the vehicle emulation to a local install of QGroundControl , described in Section 3.1.1. This step is not necessary if a graphical visualization of the vehicles is not desired. In order to forward the vehicle data to the local install of QGroundControl, a port forwarding should be enabled from each vehicle that is to be monitored to the local Experimenter's system. The transport protocol is TCP and the port number to be forwarded is 5760. If multiple vehicles are used, all their data come (multiplexed) through the same 5760 port.

For example, to ssh into the OEO-CONSOLE of the same experiment as above the Experimenter should use:

ssh -i ~/.ssh/my_aerpaw_key -L 5760:127.0.0.1:5760 root@192.168.108.62

For details on how to forward the ports on various platforms, please see this page.

Continuing to Development

Subsequent parts of this User Manual, specifically Sections 3 and 4, provide instructions for developing your Experiment, after you have logged into the AVNs and the OEO Console as above. The information in these sections are arranged as follows. (Before continuing on to them, we recommend you finish reading the remaining subsections of Section 2.)

Section 3 describes the general framework common to all AERPAW Experiments. The part of Section 3 before any of the subsections (i.e., "Experiment Structure Overview" is a perspective on envisioning scenarios that can potentially map onto AERPAW Experiments.

Section 3.1, and its subsections, describe the few categories to which all Sample Experiment software can be seen to belong - Radio, Vehicle Control, Traffic Generator, and Data Collection. Section 3.2 complements it by showing how these categories of software fit together, and how the Experimenter may monitor, control, and generally orchestrate them using the OEO Console. Together, these two subsections provide a structure in which all the software described in Section 4 fit in.

Section 3.3 provides a specific sample integration of examples of the different classes of software. Section 3.4 expands on running these pieces of software. Section 3.5 provides specific help in data post-processing.

After this, you are ready to look at information specific to each Sample Experiment, as presented in various parts of Section 4.

Page updated

Report abuse