2.4) Creating and Managing Experiments

Experiment Related Actions

Any AERPAW User (who has at least an Experimenter Role) can create Experiments in any Project of which they are a member, and join (or be added proactively to) any Experiment in any Project of which they are a member.

An Experimenter that creates an Experiment becomes its Owner.

The Experiment Owner can proactively add other Members of the same Project to be Members of the Experiment, or remove membership from existing Experiment Members.

Any AERPAW User can see a list of all Experiments in each Project of which they are a Member, and can request to join any such Experiment as a Member of the Experiment. Experiment Members need to approve or deny such requests. Any Experiment Member can approve or deny such requests.

Any Experiment Member can define the Experiment, but this can only be done before triggering the first Development Session of the Experiment. In this context, "define" indicates identifying the specific Testbed resources to be targeted by the Experiment (as discussed in Section 1.8 previously).

Any Experiment Member may view the current status of the Experiment, perform Session-related activities (start, save, exit, submit to testbed, etc.) on the Experiment, or log into the AVNs of a session of the Experiment executing in the Virtual Environment.

List of Experiments

Any AERPAW User (who has at least the Experimenter Role) can view a list of Experiments by clicking "Experiments" on the Navigation Bar. Each User will see a list that contains ALL Experiments in every Project that this User is Creator/Owner/Member of.

CAUTION: If the list is long, only the first few entries may be listed; below the list, navigation buttons (such as "Page 2 >>") allow you to visit later and earlier parts of the list. If you know the name of the Experiment you are looking for, you can also enter it in the "Search" box to filter down the list.

Here is an example of the List of Experiments; we see that this User can see several Experiments from one Public Project, and one Private Project; they can request to join any of these except one which has been already retired:

Creating Experiments

Although Experiments can be viewed in the above list, they need to be created from the Project Details screen for the Project they are to be created within, as previously mentioned in Section 2.3 of this User Manual.

Clicking the "Create" button (under the "Experiments" section) of the Project Details screen brings up the Create Experiment screen:

Please fill out the AERPAW Experiment Information Request Form by clicking the link on the Create Experiment screen:

Failing to fill out the above form with meaningful information will cause your Experiment's execution session requests to be delayed or declined at subsequent points in the Experiment lifecycle.

Note: Depending on whether you are creating a custom Experiment or not, the AERPAW Experiment Information Request Form may redirect you to fill out a different form instead - follow the instructions on the form:

The next steps differ depending on the type of Experiment you are creating (discussed in Section 1.8 previously).

Non-Canonical Experiments

If you are creating a Non-Canonical Experiment, please log out of the Portal after filing out the AERPAW Experiment Information Request Form, and await communication back from the AERPAW team regarding further planning your non-canonical experiment. Do NOT hit “Save” on this screen !

Canonical Experiments

If you are creating a Canonical Experiment, after filling out the AERPAW Experiment Information Request Form, please continue by giving your experiment a short Name and a short Description, and click "Save", in order to continue to the step of defining the Experiment.

The Name should be meaningful to you, as well as sufficiently specific so that in any conversations with AERPAW Ops, you may helpfully refer to it by name. The Description should ideally include some information as to what you are trying to accomplish, what the ARNs and vehicles might be expected to do during the Testbed execution. If your Experiment is based on a provided Sample Experiment (highly recommended, see below), then it would be very helpful to include information on what that Sample Experiment is, and how you may have modified it.

Clicking "Save" on the Create Experiment screen will bring up the Experiment Detail screen:

The Experiment Details screen has several sections. The top section provides summary information about the Experiment, the Dashboard section provides information regarding execution sessions for this Experiment, and the Overview section allows viewing and modifying Experiment membership and resources. The various actions can be taken by means of different buttons on this screen. When an action is unavailable, the corresponding button is colored light grey, and cannot be clicked.

Note: When active, different buttons have different colors such as red, green, blue. For some buttons, the active color is dark grey; please note the difference between dark grey (active) and light grey (inactive) for those buttons.

Target Resources - Defining an Experiment

As previously mentioned in Section 1.8, every Canonical Experiment has to define at the outset what specific Testbed Resources the Experiment is intended to ultimately execute on.

The Virtual Experiment created to support the Development Mode Execution Session of an Experiment is customized to the specific Testbed Resources that are so targeted by the Experiment. For this reason, the set of targeted Resources can only be specified before any Development mode execution sessions have been started. Once the first Development session is triggered by an Experiment Member, the set of targeted Resources are frozen for the remaining lifetime of the Experiment.

List of Testbed Resources

For Canonical Experiments, the available Resources are AERPAW Fixed Radio Nodes (AFRN), AERPAW Portable Radio Nodes (APRN), and vehicles (UAVs or UGVs) for APRNs (vehicles are optional for APRNs). Information about these have been previously presented in Sections 1.3, 1.4 (and subsections). Information about the locations of AFRNs is in Section 1.1 .

Each AFRN Resource is named either CCx or LWx (where x is a number), to indicate specific AFRNs in Centennial Campus and Lake Wheeler Road Fields, respectively.

Each APRN Resource is named either LPN (Large) or SPN (Small) - they have similar functionality but the LPN represents a little more computational power, but is also heavier (so flight time for UAVs carrying them is more limited). SPNs may also have custom unique radio functionality (instead of the standard SDRs in the LPN) - this is indicated in the naming of each SPN resource.

Each vehicle Resource is of type either UAV (Aerial) or UGV (Ground), and is named to indicate whether it is a Multicoptor (Aerial) or Rover (Ground), and whether it is a Large or a Small type, or Fast or Slow type.

You can see a list of all available AERPAW Resources by clicking the "Resources" item in the Navigation Bar:

Note: Not all resources can be targeted to be part of Canonical Experiments. In the list, a column "Class" provides information regarding this.

CAUTION: If the list is long, only the first few entries may be listed; below the list, navigation buttons (such as "Page 2 >>") allow you to visit later and earlier parts of the list. If you know the name of the Project you are looking for, you can also enter it in the "Search" box to filter down the list.

Deciding Resources to Target

There are various considerations in deciding what Resources to target for your Experiment. A primary consideration is that for all Experiments including air mobility (i.e. UAVs as vehicles), only the Lake Wheeler Laboratories area is available for AERPAW Experiments in the immediate future; thus for such Experiments, you should only choose one or more of the LWx nodes (not CCx nodes).

In your initial use of AERPAW, we very strongly recommend that you simply attempt to replicate one of the provided Sample Experiments in Section 4. The corresponding page on this User Manual will tell you which Resources to target.

In subsequent uses, you might modify the Sample Experiment in order to produce your own customized Experiment, by infusing changes in the source code. At that time, you will be able to take an informed decision regarding whether to keep the same set of targeted Resources, or change them.

(For example, if you originally did a channel sounding experiment involving LW1 and LW2, then for a subsequent Experiment, you might want to keep those same target Resources if your customization is to use a different traffic generator, or a different modulation. Or your customization might be to check the channel at a different geographical location, in which case you would specify LW3 and LW4 instead (say), and keep the Experiment code the same.)

Adding and Removing Targeted Resources

To define an Experiment, click the "Update" button next to "Targeted Resources" in the Overview section, which brings up the Targeted Resources screen:

Click the "Add/Update" button to view a list of Testbed resources available for targeting, and choose some number of them to add to the target list for the Experiment:

Once you click "Save", you return to the Targeted Resources screen, now showing your selected Resources. In this case, the User has chosen the AFRN called "LW1", and two APRNs of the "Large" type:

A numbering is automatically established for the various ARNs in your Experiment. This will later enable you to know which AVN (provided for you in the Virtual Environment, for a Development Session) corresponds to which of your target ARNs. You cannot change this numbering.

Specifying Details for Nodes

For each ARN, you can view/edit some details. While reasonable default values are provided for each, you should make sure these are what you want for your Experiment.

To view/edit the details of each ARN, locate the ARN on the list of targeted Resources, and click the corresponding "Modify" button; this brings up the Target Resource Detail screen for that Resource:

Name: This is the name that will be displayed for the Resource subsequently, defaulting to the Testbed Resource name. You can change it to something that is more meaningful for you. In this case we see the User has chosen to rename "LPN1" to "My-source-node_LPN1".

Node UHD: In the past, different Sample Experiments required the use of different versions of the hardware driver for the SDR at various ARNs, and the User was required to enter the correct version for each ARN. While we may need to revert to this in the future, at this time, we have streamlined the process so that the User need not enter this information. The value in this field is non-operative ("don't-care"), and you can safely leave the default value in there.

Node Vehicle: The default is "vehicle_none", and this is the only possible option for any AFRN (you will get an error if you try to specify otherwise). The other two options are "vehicle_uav" and "vehicle_ugv". If you need an APRN to be mobile during the Experiment, choose the appropriate vehicle. If not, keep the "vehicle_none" default. (There is also a "vehicle_other" choice to accommodate custom future extensions; please do not choose this - it will be equivalent to "vehicle_none" if you do.) In this case we see the User has chosen "vehicle_ugv" for this APRN.

Node Type: This identifies whether the target is an AFRN or APRN, and cannot be changed on this screen. Changes to how many ARNs of what kind are targeted for the Experiment have to be made on the Targeted Resources screen.

When you click "Save", you are returned to the Targeted Resources screen, now showing your updates:

Managing Sessions

Any single Experiment can execute in various environments in AERPAW. Each Execution Session is of one of four types: Development, Sandbox, Emulation, Testbed. These were explained previously in Section 1.3.

As explained in Section 1.3, at this time, the Sandbox environment of AERPAW is still not available to Experimenters, and the Emulation environment is not accessible by Experimenters (it exists to enable AERPAW Ops to execute a Session of the Experiment in the Virtual Environment), so most Experimenters will not need to explicitly utilize the Emulation execution environment.

The main workflow of Experimenters, as explained in Section 1.8 previously, is to trigger a Development Session, login to the AVNs and develop/customize the Experiment code, Save and Exit the Development Session, and then Submit the Experiment for Testbed execution. AERPAW Ops will then schedule the Experiment for a Testbed Execution Session, and subsequently trigger this Session, after the conclusion of which the Experimenter will receive a message and email. After this, the Experimenter should trigger another Development Session, in which they can login and view any collected data/results from the Testbed Execution. At this point, the Experimenter can iterate over further repetitions of the above workflow - further modifying the Experiment, and Submitting to Testbed again, re-examining results, and so on.

The Dashboard section of the Experiment Detail screen allows the Experimenter to execute the various aspects of the above Workflow:

Status

Every Experiment has a unique Status at any point of time, and this is shown in the top section of this screen.

Saved indicates that the Experiment is not executing in any active Session. This is the status of an Experiment immediately after creation. It is also the status to which it returns after some running Execution Session (either in the Development or Testbed Environment) completes, or is explicitly exited by the User.

Waiting indicates that the Experimenter has made some request that requires action by the automated Platform Control and/or AERPAW Operations, and the Experiment is currently waiting for that action. There are multiple variants of the Waiting status; for example an Experiment could be waiting to be instantiated in the Development Mode, or waiting to be saved, or to be scheduled by AERPAW Ops for Testbed Execution, or waiting for the time of a scheduled Testbed Execution to arrive.

Active indicates that a Session for the Experiment is actively executing in some environment - Development, Emulation, or Testbed.

The Experiment can be in only one status at any given time. (For example, you cannot submit your saved experiment to the Testbed, and then start a Development Session to continue developing it while it is also waiting to be scheduled for Testbed Execution.)

Triggering Status Changes

The Dashboard provides buttons to allow the Experimenter to trigger actions that are appropriate for any given Status.

In Saved status, the Experimenter may either "Initiate Development" in the Virtual Environment, or "Submit to Testbed". The exception is that the latter option does not become available until after at least one Development Session is successfully executed. (Advanced Users may consider Submit to Emulation; if you think you want to do so, please get in touch with AERPAW Ops to discuss this, before doing so.)

In Waiting status, there is no action for the Experimenter to take in the normal workflow, but the Experimenter may "Cancel" the request (that put the Experiment in the Waiting Status) to return it to the Saved status - for example, if you realized that your code has a problem that you need to resolve before trying to execute a Testbed Session you have already requested.

In Active status, there is no action the Experimenter can take if an Emulation or Testbed Session is active. However, if the Experiment is executing in the Development Session, the Experimenter can "Save" or "Save and Exit" the Experiment. A "Save & Exit" to return the Experiment to Saved Status is a necessary pre-requisite before "Submit to Testbed". A "Save" is just a mechanism to allow checkpointing an intermediate state of the Development Mode Experiment.

CAUTION: "Save" and "Save & Exit" are very costly operations. Each imposes a significant load on the Virtual Environment back-end, and takes a good amount of time; it is of the order of 20-30 minutes, or even more depending on the number of Target Resources in your Experiment. During this time you will also lose all control of your Experiment, and once triggered you cannot cancel this action. Please exercise judgement in using this functionality; ideally, do not use "Save", rather use "Save and Exit", and do it only when you are confident you are ready to "Submit to Testbed".

Note: If the "Initiate Development" button is not active when you expected it to be, please check that you did all the pre-requisite work completely; for example, did you add Target Resources to the Experiment? Did you add Credentials for yourself ?

Note: The very first time you "Initiate Development", it is typically a quite quick operation, taking only a few minutes; this is because a generic "blank" experiment is being created. But subsequent times, it is necessary for your specific Experiment to be retrieved and re-created, therefore it can take a significant amount of time, of the same order as "Save" or "Save & Exit".

Managing Membership

The Experiment Creator can proactively add or remove other Members to/from the Experiment by using the "Update" button next to "Members" in the "Overview" section of the Experiment Detail screen. The process is similar to managing Members for Projects as described in Section 2.3, but simpler because the concept of delegating Ownership does not exist for Experiments. All Experiment Members have equal privileges with the Experiment Creator.

The "Join Requests" section of the Experiment Detail screen allows reactively adding prospective Members, by approving their Join Request. If they deny the Join Request, the prospective Member does not in fact become a Member of the Experiment.

Other Actions

Messages

Most of the above actions generate messages in the form of emails, and/or Messages in the list of Messages that you can view through your User Profile screen.

Retiring an Experiment

When you have finished the planned activity related to an Experiment completely, and will no longer need to request Development or Testbed sessions of this Experiment, you can use the "Retire" button at the top right of the Experiment Detail screen to remove the Experiment from AERPAW, and free up resources.

Page updated

Report abuse