3.2.1) OEO Console
Overview
The OEO system has been developed by the AERPAW team in order to provide an Operator a way of monitoring and controlling an experiment. Although the OEO system has several processes, the main way of interaction between an Operator and the system is through the OEO Console.
Each instance of an OEO Console has three windows:
The events window (at the top), where the system reports significant events in the experiment; for example if a vehicle is trying to exit the geofence, an error will be displayed in this window. In the figure shown, the events window shows that vehicles on nodes 2 and 4 are disarmed.
The monitoring window (in the middle) allows the Operator to continuously monitor the status of several variables. In the figure, the node ID is show, together with the armed state of the vehicle, their current mode, altitude, and heartbeat.
The command window (at the bottom) allows the Operator to send commands to a particular node or to all nodes.
Starting the OEO Console
To start the OEO Console, first the Operator needs to gain access to the OEO Console VM. Then run this command:
cd; ./startOEOConsole.sh
This will start an OEO Console. The OEO system allows multiple simultaneous OEO Consoles to run in a single experiment: this can be done by the same Operator (by logging in repeatedly in the OEO Console container), or by different experimenters sharing an experiment. This allows one or multiple Operators to monitor different sets of variables, and to issue different commands at the same time.
To quit the OEO Console, type Ctrl+d.
Events Window
The events windows offers the operator feedback from the system for displaying info/errors while the experiment is running. In general, the syntax for information displayed on the console will follow the following format: [<sender>]-[<severity_level>] <message>.
<sender> is the system that sent the message. Most often, this will be the node_id of the node that sent the message. <severity_level> is an indication of the severity of the message. The system currently supports the following severity levels:
INFO - Informational messages that indicate normal operation or significant events.
WARNING - Messages that indicate potential issues or conditions that may require attention.
ERROR - Messages that indicate failures or errors that need attention.
CRITICAL - Messages that indicate critical failures or system-level issues.
A list of possible events that will be printed:
Invalid commands/parameters sent by the operator
MAVLink vehicle status messages
MAVLink filter aborting the experiment due to invalid vehicle commands
Vehicle Collision service aborting experiment due to forecasted collision
ADS-B flight notifications (only shown while in Testbed). For more information, see ADS-B Microservice.
Monitoring Window
The monitoring window offers the Operator the ability to monitor a large number of parameters relevant to the current experiment. The relevant commands for adding, renaming, and deleting a monitoring field (issued in the command window) are:
add <field> where <field> is the new field to be monitored.
For example add vehicle adds a field showing all the relevant values for vehicles (only for nodes having a vehicle). add computer does the same for the computer parameters.
More specific fields can be added by using / as a field separator. For example add vehicle/position/x or add /vehicle/battery/voltage.
The agent root data fields available are:
vehicle - for vehicle related data, and
computer - for node health information
remap <field> <new_name> is a simply a way to rename the field names to fit the preferences of the operator. For example remap vehicle/battery/voltage volts.
del <field> removes the specified field from the monitoring window.
save_config <config_name> saves the current configuration of the monitoring window (including the remapped fields) in a json file. If config_name is not provided, uses a default configuration loaded at startup.
load_config <config_name> loads a previously saved configuration for the monitoring window. If config_name is not provided, uses a default configuration loaded at startup.
Vehicle Parameters
Following is a list of vehicle parameters that can be monitored. All parameters can be simultaneously seen if add vehicle is entered in the command window.
status - 0 if vehicle is disarmed, and 1 if the vehicle is armed. Once armed, if there is no follow-up instruction, the vehicle automatically disarms in 10 seconds.
type - quadcopter for UAV, and rover for UGV
mode -
GUIDED while under the control of the vehicle software (unless in landing phase)
LAND when in landing phase
battery -
voltage - main battery voltage (in volts). A 3S LiPo battery (11V - 12.5V) is used for development, a 6S is used for SAMs, and a 12S is used for LAMs.
current - current drawn by the vehicle (in amperes).
level - state of charge: 0% is empty- vehicle will fall out of the sky in testbed, but will continue to fly in development mode. 100% is brand new battery. The simple way to change batteries in development mode is to restart the cvm processes (restart_cvsm in the command window).
position -
latitude - absolute coordinates
longitude - absolute coordinates
altitude - in [m]
velocity - the x, y, and z components of the ground velocity in m/s.
speed - the ground speed in m/s
gps
satellites - number of visible satellites
fix - the fix type:
0 - no GPS
1 - there is GPS but no fix is available
2 - 2D position
3 - 3D position
4 - 3D position with DGPS/SBAS corrections
5 - RTK float, 3D position
6 - RTK fixed, 3D position - this is the default in development mode
home_location - the starting position of the vehicle
latitude - absolute coordinates
longitude - absolute coordinates
geofence - the geofence the vehicle is using: ("phase 1", "phase 2")
Computer Parameters
Computer parameters provide the operator with information about the computer running the corresponding node. The following parameters are currently available:
heartbeat - a 10Hz counter indicating that the node is alive and well
heartbeat_quality - a metric (0-100%) that measures the quality of the link between the computer and the vehicle. For more information, see Link Quality Microservice
cpu_utilization - overall CPU utilization in the computer running that node. In development mode it is likely that all nodes share the same server, which will then have the same (or similar) CPU utilization at all nodes.
memory_utilization - shows the percentage of memory utilization in that node.
screens_running - shows a list of the screen processes running in each E-VM
Command Window
The command window allows an Operator to issue commands to the nodes in the experiment. The format of the commands are:
{node_id | all} agent_command [arguments]
Send agent_command to a particular node_id node or to all nodes (see supported agent commands below for details).
Example: all arm - arms all nodes that support arming
Example: 1 takeoff 20 - sends takeoff command to node 1, with a target altitude of 20m
Example: 2 start_location 35.734942 -78.703000 45.0 - changes start location of node 2 to 35.734942, -78.703000, with a heading of 45.0°.
Example: 2 start_location 35.729828 -78.698366- changes start location of node 2 to 35.729828, -78.698366. The heading will remain what it was previously.
Supported Agent Commands
For Vehicles:
arm - arms vehicle. If no further instructions are issued, the vehicle automatically disarms in 10s.
disarm - disarms vehicle
land - sets vehicle mode to LAND mode (descends until it lands, then disarms)
rtl - sets vehicle mode to RTL (return to launch). The vehicle climbs to RTL altitude, travels to the home position (where it was armed), then lands, and disarms.
takeoff <altitude> - makes vehicle take off to <altitude>. The vehicle needs to be armed and in GUIDED mode to takeoff
set_mode <mode> - sets vehicle mode to <mode> where <mode> is one of (GUIDED, MANUAL, ALT_HOLD, RTL, STABILIZE)
start_location { reset | <latitude> <longitude> [<azimuth>] } - sets vehicle home location to <latitude>, <longitude>, at heading <azimuth> (in degrees). <azimuth> is optional; by default, it will be set to 0.0. To reset the home location to default, run the command: start_location reset. The restart_cvm command must be run afterward in order for these changes to take effect. The new home location will persist through multiple restarts of the C-VM.
set_geofence <geofence_phase> - sets vehicle geofence to <geofence_phase> where <geofence_phase> is one of (1, 2). 1 corresponds to Phase 1. 2 corresponds to Phase 2. See Geofences for the different geofences for each phase. The restart_cvm command must be run afterward in order for these changes to take effect.
Vehicle Camera Tracking Commands (For more information, see Vehicle Camera Tracking)
track_camera { on | off | auto } - Tells the camera to track the vehicle node that was used in this command. The arguments tell the camera when to record:
on - The camera will immediately start recording (if it is not already)
off - The camera will stop recording
auto - The camera will record when the vehicle is armed
list_recordings - Lists all the recordings that have been recorded. Lists them with an index/recording_id attached to each (starting at 0).
download_recording <recording_id> - Downloads the recording corresponding to <recording_id> to the E-VM ~/Results folder. The <recording_id> is the index printed when using the list_recordings command
For Computer:
start_experiment - runs /root/startexperiment.sh on node’s E-VM
stop_experiment - kills all screen sessions on node’s E-VM
restart_cvm - restarts c-vm scripts in the corresponding node
transfer_logs - transfers MAVLink logs, MAVLink Filter logs, OEO logs, dataflash binaries from the C-VM to the E-VM /root/Results/c_vm_logs directory
download_logs - downloads last date's dataflash log files from the UAV or UGV's internal autopilot storage to the C-VM. Use transfer_logs to transfer this data in the E-VM.
check_params - runs the parameter checker to report on common vehicle parameters and ensure they are within acceptable ranges.