Flight software consists of software primarily running on the hardware for the C&DH subsystem. It can also encompass smaller portions of software on other subsystems on the spacecraft, especially portions of software intended for inter-subsystem communication.
Phase B - Review of Design and Preliminary Testing
▪Drivers Testing
▪Test Can driver
▪Test SPI driver
▪Test Flash driver
▪Test RTC driver
▪Test Watchdog timer driver
▪Test MRAM driver
Phase C
▪Inter-subsystem Testing
▪Telemetry data points
▪Ground Commands
▪testing of inter-subsystem communication (using prototype hardware)
▪Test/Verify telemetry data is sent between subsystems properly
▪Test/Verify ground commands
▪Preliminary testing of Concepts of operations on CDH
Phase D
▪Spacecraft Level Verification
▪Test/Verify commissioning activities
▪Set up a Flat-Sat with all subsystem prototypes
▪Simulate power-loss conditions on spacecraft
In-Application Programming (IAP) is a feature that allows Ground Station to upload a new application with updated Flight Software. If implemented, IAP could be the most complex feature of the Flight Software. Compared to other processors, IAP for Cortex-M3 devices is fairly well documented. For the Smartfusion2, it is well documented as well. At minimum, the following steps would be needed:
1. During the upload from Ground, the new application image would need to be stored in external Flash.
2. Verification of the image must be conducted.
3. The image would need to be programmed to the SoC. This could be done by a command from Ground, and will most likely require a bootloader.
If there is a bug in this process, this could cause the flight computer to brick, making it impossible to communicate with the spacecraft. At this moment, this feature is being explored, but is not planned as guaranteed feature for the spacecraft.
The high-level design of the software is largely driven by requirements of the Operations, Payload, and CDH subsystems. Due to the design of the software being tightly coupled with the design of other subsystems (i.e. due to hardware and operations), it should be kept in mind that implementation of certain flight software features may need to be tuned / changed based on finalized designs for other subsystems.
The flight software must be capable of doing the following:
Handle communication with Ground Station via radio.
Handle communication from the on-board device to other devices via CAN, SPI, etc.
Collect telemetry data from the payload and ADCS modules.
Monitor the health of the spacecraft, and manage relevant spacecraft modes as specified by Operations Design.
Process commands from other subsystems (mainly ground station).
Command and configure other subsystems.
Transmit on-board telemetry back to Ground Station.
This block diagram represents the software architecture for the Flight Software on C&DH. Software should be modular, as this makes it easier to design and test individual modules. Layering the software properly also makes the software easier to reuse or change. Arrows from one software module to another means “this module uses this module”.
Software Block Diagram
The HAL (hardware access level) contains modules for talking to the hardware (i.e. CAN drivers, SPI drivers, GPIO drivers), and abstracts the low-level details of the hardware from the higher levels. Unique to the hardware.
The CAN and SPI “drivers” are already provided for our processor (which is expected). The CAN and SPI “modules” will abstract the low-level details and make the drivers easier to use.
The Platform Framework contains the rest of the modules that are needed for writing the application / operations. T
The Application layer contains the subsystems applications. The PCU, ADCS, etc blocks are modules that are used to do the operations needed for these subsystems and to command or handle messages from those boards. Unique to the spacecraft.
Each software module consists of a source file (*.c) and a header file (*.h), which contains the code related to one specific function of the software.
The application layer of the MBSAT flight software handles operations unique to the MBSAT mission, such as collecting and processing payload data, handling ADCS data/control, communicating with the ground station etc.
The application layer functions are divided into two groups. Tasks that take a long time to execute or that will be run consistently, will be programmed as FreeRTOS tasks that run concurrently and are enabled/disabled by the time-tagged task runner. Functions that execute quickly and tasks that are scheduled manually will be programmed as functions run directly from the time-tagged task runner.
Tasks:
ADCS Control Loop
Payload data handling/processing
Thermal control loop
Software Update
The Main Task
Functions:
Request/get telemetry
Changing settings for CDH or subsystems
Change operations mode
Send telemetry
The main task handles tasks like command processing, operations state management, and more. More details will be added here as the software is developed. The goal is that most things will be controlled from here and this will be the “top-level” task.
A file system is included in the MBSAT flight software to provide organization for the flash memory. This will be used for storing copies of data for redundancy and holding long term data and telemetry/event logging. The file system used is littleFS. This file system is actively developed by the ARM Mbed organization, and includes features for power loss resilience, wear leveling and static memory allocation. The file system uses around 30kb of program memory and has configurable options to change the RAM usage and performance.
In order to use littleFS in our software, a driver must be written to provide 3 main functions. The driver must connect the flash memory drivers so that the file system can access the memory. The driver must integrate FreeRTOS with the file system so that multiple tasks can use the file system, and the driver should simplify dealing with redundant data.
A flowchart for the file system driver is shown. For all of the littleFS functions, the driver will provide a wrapper function that uses FreeRTOS mutexes to make the file system thread safe. The driver will also check if a file has or should have a backup copy and will perform the function on both the original and backup, verifying data if needed.
The time tagged task scheduler is used for scheduling tasks that must be executed at a future time. The module receives tasks along with a scheduled execution time and stores them in a priority queue. This queue uses time as the priority, therefore keeping the tasks sorted by earliest execution time. The scheduler then periodically checks the queue and will execute a task at the specified time. Valid tasks will be predefined in the software and will be short segments of code. Tasks that take long to execute will run as FreeRTOS tasks and will only be started/stopped or sent messages from the time tagged task scheduler.
During typical operation, a scheduler will be used to manage the tasks for the on-board computer, with 3 priorities of periodic tasks. Tasks should run only while tasks of higher priority are not running.
0. Interrupts - Interrupts are not periodic tasks, but they will execute at "higher priority" than all other tasks.
Messages from radio/CAN should be placed into a queue via an interrupt mechanism.
1. High Priority Tasks - Tasks that should always occur when possible.
Process messages received via radio.
Transmit messages via radio.
Operations control (i.e. management of spacecraft states).
2. Medium Priority Tasks - Tasks that should occur as often as possible, but have less strict timing requirements.
Process messages received via CAN.
Transmit messages via CAN.
Service watchdog timer.
3. Low Priority Tasks - These tasks are not essential to the health and operation of the satellite, and therefore are placed lower priority than other tasks.
Take payload readings and measurements, process them, and then store them in non-volatile memory (for later transmission).
Store diagnostic data to non-volatile memory (for later transmission). Note that diagnostic data will be logged throughout the lifetime of the software.
At some point, when entering typical operations, data should be written to non-volatile memory, indicating that if a power reset occurs, to resume normal execution.
The event logging system is the method by which any data is saved and stored until it is needed for transmission to ground or other subsystems. The event logging system is mainly an organizational structure and guidelines; the actual code provided by the module is meant to provide convenient access for other software modules and increased readability.
Data logged by the event logging system is stored in flash memory. Data is organized in files that are kept in folders.
Every day a new file is created in each folder, which will hold all data related to the system collected during that day. This is done to limit the size of files, which improves file system performance, and for easier recall of data later, since it is already sorted by day. A recurring time-tagged task will create new files once each day.
Within each file, each data item is stored with a timestamp, a priority flag, a unique ID and a deletion flag:
The timestamp is 8 bytes and can be provided by the logging system or the user software. –
The priority flag is used for flagging important events or data. The priority levels are “info”,” warning” and “error”. Data or events with the “error” priority will be sent to ground station on the next telemetry transmission. Data or events with the “warning” priority will be scheduled with the next convenient telemetry transmission.
The unique ID is used to identify the data and categorize the data to its respective folder.
The deletion flag is used to mark data that has been sent to ground and can be deleted.
The event logging system will provide the following functions:
LogData: Stores data in the appropriate file
filterData: Returns a file name(s) and position(s) based on filter items such as time, ID, priority etc.
getData: Returns a data item with the meta data removed
Iris team will not be designing a custom bootloader and will be utilizing Microsemi's bootloader design. The bootloader performs the function of loading and executable images to run or debug. A high-level bootloader functional flow diagram is shown below.
The bootloader selects and boots the default application image stored in the 256KB embedded flash memory(eNVM), which is at location 0, the addresses that the MCU is running from. Then, the bootloader initializes the Cortex-M3 processor stack pointer and the reset handler. Using the 64KB eSRAM part of the MSS, stack RAM content is initialized to initialize error detection and correction (EDAC). The initial value of variables is copied from the flash to RAM and memory is set to 0 for any uninitialized variables. Heap RAM content is initialized to initialize EDAC after which the drivers for CAN, SPI, RTC, Watchdog, UART are set up and tested. Finally, tasks are created to start the Free Real-Time Operating System (RTOS).
The software update module manages over-the-air(OTA) updates of the Iris flight software and FPGA configuration. The Smartfusion2 SOC has built-in programming and programming recovery systems, so the software update system’s main purpose is to handle receiving new software and initiating the update process.
NOTE: in the statements below, updating the golden image means uploading a fresh copy of the golden image. The golden image would not change but since it is stored on a non-radiation hardened flash memory it is likely that the image could be corrupted from radiation. An alternative to this would be to store it in the eNVM part of the MSS which is radiation hardened, however, this would require a custom bootloader code and update process, since the built-in code and In-Application Programming (IAP) process are not designed to accommodate this. While this process would be more robust, it would be complicated and require more development and testing time.
Add a step to verify integrity backup versions of both the nominal and golden images as well and not just the golden image, as the current process indicates.
In order to use the programming and recovery service, the firmware must be stored in memory connected to SPI0. This memory will hold 2 firmware versions, an updated version and the golden image, at fixed locations along with a directory. If there is a power interruption during the programming operation, the golden image will be programmed when the system is powered up, allowing the system to recover. A copy of both firmware versions will be kept in the main flash memory as well for redundancy. Before any update, the golden image should be verified and swapped with the backup if needed.
The software update system will run as a FreeRTOS task and will be enabled or disabled by the main task. The software update system has 3 modes; receiving update, verifying update and programing update. The results of each operation will be logged, and results transmitted to the ground station. Critical operations such as updating the software and replacing the golden image, will require a 2nd command in order to be carried out.
If ROM (Flash) corruption is detected, the flight software is expected to:
Log the event.
If it cannot be corrected, discard the corrupted data.
If RAM corruption is detected, the flight software is expected to:
Log the event.
If it cannot be corrected, write any unsaved data to non-volatile memory.
Once unsaved data is saved to non-volatile memory, trigger a watchdog failure, thereby resetting the device.
Note: If RAM corruption is not "detected", 3 will presumably occur, as a lock-up will cause the software to stop functioning.
If power loss occurs (e.g. due to watchdog failure), the flight software is expected to:
Once powered back on, return to regular operation.
Log the event to non-volatile memory.
Check the health of other subsystems on the spacecraft.
This is an external library used as the link, network and maybe transport layer for CAN bus communication. More info about how we are using CSP can be found in the “Command Structure” document.
For the implementation, a CSP server (FreeRTOS task) will be set up which will handle incoming CAN bus messages. The decoded messages are the passed to The Main Task, which will actually handle the message.
The overall schedule for flight software is driven by software milestones. On each proposed release, the flight software is intended to gain more and more features. The desired features are based on the high level software block diagram.
Memory corruption: can be a significant hazard to a long running space mission.
•Hardware-based mitigation:
•Implemented on C&DH for preserving data integrity
•Software-based mitigation:
•Checksums/CRCs for all data stored in non-volatile memory.
•Checksums for communication streams (both on the spacecraft and to ground station).
•Be aware of and minimize RAM usage (e.g. tables, call stack).
•When memory corruption is detected, Flight Software will log the event and try to correct it. If unsuccessful, will discard the corrupt data. If RAM corruption occurs, a watchdog reset should occur.
•Flight software will be designed such that it is capable of resuming normal operation on a power reset
Software Bugs:
•Even the most simple software have dependencies that carry risk (e.g. OS, hardware, libraries), which can lead to bugs in the software.
•A coding standard will be chosen and followed.
•The software will be kept modular. Hardware logic will be abstracted.
•A version control system will be used (i.e. git).
•Issue tracking will be done with Github.
•Software will be tested often, and unit tested where possible.
The source code and header files for flight software can be found in the GitHub repository: https://github.com/joehowarth17/ManitobaSat-Flight-Software
The repository contains software for the ManitobaSat Command and Data Handling (CDH) board. The directory structure is like so:
- /mbsat-fsw-libero contains the Libero project required for configuring the device.
- /mbsat-fsw-softconsole contains the SoftConsole project required to write and debug the software.
This project requires the Microsemi Smartfusion2 Maker Board.
This project requires SoftConsole 6.1 and Libero 12.1.
Open the *.prjx found in mbsat-fsw-libero, with Libero 12.1.
Connect the MSR board to the computer.
In the "Design Flow" panel on the left, double-click "Run PROGRAM Action".
Select the root folder (i.e. ManitobaSat-Flight-Software) as your SoftConsole workspace.
Go to File -> Import.
On the Import window, select General -> Existing C/C++ Projects into Workspace.
Add "mbsat-fsw-softconsole" to your workspace.
To build the SoftConsole project, go to Project -> Build All, or use Ctrl-B.
Go to Run -> Debug Configurations.
Double-click GDB OpenOCD Debugging to create a new Debugging configuration.
Under the "Main" tab, browse and select the "mbsat-fsw-softconsole" project as the project.
Under the "Debugger" tab, change the Config options to: "--command "set DEVICE M2S010" --file board/microsemi-cortex-m3.cfg"
Under the "Startup" tab, make sure that "Pre-run/Restart reset" is not checked.
With the MSR board connected, click "Apply", and then "Debug" to run the software.
In-Application Programming (IAP) is a feature that allows Ground Station to upload a new application with updated Flight Software. If implemented, IAP could be the most complex feature of the Flight Software. Compared to other processors, IAP for Cortex-M3 devices is fairly well documented. For the Smartfusion2, it is well documented as well. At minimum, the following steps would be needed:
1. During the upload from Ground, the new application image would need to be stored in external Flash.
2. Verification of the image must be conducted.
3. The image would need to be programmed to the SoC. This could be done by a command from Ground, and will most likely require a bootloader.
If there is a bug in this process, this could cause the flight computer to brick, making it impossible to communicate with the spacecraft. At this moment, this feature is being explored, but is not planned as guaranteed feature for the spacecraft.
https://github.com/joehowarth17/ManitobaSat-Flight-Software/tree/master/UnitTesting
- Template and example files are given in the ./tests directory. Refer to the CppUTest manual for information on writing your first unit tests: https://cpputest.github.io/manual.html
- Place source files (i.e. *.c, *.cpp, and *.h) files into the "./src" or "./tests" directories.
- Run "make" in this directory to build the test executable (./run_tests). Run "make clean" to clean.
- Ubuntu is needed to use the library files template project (e.g. use WSL with Ubuntu).
To rebuild the libraries for a different system, go to the CppUTest website, and download the latest source code: https://cpputest.github.io/
Make sure you have the following libraries:
sudo apt-get update (use to update all libraries to their latest revision)
sudo apt-get install make
sudo apt-get install gcc
sudo apt-get install g++
sudo apt-get install autoconf
sudo apt-get install libtools
sudo apt autoremove (delete any obsolete libraries)
Run the following commands in the uncompressed source code:
cd cpputest_build
autoreconf .. -i
cd ../configure
make
The libraries can be found in ../lib.
This section outlines the testing activities for the flight software. It includes test procedures for both Phase C and Phase D verification activities, a FlatSat test plan and list of required Ground Support Equipment (GSE). Since flight software requires CDH hardware, all FSW test plans have been designed to work with CDH test plans and avoid duplicate tests.
The FlatSat test plan for flight software is detailed below.