Workshop on Quality and Reliability Assessment of Robotic Software Architectures and Components

The Key Role of System-Level Non-Functional Properties in Robotics Software

Non-Functional properties (NFPs) play a key role in most software-intensive systems. There is a lot of literature on what NFPs are but, sadly, there is no consensus on how to deal with them. In the context of robotics, NFPs such as safety, dependability or resource consumption must not only to be taken into account, but must be guaranteed to the greatest possible extent for robots to be viable, in particular, when they are intended to interact with humans or to perform critical missions. In the context of the RobMoSys H2020 Project, NFPs are treated as first-class citizens and their modeling, estimation and management is considered a key facilitator to achieve better robotics software. RobMoSys has funded two Integrated Technical Projects (ITPs) focused on NFPs: RoQME and MIRoN. The former has contributed a model-based framework enabling the modeling and estimation of Quality-of-Service (QoS) metrics defined on system-level NFPs, while the latter aimed at enabling the modeling and implementation of adaptive robot behaviors, based on how the previous QoS metrics evolve over time and trying to balance the (usually contradicting) requirements imposed on the different NFPs. The latest results obtained in both RoQME and MIRoN will be presented in the talk.

Verifiable Software Architectures for Autonomous Robotic Systems

Autonomous robotic systems are increasingly deployed in a wide range of safety- and mission-critical applications and environments, including aviation, manufacturing, healthcare and the automotive industry. As a result, it is essential to be able to verify these systems to ensure that they are safe and reliable for deployment. Verification is often applied to robotic systems after they have been developed. However, it is often preferable to design the robotic system with verification in mind to allow us to maximize the utility of the various verification techniques available. In this talk we examine some of these verifiable autonomous robotic system architectures, show how they are being used at the University of Liverpool for space and offshore applications on the FAIR-SPACE and ORCA hubs, and present recommendations for ensuring that these architectures can be verified to be safe and reliable.

Incorporating Real-World Semantics into Program Analysis of Robot Systems

Robotic software is plagued both by faults that menace all software (null-pointers, index-out-of-bounds) and also faults specific to its physical interaction with the real world, such as dimensional inconsistencies. These software hazards occur when developers incorrectly manipulate real-world quantities with physical units, such as confusing force with torque or measuring an angle in degrees instead of radians---something we have shown frequently happens in practice. We also found that existing solutions to these problems are time-consuming and error-prone. To address the state of the art, we designed a program analysis technique and its corresponding tool 'Phys' to automatically detect dimensional inconsistencies in robotic software with minimal developer burden. Phys uses probabilistic reasoning and dataflow analysis to infer what variables mean in the real world. Phys works on systems that use the popular `Robot Operating System' (ROS). I will present an evaluation showing that Phys has an 85% True Positive rate. I will present evidence that dimensional inconsistencies lurk in at least 6% ( 211 / 3,484 ) of open-source robotic software repositories. I will further show the results of an empirical study showing that developers correctly identify the physical units of variables only 51% of the time, motivating our future work on automatically suggesting physical unit types. Finally, I will present a vision of future robotic software research enabled by our techniques that aims to help developers build robots with more reliable robotic software.

Diagrammatic physical robot models in RoboSim

Simulation is a favoured technique for analysis of robotic systems. Lack of standardisation and portability between simulators, however, has impact on usability and cost of simulations. We present RoboSim, a diagrammatic tool-independent domain-specific language to model robotic platforms and their controllers. It can be regarded as a profile of UML/SysML enriched with time primitives, differential equations, and a formal process algebraic semantics. In RoboSim, a robotic platform is specified by a block diagram, which can be linked to a data model to characterise how events, variables, and operations of the software controller map to inputs and outputs of sensors and actuators. The behaviours of inputs, outputs, and joints are specified by systems of differential algebraic equations. Simulations and mathematical models for proof can be generated automatically from RoboSim models.

ROS Bugs: what do we learn about robotic systems engineering from studying and finding issues in rosdistro repositories?

What kinds of bugs do programmers encounter when developing ROS applications? Are those bugs unique to ROS? What can we do to avoid them? To answer these questions, we have been conducting a joint industry-academia effort over a two-year period to systematically identify and forensically analyse 221 bugs across 9 ROS packages from diverse projects. We created ROBUST, a publicly available set of ROS bugs developed in collaboration of several companies and universities engaged with robotics engineering. The dataset consists of carefully described and classified historical issues, along with a scripting machinery that allows to reconstruct a historical setup in which each bug was reported, in order facilitate reproduction and analysis research.

I will analyze the data set of historical issues collected from ROS repositories and ask what kind of bugs are faced by ROS developers. We will classify the bugs and ask ourselves why do they appear, which can be found with automatic analysis and testing tools, and which are inherently difficult due to interactions with hardware and the environment. Finally, we will attempt to derive lessons and best practices for robot developers helping to avoid the identified issues.

Software Quality as a Key Requirement for Safe and Secure Robot Applications

The development of intelligent robotic systems, both at present and in the future, will require greatly strengthened capabilities across sensing, reasoning, information management, and acting. The innovations required in these fields will primarily rely on the development of enhanced software components, software connectivity and software architectures. This need is further emphasized by the increasing adoption of modular, open-source, and open-data contributions both within the research community and industry. Many significant research contributions deal with the analysis of correctness, robustness, or reliability of algorithms and conceptional formulations of robot capabilities. Relatively few deal with design and analysis concerning the quality and reliability of the software that supports the execution of these capabilities.

Safety-the protection of humans from physical harm- has always been a significant key element in robotics. Security has become an urgent requirement for robots in recent years. In particular, the protection of subsystems implementing software functions for safe operation is crucial. As robotic technology progresses and is more and more pervaded by general-purpose computing technologies, a third "S" besides safety and security arises: Software(quality). More and more functions of robots are implemented using modern software techniques bringing along also the drawbacks of complex software systems. On the other hand, this also allows for the usage of software verification techniques to strengthen safety and security functions along with the robots business functionality.

In this talk, we survey the landscape in robot safety and security along with the most important standards and guidelines. Approaches that have the goal to bridge the gap between practical software engineering, program verification, and safety evaluation applicable in the robotics will be brought to the fore. We then highlight where quality of software is crucial to ensure safe and secure robots and discuss research activities that point out where modern software engineering techniques could be used in future to improve robot safety and security.