Student Theses

Efficient Planning for Safe Air Traffic Control With STL Constraints Using Deep Reinforcement Learning

Student: Yunbo Huang (Master's)

Affiliation: Eindhoven University of Technology, Eindhoven, The Netherlands

Abstract: Air traffic control (ATC) is an important problem in aerial traffic management that can be formulated as a planning problem with predefined signal temporal logic (STL) specifications. This problem can be further converted to a nonconvex mixed integer programming (MIP) problem which is computationally expensive due to the large amount of variables and constraints. Nevertheless, the safety requirements and the time limitations of practical applications require solving feasible solutions efficiently. In this report, we promote the efficiency of solving MIP by leveraging a large neighborhood search (LNS) algorithm which aims at a suboptimal solution with smaller computational complexity by iteratively selecting subsets of variables and constraints to compose suboptimal problems. This subset is automatically chosen by a graph convolutional neural network (GCN) trained using a reinforcement learning (RL) algorithm with the non-labeled data extracted from an off-the-shelf solver. We have evaluated the proposed RL-based LNS method on a dataset generated using a basic taxiing ATC scenario. Results have shown that our approach provides faster solutions than an open-source heuristic solver SCIP with a reduction of computation time at 68.6%.

VernaCopter: Natural Language-based Drone Control using Large Language Models and Formal Specifications

Student: T.A. Van de Laar (Master's)

Affiliation: Eindhoven University of Technology, Eindhoven, The Netherlands

Abstract: The ability to control robots was traditionally chiefly attributed to experts. However, the recent emergence of Large Language Models (LLMs) enables users to command robots using LLMs’ exceptional natural language processing abilities. Previous studies applied LLMs to convert tasks in natural language into robot controllers using a set of predefined high-level operations. However, this approach does not guarantee safety or optimality. This thesis introduces VernaCopter, a system that empowers non-technical users to control quadrocopters using natural language. Signal Temporal Logic (STL) functions as an intermediate representation of tasks specified in natural language. The LLM is responsible for task planning, whereas formal methods handle motion planning, addressing the abovementioned limitations. Automatic LLM-based syntax and semantics checkers are employed to improve the quality of STL specifications. The system’s performance was tested in experiments in varying scenarios, varying user involvement, and with and without automatic checkers. The experiments showed that including the user in conversation improves performance. Furthermore, the specific LLM used plays a significant role in the performance, while the checkers do not benefit the system due to frequent miscorrections。

Modular Risk Assessment: A Compositional Verification Method for Approximated Automaton

Student: Ryo Sugimura (Master's)

Affiliation: Eindhoven University of Technology, Eindhoven, The Netherlands

Abstract: Model checking is a formal verification method to assess whether a system satisfies specifications. It can be viewed as a form of risk assessment where specifications represent safety requirements. Model checking identifies violations in these safety specifications. A controller that minimizes risks can be synthesized based on model checking results. However, the formal verification method suffers from a state space explosion problem, where state space grows exponentially with the number of state variables. This work proposes a modular solution that 1) approximates an automaton of system behavior, which is 2) decomposed to create multiple subproblems, of which 3) the state variables are decoupled. The decomposition and decoupling address the state space explosion in model checking. The solution is applied to an overtaking scenario, demonstrating a reduction in computation time by a factor of 534 and memory usage by a factor of 155 compared to the formal verification method.

Risk-Intention-Aware Autonomous Driving Based on Signal Temporal Logic Specifications

Student: Qiyang Zong (Master's)

Affiliation: Technical University of Munich, Munich, Germany

Abstract: The next generation of autonomous driving should possess the capability to predict risks in realtime, recognize the intentions of other vehicles, and respond accordingly within a short time frame. This paper applys signal temporal logic(STL) specifications to introduce the scene into control system and to solve a path planning problem in a known risk level in autonomous driving. Using a typical lane merge scenario as a case, the paper addresses the challenge of path planning for autonomous vehicles navigating in the presence of multiple target vehicles. The research approach comprises the following key steps:First, a neural network model, Trajectron++ is employed to conduct multi-modal prediction on the trajectories of target vehicles, generating a set of trajectory points.In the second step, we employ a Gaussian Mixture Model (GMM) to precisely fit the generated trajectory points. By utilizing the derived GMM, we are able to identify specific probability density intervals, thereby establishing corresponding risk levels. Based on this setup, we then obtain boundary ellipses, which are used as constraint conditions to optimize the route planning of autonomous vehicles. Finally, path planning is accomplished utilizing the open-source library stlpy, which resolves the problem based on STL specifications. This solution is subsequently transformed into a Model Predictive Control (MPC) problem to achieve global waypoints for safe and efficient autonomous driving.

Human-Robot Skill Transfer with Enhanced Compliance via Dynamic Movement Primitives

Student: Jayden Hong (Master's)

Affiliation: University of British Columbia, British Columbia (BC), Canada

Abstract: Improving robots' performance relies on efficiently planning their trajectories. One way to do this is by using the Learning from Demonstrations (LfD) method to transfer human-like skills to robots. However, human motion isn't naturally suited for robots due to differences between human biomechanics and robot dynamics. To address this issue with LfD, the Dynamic Movement Primitives (DMP) framework offers a promising solution. However, it requires tuning of second-order dynamic system parameters. Our contribution is a systematic method that extracts dynamic features from human demonstrations to auto-tune the parameters in the DMP framework. This method is also valuable for use in conjunction with Reinforcement Learning (RL) in robot training. Through our approach, these extracted features enable robots to explore potential trajectories more effectively, significantly enhancing robot compliance. We achieve this by optimizing similarity in the parameter space while maintaining human-likeness. Our method was implemented on an actual human-robot setup. We extracted dynamic human features and generated robot trajectories through LfD and RL with DMP. This experiment resulted in a stable robot performance while preserving a high degree of human likeness, as good as the best heuristic tuning.

Constrained Optimal Coverage Control of a Multi-Unicycle System

Student: Nhan Khanh Le (Bachelor's)

Affiliation: Technical University of Munich, Munich, Germany

Abstract: This thesis presents a reliable control method for a group of Wheeled Mobile Robots (WMRs) to cover a specific region. There are many useful applications of coverage control such as surveillance, exploration, and rescue operation. The motivation of the proposed control method is to ensure the performance of coverage control while being able to handle a given constraint set. There are three constraints considered in this work. The state constraint requires all agents to maintain a specific range of coverage regions to ensure communication ability. The input constraint is the bounded rotation velocity of a WMR due to the limits of its hardware components. The third constraint is the constant heading velocity, which makes the design of the controller challenging due to the underactuation. Input constraints are ubiquitous and inevitable in practice. For this reason, this thesis is motivated to find a possible control law that is able to handle many constraints during the operation. In this project, we use the nonlinear anti-windup technique and the theorem of the barrier Lyapunov function to deal with the above-mentioned constraints. To test the feasibility and evaluate the performance of the control strategy, we create a platform and simulate the coverage control. The proposed controller is proven and tested under different scenarios, such as the number of agents or regions with varying complexity. The proposed controller is distributed, feasible, and applicable.

Model-free Global Navigation of under-actuated mobile robots

Student: Yichen Hu (Master's)

Affiliation: Technical University of Munich, Munich, Germany

Abstract: Autonomous navigation is one of the most life-relevant applications of artificial intelligence. In this thesis, localization-based navigation is achieved under the actuator level without a model. We formulated the problems as a deterministic Markov decision process and extracted the policy using reinforcement learning methods. The policies are trained in both the map with and without constraints cases. A heuristic reward function was designed for the navigation tasks. Furthermore, the inner relation between the simulation results and the trained policies was discussed. Besides simulation, a real-world navigation experiment was also conducted on the TAS car in a basement-like map. The results show that with our formulation and solution, the trained policy is able to accomplish the preset navigation tasks successfully.

Optimal filtering and control of information-spreading processes

Student: Cheng Zhang (Master's)

Affiliation: Technical University of Munich, Munich, Germany

Abstract: This thesis presents two control strategies for the information-spreading process with stochastic noise on linear observation. Firstly, the information spreading on social networks is modeled as epidemics spreading and our later research is based on the epidemical node-based SIS model. Next, optimal filtering for an incompletely measured information-spreading process is investigated to obtain the corresponding optimal estimates and error variance. Then, the node-based SIS model is verified by importing experimental data. After that, based on system stability analysis, an intuitive feedback controller is designed to solve the closed-form control problem. Meanwhile, by using the separation principle, an optimal controller is also implemented to solve the optimal control problem. The final simulation part of this thesis examines the effectiveness of both controllers. The simulation results also show the performance and accuracy of both control strategies, respectively.

Supplementary Techniques for Range Finders on Indoor Navigation of Mobile Robots

Student: Yamen Mohisn (Bachelor's)

Abstract: Simultaneous Localization and Mapping (SLAM) is essential in the field of mobile robots. Mobile robots should be able to construct maps, localize themselves, and navigate in different conditions and environments. The focus of this paper is on improving the localization of a car-like robot in indoor environments. Laser scanners are one of the most used range finders for SLAM, but they fail to localize the mobile robot in some environments and conditions. This paper seeks to solve the localization problem of the laser scanner. The approach is to correct the wrong light detection and ranging odometry (Lidar odometry) by fusing data from optical wheel encoders and an inertial measurement unit (IMU) with the Lidar odometry. The sensor fusion algorithm is based on the Kalman filter. The paper examines in the second part the possibility of replacing the laser scanner with an intel T265 tracking camera due to the high cost of the laser scanner and its wrong odometry in some cases. The results demonstrate that the sensor fusion corrects the wrong lidar odometry and the T265 camera can be used instead of the laser scanner for localization and navigation purposes. The achieved results can be generalized as a solution for localization problems of car-like robots.

Human motor control based on sliding mode control extended framework

Student: Marwan Shalaby (Bachelor's)

Affiliation: Technical University of Munich, Munich, Germany

Abstract: Robots are expected to freely reside within common human environments and to be physically more interactive with their surroundings. A key factor for their successful co-existence with humans lies in having a natural acceptable human-like motion. Simulating human motor control is one way of achieving such naturality. However, it is considered a challenging task due to the complex biological noise structure, together with the need for experimental validation. In this thesis, we introduce a sliding mode control extended framework; a stochastic optimal sliding mode controller with an adaptive Kalman filter and a non-linear sliding surface, and utilize it to model and understand human motor control behaviour. We conducted a user experiment where subjects performed both free and disturbed reaching motions. The results show that the introduced framework is able to reproduce similar kinematic profiles of human motion as well as react to disturbances the same way humans do.