Internship Projects
Title: Autonomous Driving Using Reinforcement Learning
Students: Lin-Chi Wu
Supervisors: M. Sc. Zengjie Zhang, Dr. Sofie Haersaert, and Dr. Zhiyong Sun
Date: 25.08.2023
Place: Eindhoven, Netherlands
Abstract: Reinforcement learning (RL) is an effective approach to motion planning in autonomous driving, where an optimal driving policy can be automatically learned using the interaction data with the environment. Nevertheless, the reward function for an RL agent, which is significant to its performance, is challenging to determine. The conventional work mainly focuses on rewarding safe driving states but does not incorporate the awareness of risky driving behaviors of the vehicles. In this paper, we investigate how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. Based on the essential requirements that prescribe the safety specifications for general autonomous driving in practice, we propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors. A simulation study in OpenAI Gym indicates the advantage of risk-aware reward shaping for various RL agents. Also, we point out that proximal policy optimization (PPO) is likely to be the best RL method that works with risk-aware reward shaping.
Title: Implementation of Real-time Control by ROS and ROS2
Students: Robert Lefringhausen and Qiaoyue Yang
Supervisors: M. Sc. Zengjie Zhang and Dr. Dirk Wollherr
Date: 14.07.2020
Place: Munich, Germany
Abstract: In this project, a real-time controller for an inverted pendulum and a model of an inverted pendulum are implemented in ROS 1 with the ROS control package and in ROS 2. Afterwards, the real-time control performance of both ROS versions is compared. Several experiments are run with both systems. The performance measurements of both controllers running on a real-time operating system and on a non-real-time operating system are compared. The influence of CPU stress on the real-time performance is also investigated. The comparison of the performance data shows that ROS 2 has a lower latency and is less influenced by system stress compared to ROS 1. For ROS 2 the difference between the real-time and the nonreal-time operating system is significant whereas there is only a small difference for ROS 1.
Title: Object Distance Measurement and Virtual View Synthesis Based on Stereo Vision Technique
Students: Haoming Song
Supervisors: M. Sc. Zengjie Zhang and Dr. Dirk Wollherr
Date: 20.09.2019
Place: Munich, Germany
Abstract: Stereo vision is used to produce two stereoscopic pictures of a given object, it consists of two cameras of parallel optical axes separated horizontally from each other by a small distance, and these two cameras are combined together in a single frame. The distance between the cameras and the object can be measured depending upon the distance between the positions of the objects in both pictures, the focal lengths of both cameras as well as the distance. By comparing information about a scene from two vantage points, depth information can be extracted by examining the relative positions of objects in the two panels. This is similar to the biological process of Stereopsis. Throughout this work, two webcams are used to find the object distance. After the calculation of the depth image, a synthesis-virtual view can be generated. Six tests were executed and the measured object distances were 35 cm, 40 cm, 50 cm, 60cm, 72.5 cm, and more than 100 cm. The measured distances were compared with those measured at the same time by a depth camera and there were good agreement between them with percentage error ranging from 0:3315% to 3:576%.
Title: Robot Collision Detection by Recurrent Neural Network
Student: Zhong Chen,
Supervisors: M. Sc. Zengjie Zhang and Dr. Dirk Wollherr
Date: 19.02.2019
Place: Munich, Germany
Abstract: As the presence of robotic technologies in daily life grows rapidly, research on Human-Robot Interaction becomes of utmost importance. Physical Human-Robot Interaction is still considered as one of the major challenges in robotics, it is easily affected by mechanical vibration and noise in real-world scenarios. In this report, a Recurrent Neural Network(RNN) classification method is proposed to discriminate physical interaction and collision. The dataset has 3 classes and each contains thousands of samples from 7 robot joints. The experimental average accuracy rate of the recurrent neural network was 95.00%. Finally, the classification results of the proposed method are compared with other machine learning methods.
Title: Dual optimization for human-robot collaborative control
Student: Xinchen Du
Supervisors: M.Sc. Zengjie Zhang and Dr. Dirk Wollherr
Date: 20.11.2018
Place: Munich, Germany
Abstract: Human-robot collaboration is considered an important trend in research and in industry. Dual optimal control for the system was previously conducted in Cartesian space, where optimization is simplified as a linear problem by building an impedance system model. To get rid of requirements of impedance control (such as constraints of sampling width), optimization in joint space is investigated. With the dynamic model in joint space, the control problem is transformed into solving a nonlinear HJB equation, which can be numerically solved by techniques like policy iteration or dynamic programming. In the collaboration, the leadership between human and robots is assigned by setting their weight matrices in the cost function.
Title: Robot Collision Detection and Classification Based on Time-Frequency Analysis Transformations
Student: Wenqian Xu
Supervisors: M.Sc. Zengjie Zhang and Dr. Dirk Wollherr
Date: 22.10.2018
Place: Munich, Germany
Abstract: A common method to analyze the robot collision is using a Neural Network (NN) and Support Vector Machine (SVM). In this research, aiming to get a higher accuracy in predicting the robots' reactions to different environmental situations, the reaction signals will be transformed into time-frequency maps and classified with the help of a Convolutional Neural Network (CNN). The research proves that the prediction rate is improved compared with the traditional method.
Title: Robot Collision Detection and Classification Based on Support Vector Machine
Student: Hang Xu
Supervisors: M. Sc. Zengjie Zhang and Dr. Dirk Wollherr
Date: 05.04.2018
Place: Munich, Germany
Abstract: Physical human-robot interaction is still considered one of the major challenges in robotics. It is easily affected by mechanical vibration and noise in the test process, which makes the test waveform more complicated. In order to discriminate interaction and collision, a set of time domain and frequency domain features and wavelet features will be generated and then selected by 2 tests and a recursive feature elimination method. Meanwhile, a classification system based on Support Vector Machine(SVM) will be implemented.
Title: Robot Collision Detection by Machine Learning Method
Students: Pengfei Xia, Yuan Yuan, and Xiaohui Hu
Supervisors: M. Sc. Zengjie Zhang and Dr. Dirk Wollherr
Date: 11.01.2018
Place: Munich, Germany
Abstract: Nowadays more and more robots are used to service humans and it is of great importance to take physical Human Robot Interaction (pHRI) into consideration. In previous studies, interaction and collision were hard to distinguish. In this Experiment, interaction and collision can be discriminated by three different machine learning methods and Support Vector Machine(SVM) has the best training effect.