Research Projects
Meta-Load Fly
Abstract:
The safe transport of cable-suspended loads by a group of Unmanned Aerial Vehicles (UAVs) relies heavily on their capacity to effectively respond to fluctuations caused by variations, such as wind gusts. However, traditional adaptive learning methods are challenging to adapt to changes in both the dynamics model and safety constraints. In this paper, we present Meta-Load Fly, a learning-based framework for adaptive load transportation in the presence of disturbances and formation changes within a team of UAVs. Meta-Load Fly consists of an adaptive load trajectory tracking module based on meta model-based reinforcement learning with online adaptation and correction, and an adaptive path planning module based on a collision predictor trained by a neural network. The simulation results show that, when dealing with aforementioned variations, the proposed Meta-load Fly achieves superior performance with less tracking error of the load and more robustness compared with state-of-the-art model-free and model-based and variational inference methods. Moreover, the adapted paths generated by Meta-Load Fly guarantee collision-free safety when navigating among obstacles.
Keyword: Model-based Reinforcement Learning, Meta Learning, Path Planning, Neural Representation Learning, Cooperative Transport
This work is currently under reviewed.
The code and presentation will be released later on.
Multi-agent Navigation to Grasping Points on a Load
Abstract:
Deep reinforcement learning, by taking advantage of neural networks, has made great strides in the continuous control of robots. However, in scenarios where multiple robots are required to collaborate with each other to accomplish a task, it is still challenging to build an efficient and scalable multi-agent control system due to increasing complexity. In this paper, we regard each unmanned aerial vehicle (UAV) with its manipulator as one agent, and leverage the power of multi-agent deep deterministic policy gradient (MADDPG) for the cooperative navigation and manipulation of a load. We propose solutions for addressing navigation to grasping point problem in targeted and flexible scenarios, and mainly focus on how to develop model-free policies for the UAVs without relying on a trajectory planner. To overcome the challenges of learning in scenarios with an increasing number of grasping points, we incorporate the demonstrations from an Optimal Reciprocal Collision Avoidance (ORCA) algorithm into our framework to guide the policy training and adapt two novel techniques into the architecture of MADDPG. Furthermore, curriculum learning with the attention mechanism is utilized by reusing knowledge from fewer grasping points to facilitate the training of a load with more points. Our experiments were validated by a load with three, four and six grasping points respectively in Coppeliasim simulator and then transferred into the real world with Crazyflie quadrotors. Our results show that the average tracking deviations from the desirable grasping point to the final position of the UAV can be less than 10 cm in some real-world experiments. Compared with state-of-the-art model-free reinforcement learning and swarm optimisation algorithms, results show that our proposed methods outperform other baselines with a reasonable success rate especially in the scenarios with more grasping points. Furthermore, the learned optimal policies enable UAVs to reach and hover over all the grasping points before manipulation without any collision. We conducted a comprehensive analysis of both targeted and flexible navigation, highlighting their respective advantages and disadvantages.
Keyword: Multi-agent Reinforcement Learning, Demonstration Learning, Curriculum Learning, Cooperative Navigation
Paper: A deep multi-agent reinforcement learning framework for autonomous aerial navigation to grasping points on loads
Citation:
@article{chen2023deep,
title={A deep multi-agent reinforcement learning framework for autonomous aerial navigation to grasping points on loads},
author={Chen, Jingyu and Ma, Ruidong and Oyekan, John},
journal={Robotics and Autonomous Systems},
volume={167},
pages={104489},
year={2023},
publisher={Elsevier}
}
Presentation: https://drive.google.com/file/d/1W4y5NYy9OUnf1OauCdDr6KfAIPNrzIG8/view?usp=sharing
Github: https://github.com/wawachen/MARL_transport
Swarm Optimisation for Stable Transport
Abstract:
Collaboration of agents in a natural swarm enables the accomplishment of tasks that would be difficult or impossible for a single agent to complete alone. For example, a swarm of autonomous Unmanned Aerial Vehicles (UAVs) enables the collaborative sensing of bulky loads for transportation over impassable terrains when the load weighs several times more than each UAV.
In this work, we propose a hierarchical algorithmic architecture that supports the search and coverage of various unknown payload profiles for subsequent transportation. The grasping formation of UAVs over the payload emerges from the synthetic behaviours in the architecture without any path planning. Experiments show that our proposed design can be successfully applied in searching and coverage of various loads and has been validated in the real world through the use of Crazyflie micro-UAVs. Furthermore, the proposed grasping formation satisfies static equilibrium thereby reducing orientation changes in the load-swarm system during transportation.
Keyword: Bio-inspired optimisation, Hierarchical Architecture, Swarm Robotics, Manipulation and Transport
Paper: Behavioural Swarm Optimisation for Stable Slung-load Aerial Transportation
Citation:
@inproceedings{ChenO23-3,
title = {Behavioural Swarm Optimisation for Stable Slung-Load Aerial Transportation},
author = {Jingyu Chen and John Oyekan},
year = {2023},
doi = {10.1109/CEC53210.2023.10254023},
url = {https://doi.org/10.1109/CEC53210.2023.10254023},
researchr = {https://researchr.org/publication/ChenO23-3},
cites = {0},
citedby = {0},
pages = {1-8},
booktitle = {IEEE Congress on Evolutionary Computation, CEC 2023, Chicago, IL, USA, July 1-5, 2023},
publisher = {IEEE},
isbn = {979-8-3503-1458-8},
}
Presentation: https://drive.google.com/file/d/16zZ5tV_T72e0MZ-eZyvJbsPxhpFN6bBg/view?usp=sharing
Github: https://github.com/wawachen/Swarm_optimisation_transport