UAV Path Planning using Machine Learning

Project Approach

This project aims to develop a 3D path planning for Unmanned Aerial Vehicle using machine learning. We use deep RL technique so that the UAV can self-learn from its environment. Here we use 2 drones, one is delivery drone which is to deliver parcels and the other one is aerial base station drone which flies in a restricted area providing cellular connectivity and other instructions to delivery drones. By using RL, we plan to train the drones to reach the destination with minimum time, in an energy efficient way. Additionally, these drones will be trained to take a path which could utilize maximum cellular coverage and charging possibilities using HAPS(High Altitude Platform Station).

Our idea suggests an environment with different layers which is used by a deep reinforcement agent to calculate its rewards for each action and generate a model which could self propagate through these layers. The layers are defined below

By considering a delivery drone's environment as a 12x12 grid

Layer 1 – Self and the destination

This represents the source and the destination. represents the delivery drone which we are focused on and DES represents the destination where we need to reach.

Layer 1

Layer 2 – Obstacles

This layer represents obstacles like a tall building or a tree which UAV is not supposed to get closer to

Layer 2

Layer 3 – Monitoring drones/Arial base stations

Monitoring drones M or coverage drones, which distributes coverage to delivery drones and monitors their movement. These drones could also work as remote charging stations for delivery drones. The delivery drones are supposed to fly under the area of influence of each monitoring drones.

Layer 3

Layer 4 – Other delivery drones

These are other delivery drones similar to the one we are interested in. We are not supposed to get too closer to these drones to avoid collision. These drones also learn themselves from environment as well from the drone we are focused on (In layer 1).

Layer 4

Layer 5 – Cellular Coverage

We are assuming the coverage is provided only by the monitoring drones. Below diagram is a representation of the coverage area each monitoring drones distribute.

Layer 5

Combining all these layers and applying Reinforcement Learning formulas will yield reward values something like below. Where the greener cells are more favorable areas to move and the red cells are the ones to be avoided

To the above model, we define some adjacency matrix which follow below rules to avoid collisions between the drones:

The delivery drones cannot come less than 2 units closer to each other.
The monitoring drones cannot come less than 3 units closer to maximize the coverage area.
Delivery drone should not go father than 3 units from a monitoring drone
The monitoring drones cannot go farther than 6 units to avoid gaps in the coverage area.
The delivery drones can go as far as possible from each other until they are inside the coverage area.
Delivery drones which are diagonal to each other should not come less than as per Pythagoras right angle triangle theorem.

Combining individual rewards and the adjacency rewards gives us the final reward function

Relationship Representation

Page updated

Google Sites

Report abuse