Embedded Files
There are very limited number of computationally efficient and/or light simulators to develop and analyze autonomous driving behaviors. The lane changing and merging problem is the most common scenario studied by the RL community that marks as a starting step to discover more complex behaviors.
The 3D simulators although made efficient in the perspective of graphics rendering, it is quite difficult considering the neural networks, to compute based on the image data available. There are multiple 3D simulators like -
TORCs
Udacity Self driving Car
MACAD Gym
And more..
Out of these, the TORCs (Racing) and Udacity's self driving car simulator are single vehicle based car simulators. The Multi Agent Connected Autonomous Driving (MACAD) Gym is a 3D simulator which enables us with the power to develop multi agent behaviors. This environment has been developed with various scenarios whose combinations can be used to analyze the complex situations that most commonly realize the modern day driving scenes. The limitation to use this simulator is that it is a development in progress . Only a stable intersection scenario has been designed hence, in order to simulate our project, we choose 2D simulators.
Bark is a 2D simulator that helps us to simulate the lane changing/merging scenario.
"Bark is a 2D semantic simulation framework for autonomous agents with a special focus on autonomous driving. Its behavior model-centric design allows for the rapid development, training and benchmarking of various decision-making algorithms. Due to its fast, semantic runtime, it is especially suited for computationally expensive tasks, such as reinforcement learning "
But, it is computationally heavy and depends on the build of the system.
Highway-env
Highway-env
So we finally end up with highway-env. It is a computationally light and free to use minimalist environment developed for decision making for autonomous vehicles. The simulator uses PyGame to render the environment and is a better option to develop and analyze.
A screenshot from the environment.
Overview of the simulator:
Designed with OpenAI gym as a base
Compatible with RL algorithms library like Stable Baselines
Multiple scenarios visualized - Lane Changing, Merging, Intersection, Roundabout, Parking.
Expandable to Multi Agent
A detailed Documentation : highway-env.readthedocs.io/en/latest/
Installation:
You can either install through pip -
pip install --user git+https://github.com/eleurent/highway-env
Or clone the repository and install the requirements-
pip install requirements -e .
Also, if you are running from repository, then the dqn policy can be loaded with this repository - RL-Agents
Parameters for our implementation of Multi Agent lane changing:
Number of Controlled Vehicles: 2
Color of the controlled vehicles: Green (MDP agent)
Color of the IDM Vehicle: Blue
Initial Spacing between two vehicles: 2 units
Count of IDM Vehicles: 10
Policy frequency: 1
Simulation frequency: 15
Reward speed range: [20, 30]
Policy: DQN with MLP and Epsilon greedy
Repo: https://github.com/Prasheel24/highway-env/tree/multi-agent-rl/Code
In order to be compatible with multi agent behaviors, we have changed the action space, observation space, reward formulation and terminal conditions.
Page updated
Google Sites
Report abuse