Progress Report

Plan [9-29-2018]:

1. ✅Set up Unity3D and ML-agent toolkits.
2. ✅Create project and Build the simulated world in Unity3D.
3. ✅Separated the maps with 4 streets and make it looks like a 3*3 grid.
4. ✅Set up the roads and traffic lights.
5. ✅Make rules for every traffic light.
6. ✅Create Vehicles and the routes they follow.
7. ✅Set up self-driving car and its agent.
8. ✅Test and debug.
9. ✅Train the self driving car.

Progress:

[09-10-2018] Enroll EE-800-L. Review relative or previous papers and projects.

[09-16-2018] Determined the project topic.

[09-16-2018] Build up project website roughly.

[09-18-2018] Read relative papers and projects online.

[09-20-2018] Setup Unity3D, Python 3.6.5 and TensorFLow environment for machine learning agents.

[09-24-2018] Update website.

[09-25-2018] Choose training environment (games).

[09-26-2018] Set goals or purposes for the first time training.

[09-27-2018] Make flow chats for machine learning agents.

[09-28-2018] Redetermined the purpose and topic of this project. Delete the environment which was built for previous plan.

[09-29-2018] Make a new plan for new topic.

[09-30-2018] Reset Unity3D and ML-agent.

[10-01-2018] Updated the website for weekly report.

[10-04-2018] Build up a simple test example for ML agents.

[10-05-2018] Set the agents to recognize color and choose the the way which has the same color.

[10-07-2018] Train the agents with 16 processes at the same time.

[10-08-2018] Update website.

[10-10-2018] Training the prototype 1.

Goal: Move to the goal which corresponds to the color of the block in the right way.

Agents: The environment contains one agent linked to a single Brain.

Agent Reward Function (independent):

+1 For moving to correct way.

-0.1 For moving to incorrect way.

Vector Observation space: 30 corresponding to local ray-casts detecting objects, goals, and walls.

Vector Action space: (Discrete) 1 Branch, 4 actions corresponding to agent rotation and forward/backward movement.

Mean Reward: 0.7

[10-13-2018] Finished the training of prototype1.

[10-14-2018] Build up the cross road unit for prototype2.

[10-15-2018] Model the self-driving car for prototype2.

[10-17-2018] Set the traffic path.

[10-19-2018] Remodel the truck and add some new vehicles.

[10-18-2018] Build up traffic light.

[10-18-2018] Debug the traffic lights.

[10-20-2018] Run and debug.

[10-25-2018] Add player vehicle for test.

[10-26-2018] Set keyboard control for player.

[10-30-2018] Start to write code for self-driving car.

[10-31-2018] Program crashed. Trying to figure out what happened.

[11-01-2018] Rebuild project twice and meet the same error.

[11-03-2018] Null propagation operator error is found.

[11-04-2018] MonoDevelop is using C# 4.0 that cannot support 'null propagation operator.' Update Unity3D and switch to Visual Studio to code and compile.

[11-08-2018] Fixed version error and some bugs.

[11-10-2018] Start to write brain and academy scripts for self-driving car.

[11-15-2018] Focus on coding the scripts of self-driving car.

[11-16-2018] Some bugs jumped out. The self-driving car is moving with an odd path. And the direction of the car is not changing while the car is turning around.