DMAS group A1:

• Daniel Bick (s3145697)
• Thijs Eker (s2576597)
• Ewout Bergsma (s3441423)
• Thomas Bakker (s2468360)

Original GitHub repository

Reference paper

Dear assessors, welcome to the GitHub page of our project. The file you are currently reading is meant to explain all that is required to train, visualize and obtain results of our project. All other README.md files are from the original repository and not our work, excluding the ones found in the checkpoints directory.

By using and adjusting the above mentioned orignial GitHub repository we constructed a multi agent traffic simulation, using reinforcement learning. The following files with code were created by us for creating the RL agents:

• examples/rllib/multiagent_exps/multiagent_bottleneck.py
• flow/envs/multiagent/bottleneck.py

The multiagent_bottleneck.py file is the main file to run, if one would want to train the model. The bottleneck.py file holds the environment classes that are used in the aforementioned main file, thus holding code for the state, action and reward.

The following files hold code that were created by us in order to obtain results:

• flow/visualize/visualizer_rllib.py <- Main file to visualize results
• flow/benchmarks/baselines/our_bottleneck.py <- Baseline variant of flow/benchmarks/our_bottleneck.py (file can be run)
• flow/benchmarks/our_bottleneck.py <- The main benchmark file
• flow/benchmarks/our_bottleneck_10_perc_av.py <- Same as our_bottleneck.py but with 10% AVs
• flow/benchmarks/rllib/our_ppo_runner.py <- The file to run the benchmark with PPO agents

Additionally, our report can also be found on this page: Report.pdf.

In order to train a model Google Colab can be used. Following these instructions:

• Download the Google_Colab.ipynb found on this page
• Go to the Google Colab website: https://colab.research.google.com
• Log in to a Google account (if not logged in already)
• Upload the file that was uploaded in step 1 (file > upload notebook > navigate to downloaded file and select)
• Follow the instructions in the file that has just opened in the browser

Before installing the project that there are two videos uploaded to Youtube, for visualization purposes. Additionally one can train the model using Google Colab. If one would still want to install the project locally, follow these steps:

• Git clone https://github.com/gitaar9/groupA1-coop-av-flow.git
• Navigate to the base directory of git repo
• Follow the instructions in the Install_locally.sh file (may want to open it in a text editor)

For visualization of the simulation there are two options. Firstly, we have uploaded two visualization videos to Youtube:

• Reward function r_velocity with 50% AV: https://youtu.be/O65Y8ObD3qI
• Reward function r_time with 10% AV: https://youtu.be/np_Wf7nnfUo

The second option is to render visualizations locally. This requires to install the project locally first. Afterwards follow these instructions:

• Navigate to the root of this project
• Activate conde env: conda activate flow
• Example command to visualize the 160th checkpoint of the r_velocity model:
  python3 flow/visualize/visualizer_rllib.py checkpoints/r_velocity/PPO_BottleneckThijsMultiAgentEnv-v0_0_2019-10-24_11-42-2561p4zgee 160
• In sumo click the green play button 3 times (we think this has to do with the fact we use 4 cores)
• When the play button is pressed the 4th time, the simulation will run
• With the step button next to the play button, you can step through the simulation to get a better view of what the cars are doing.

The red cars are the cars controlled by the trained model, white cars are controlled by a simple algorithm designed to mimick human behaviour (provided by the flow library).

Example of command in order to run the code for obtaining results, running from root of this project: python flow/benchmarks/rllib/our_ppo_runner.py --checkpoint CHECKPOINT_FILE --benchmark_name our_bottleneck --num_rollouts 8 --num_cpus 2
This would append the results for every completed simulation to a file called result. This result file can be summarized using python flow/benchmarks/rllib/summary.py result
Obtaining the baseline results is a lot easier and can be done by running python flow/benchmarks/baselines/our_bottleneck.py