Rhis project aims to train an agent in the TORCs simulator environment using RL algorithms. Starting with naive algorithm using only available state space variables, Deep Deterministic Policy Gradients (DDPG) is trained. We further implement Behavior Cloning (BC) and our variation of Generative Adversarial Imitation Learning (GAIL). In GAIL, we train a multi-modal based deep RL algorithm including visual inputs and causal information encoding to train the agent. We also present the utilization of the causality of the latent code in an expert trajectory consisting of different sub-tasks and leverage this in the model

The code is developed based on the paper https://arxiv.org/abs/1703.08840 The detailed explanation of TORCS for AI research . (https://arxiv.org/pdf/1304.1672.pdf)

Python 2.7
Tensorflow 0.12.1
PyTorch 0.4.1
Keras 1.2.2
xautomation

Install all the dependencies for TORCs ( http://torcs.sourceforge.net/ for more info)
git clone https://github.com/apbose/RLProject_AutomousNavigation_Torcs
For the two subtasks

pass -
cd pass_traj_train
Training -
Latent code embedded model - python train.py
Behavior Cloning model - python BC.py
DDPG Model - python ddpgMain.py
Evaluation - python evaluate.py

turn -
cd turn_traj_train
Training -
Latent code embedded model - python train.py
Behavior Cloning model - python BC.py
DDPG Model - python ddpgMain.py
Evaluation - python evaluate.py

For the causal model implementation cd pass_traj_train
Training - python train.py -causal 1
Evaluation - python evaluate.py