This is the main code base accompanying the paper with the above title, under review in CoRL 2020.

To install the package and its dependenclies run

Inside the folder, Stoch2_gym_env:

    python -m pip install .

The code base was tested with gym (0.17.2), pybullet (2.8.2) with a python version of 3.6.9. However it is expected to work fine for any future versions of these packages, though they havent been tested.

As explained in the paper, we take a guided learning approach wherein the role of a initial policy is quite crucial.To train your own initial policy,run the following command

    python create_initial_policy.py --policyName filename

This saves the initial policy as filename.npy in the initial_policies folder.This file is to be loaded later as the initial polcy when you want train your own polciy.However there are a few initial policies present in the same folder which could be directly used to start the ARS training.

Note: The initial policies are by default saved in the initial_policies folder.

This is where the ARS trainining starts,with the initial polciy trainied in the previous step.

    python train_policy.py 

The above command starts the training in the default and the best by far hyperparameters and experiment settings. However, the following parameters can be customized in the training as desired by the user.

For example,

  python train_policy.py --lr 0.05 --noise 0.04 --logdir testDir --policy init_policy_TS.npy --msg "Training with some paramters" --episode_length 400

Note:

1. The initial policies are by default loaded from the initial_policies folder and the log directory is saved inside the experiments folder.

2. The are a few other insignificant parameters which need not be changed for the training, for more info about the parameters run

    python train_policy.py --help
   

To run a policy in default conditions, the following command is to be used.

    python test_policy.py

the following test parameters can be changed while testing the policy,

Thus, for a

1. custom test

    python test_policy.py --PolicyDir 23July3 --WedgeIncline 11 --WedgeOrientation 15 --FrontMass 0.1 --FrictionCoeff 0.6
   

2. random test

    python test_policy.py --PolicyDir 23July3 --RandomTest True --seed 100
   

To run a policy on a staircase of fixed dimensions, the following command is to be used.

    python test_policy.py --Stairs True

To run a policy on a arbitary slope track, the following command is to be used.

    python arbitary_slope_test.py

Note:

1. The test policies are by default loaded from the path experiments/given_logdir_name/iterations/best_policy.npy", if not specified it loads the best ever policy pre-trained by us.
2. For loading the policies from other directories, you might have to change the path from within the test_policy.py file.
3. In our method we only train for +ve roll and -ve pitch conditions of support plane, the trained policy is able to generalize for other conditions too.
4. Our env is not fully supported for training in downhill case, but you can evalute policy in downhill conditions.