Fork of original repository modified for NIPS 2018.

Reason8.ai code for 3th place NIPS learning to run challenge.

We are porting this code to pytorch here

1. Get opensim package. You can use default package as described here or build by youself faster version here

2. Run setup script:

    $ ./setup.sh
   

You may want to change conda env name in script and comment last line if not building opnesim by yourself

3. If you experience theano errors try to move .theanorc file:

    $ mv ~/.theanorc ~/.theanorc.backup
   

Install dependencies:

    $ conda install numpy scipy scikit-learn mkl theano
    $ pip install https://github.com/Lasagne/Lasagne/archive/master.zip

1. Activate environment:

    $ source activate nips_rl_fast3
   

2. Run code with best flags:

    $ python run_experiment.py --param_noise_prob 0.3 --flip_prob 1 --layer_norm
   

   New model parameters:

    --accuracy 0.01 --modeldim 3D --prosthetic False --difficulty 0 --actor_layers \(64,64\) --critic_layers \(64,32\)
   

The final submitted model was trained in this commit.

There are lot of branches with various ideas tested during competition but without documentation, you could check for example following branches:

• distributed ddpg with pyro4 inspired by ctmarko repository
• distributed CEM with pyro4 I am not sure that this is canonical implementation, it was done in the last night.
• we even tried to do planning as described in this article