Reinforcement learning algorithms in RLlib and PyTorch.
Raylab provides agents and environments to be used with a normal RLlib/Tune setup.
import ray
from ray import tune
import raylab
def main():
raylab.register_all_agents()
raylab.register_all_environments()
ray.init()
tune.run(
"NAF",
local_dir=...,
stop={"timesteps_total": 100000},
config={
"env": "CartPoleSwingUp-v0",
"exploration_config": {
"type": tune.grid_search([
"raylab.utils.exploration.GaussianNoise",
"raylab.utils.exploration.ParameterNoise"
])
}
...
},
)
if __name__ == "__main__":
main()One can then visualize the results using raylab dashboard
pip install raylab| Paper | Agent Name |
| Actor Critic using Kronecker-factored Trust Region | ACKTR |
| Trust Region Policy Optimization | TRPO |
| Normalized Advantage Function | NAF |
| Stochastic Value Gradients | SVG(inf)/SVG(1)/SoftSVG |
| Soft Actor-Critic | SoftAC |
| Model-Based Policy Optimization | MBPO |
| Streamlined Off-Policy (DDPG) | SOP |
For a high-level description of the available utilities, run raylab --help
Usage: raylab [OPTIONS] COMMAND [ARGS]...
RayLab: Reinforcement learning algorithms in RLlib.
Options:
--help Show this message and exit.
Commands:
dashboard Launch the experiment dashboard to monitor training progress.
episodes Launch the episode dashboard to monitor state and action...
experiment Launch a Tune experiment from a config file.
find-best Find the best experiment checkpoint as measured by a metric.
rollout Wrap `rllib rollout` with customized options.
test-module Launch dashboard to test generative models from a checkpoint.The project is structured as follows :
raylab
├── agents # Trainer and Policy classes
├── cli # Command line utilities
├── envs # Gym environment registry and utilities
├── losses # RL loss functions
├── logger # Tune loggers
├── modules # PyTorch neural network modules for algorithms
├── policy # Extensions and customizations of RLlib's policy API
├── pytorch # PyTorch extensions
├── utils # miscellaneous utilitiesThis package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.