SMARTS (Scalable Multi-Agent RL Training School) is a simulaton platform for reinforcement learning and multi-agent research on autonomous driving. Its focus is on realistic and diverse interactions. It is part of the XingTian suite of RL platforms from Huawei Noah's Ark Lab.
import gym
from smarts.core.agent_interface import AgentInterface, AgentType
from smarts.core.agent import AgentSpec, AgentPolicy
class Policy(AgentPolicy):
def act(self, obs):
return "keep_lane"
agent_spec = AgentSpec(
interface=AgentInterface.from_type(AgentType.Laner, max_episode_steps=None),
policy_builder=Policy,
)
agent_specs = {
"Agent-007": agent_spec,
"Agent-008": agent_spec,
}
env = gym.make(
"smarts.env:hiway-v0",
scenarios=["scenarios/loop"],
agent_specs=agent_specs,
)
agents = {
agent_id: agent_spec.build_agent()
for agent_id, agent_spec in agent_specs.items()
}
observations = env.reset()
for _ in range(1000):
agent_actions = {
agent_id: agents[agent_id].act(agent_obs)
for agent_id, agent_obs in observations.items()
}
observations, _, _, _ = env.step(agent_actions)
# For Mac OS X users, make sure XQuartz is pre-installed as SUMO's dependency
# git clone ...
cd <project>
# Follow the instructions given by prompt for setting up the SUMO_HOME environment variable
./install_deps.sh
# verify sumo is >= 1.5.0
# if you have issues see ./doc/SUMO_TROUBLESHOOTING.md
sumo
# setup virtual environment; presently only Python 3.7.x is officially supported
python3.7 -m venv .venv
# enter virtual environment to install all dependencies
source .venv/bin/activate
# upgrade pip, a recent version of pip is needed for the version of tensorflow we depend on
pip install --upgrade pip
# install [train] version of python package with the rllib dependencies
pip install -e .[train]
# make sure you can run tests (and verify they are passing)
make test
# then you can run a scenario, see following section for more details
We use supervisord to run SMARTS together with it's supporting processes. To run the default example simply build a scenario and start supervisord:
# build scenarios/loop
scl scenario build --clean scenarios/loop
# start supervisord
supervisord
With supervisord
running, visit http://localhost:8081/ in your browser to view your experiment.
See ./envision/README.md for more information on Envision, our front-end visualization tool.
Several example scripts are provided under SMARTS/examples
, as well as a handful of scenarios under SMARTS/scenarios
. You can create your own scenarios using the Scenario Studio. Here's how you can use one of the example scripts with a scenario.
# Update the command=... in ./supervisord.conf
#
# [program:smarts]
# command=python examples/single_agent.py scenarios/loop
# ...
SMARTS provides a command-line tool to interact with scenario studio and Envision.
Usage
scl COMMAND SUBCOMMAND [OPTIONS] [ARGS]...
Commands:
- envision
- scenario
- zoo
Subcommands of scenario:
- build-all: Generate all scenarios under the given directories
- build: Generate a single scenario
- clean: Clean generated artifacts
Subcommands of envision:
- start: start envision server
Subcommands of zoo:
- zoo: Build an agent, used for submitting to the agent-zoo
# Start envision, serve scenario assets out of ./scenarios
scl envision start --scenarios ./scenarios
# Build all scenario under given directories
scl scenario build-all ./scenarios ./eval_scenarios
# Rebuild a single scenario, replacing any existing generated assets
scl scenario build --clean scenarios/loop
# Clean generated scenario artifacts
scl scenario clean scenarios/loop
See the provided ready-to-go scripts under the examples/ directory.
Please read Contributing
Assuming you have run pip install .[dev]
.
make docs
python -m http.server -d docs/_build/html
# Open http://localhost:8000 in your browser
If you want to easily visualize observations you can use our Visdom integration. Start the visdom server before running your scenario,
visdom
# Open the printed URL in your browser
And in your experiment, start your environment with visdom=True
env = gym.make(
"smarts.env:hiway-v0",
scenarios=["scenarios/loop"],
agent_specs=agent_specs,
visdom=True,
)
If you're comfortable using docker or are on a platform without suitable support to easily run SMARTS (e.g. an older version of Ubuntu) you can run the following,
docker build -t smarts .
docker run --rm -it -v $(PWD):/src -p 8081:8081 smarts
# <press enter>
cd /src
pip install -e .[train]
# Run Envision server in the background
$ scl envision start -s ./scenarios -p 8081 &
$ python examples/single_agent.py scenarios/loop
# On your most machine visit http://localhost:8081 to see the running simulation in
# Envision.
- If you are having issues see: SETUP and SUMO TROUBLESHOOTING
- If you wish to find binaries: SUMO Download Page
- If you wish to compile from source see: SUMO Build Instructions
- If you build from the git repository use: SUMO version 1.7.0 or higher