git clone https://github.com/liruiw/OMG-prerelease.git --recursive-
Setup: Ubuntu 16.04 or above, CUDA 10.0 or above
-
Install anaconda and create the virtual env for python 2 / 3
conda create --name omg python=3.6.9/2.7.15
conda activate omg
pip install -r requirements.txt
-
Install ycb_render
cd ycb_render python setup.py develop -
Install the submodule Sophus. Check if the submodule is correctly downloaded.
cd Sophus mkdir build cd build cmake .. make -j8 sudo make install
-
Install Eigen from the Github source code here
-
Compile the new layers under layers we introduce.
cd layers python setup.py install -
Install the submodule PyKDL
cd orocos_kinematics_dynamics
cd sip-4.19.3
python configure.py
make -j8; sudo make install
export ROS_PYTHON_VERSION=3
cd ../orocos_kdl
mkdir build; cd build;
cmake ..
make -j8; sudo make install
cd ../../python_orocos_kdl
mkdir build; cd build;
cmake .. -DPYTHON_VERSION=3.6.9 -DPYTHON_EXECUTABLE=~/anaconda2/envs/omg/bin/python3.6
make -j8; cp PyKDL.so ~/anaconda2/envs/omg/lib/python3.6/site-packages/- run
./download_data.shfor data (Around 600 MB). - Run the planner to grasp objects.
| python -m omg.core -v -f demo_scene_0 | python -m omg.core -v -f demo_scene_1 |
|---|---|
 |
 |
| python -m real_world.trial -s script.txt -v -f kitchen0 | python -m real_world.trial -s script2.txt -v -f kitchen1 |
|---|---|
 |
 |
- Loop through the 100 generated scenes and write videos.
python -m omg.core -exp -w
-
Install PyBullet
pip install pybullet gym(build with eglRender for faster rendering) -
Run planning in PyBullet simulator
| python -m bullet.panda_scene -v -f demo_scene_2 | python -m bullet.panda_scene -v -f demo_scene_3 |
|---|---|
 |
 |
| python -m bullet.panda_kitchen_scene -v -f kitchen0 | python -m bullet.panda_kitchen_scene -v -f kitchen1 -s script2.txt |
|---|---|
 |
 |
-
Loop through the 100 generated scenes and write videos.
python -m bullet.panda_scene -exp -w
-
Generate demonstration data data/demonstrations.
python -m bullet.gen_data -w
-
Visualize saved data.
python -m bullet.vis_data -o img
-
Generate related files for your own mesh. (.obj file in data/objects/)
python -m real_world.process_shape -a -f box_box000
-
Graspit can be used to generate grasps with this ros_package and the panda gripper. Then save the poses as numpy or json files to be used in OMG Planner. Alternatively one can use DexNet or direct physics simulation in Bullet.
├── ...
├── OMG
| |── data
| | |── grasps # grasps of the objects
| | |── objects # object meshes, sdf, urdf, etc
| | |── robots # robot meshes, urdf, etc
| | |── demonstrations # saved images of trajectory
| | └── scenes # table top planning scenes
| |── bullet
| | |── panda_scene # tabletop grasping environment
| | |── panda_kitchen_scene # pick-and-place environment for the cabinet scene
| | |── panda_gripper # bullet franka panda model with gripper
| | |── gen_data # generate and save trajectories
| | └── vis_data # visualize saved data
| |── layers # faster SDF queries with CUDA
| |── omg # core algorithm code
| | |── core # planning scene and object/env definitions
| | |── config # config for planning scenes and planner
| | |── planner # OMG planner in a high-level
| | |── cost # different cost functions and gradients
| | |── online_learner # goal selection mechanism
| | |── optimizer # chomp and chomp-project update
| | └── ...
| |── real_world # auto-encoder networks
| | |── trial # cabinet environment with an interface
| | |── process_shape # generate required file from obj
| | └── ...
| |── ycb_render # headless rendering code
| | |── robotPose # panda-specific robot kinematics
| | └── ...
| └── ...
└── ...
- The config parameters are not perfectly tuned and there can be some stochasty in the goals and plans.
- The grasp optimization code is not used since the grasps are mostly good (standoff pregrasp is used).
- Please use Github issue tracker to report bugs. For other questions please contact Lirui Wang.
If you find OMG-Planner useful in your research, please consider citing:
@inproceedings{wang2020manipulation,
title={Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection},
author={Wang, Lirui and Xiang, Yu and Fox, Dieter},
booktitle={Robotics: Science and Systems (RSS)},
year={2020}
}
The OMG Planner is licensed under the MIT License.