Algorithms used in the paper Learning Visual Servoing with Deep Features and Fitted Q-Iteration.
The goal is for a quadcopter/drone to follow a car around the city using only image observations. The following images are first-person views of the quadcopter successfully following the car. These are test executions of our policy based on VGG conv4_3 feature dynamics. The executions on the left use the cars seen during training and the ones on the right use novel cars.
The following executions are longer executions for each of the 5 novel cars.
Our policy was trained with the fitted Q-iteration algorithm that we propose using only 20 trajectories for reinforcement learning. To see executions of other methods, check out the paper's website.
git clone git://github.com/Theano/Theano.git
cd Theano
git apply patches/theano_matrix_inverse.patch
python setup.py develop --prefix=~/.local
git clone https://github.com/Lasagne/Lasagne.git
cd Lasagne
git apply patches/lasagne_dilation.patch
pip install -r requirements.txt
pip install --editable . --user
sudo apt-get install python-opencv
Follow the instructions from the CitySim3D site.
git clone git@github.com:alexlee-gk/visual_dynamics.git
cd visual_dynamics
pip install -r requirements.txt
Install desired version of python 3 (e.g. 3.5.2). Make sure to use the --enable-shared flag to generate python shared libraries, which will later be linked to.
env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.5.2
git clone git://github.com/Theano/Theano.git
cd Theano
pyenv local 3.5.2
python setup.py develop
git clone https://github.com/Lasagne/Lasagne.git
cd Lasagne
pyenv local 3.5.2
pip install -r requirements.txt
pip install --editable .
Make sure python-dev is installed for the python version being used, e.g.
sudo apt-get install python3.5-dev
git clone git@github.com:opencv/opencv.git
mkdir opencv_build
cd opencv_build
pyenv local 3.5.2
cmake \
-DWITH_CUDA=OFF \
-DCMAKE_BUILD_TYPE=RELEASE \
-DPYTHON3_EXECUTABLE=~/.pyenv/versions/3.5.2/bin/python3.5 \
-DPYTHON3_INCLUDE_DIR=~/.pyenv/versions/3.5.2/include/python3.5m \
-DPYTHON3_INCLUDE_DIR2=~/.pyenv/versions/3.5.2/include/python3.5m \
-DPYTHON_INCLUDE_DIRS=~/.pyenv/versions/3.5.2/include/ \
-DPYTHON3_LIBRARY=~/.pyenv/versions/3.5.2/lib/libpython3.so \
-DPYTHON3_NUMPY_INCLUDE_DIRS=~/.pyenv/versions/3.5.2/lib/python3.5/site-packages/numpy/core/include \
-DPYTHON3_PACKAGES_PATH=~/.pyenv/versions/3.5.2/lib/python3.5/site-packages \
-DINSTALL_PYTHON_EXAMPLES=ON \
-DINSTALL_C_EXAMPLES=OFF \
-DBUILD_EXAMPLES=ON \
-DBUILD_opencv_python3=ON \
../opencv
make -j4
sudo make install
ln -s /usr/local/lib/python3.5/site-packages/cv2.cpython-35m-x86_64-linux-gnu.so ~/.pyenv/versions/3.5.2/lib/python3.5/site-packages/cv2.so
For python 2, the cmake command is the following:
cmake \
-DWITH_CUDA=OFF \
-DCMAKE_BUILD_TYPE=RELEASE \
-DPYTHON2_EXECUTABLE=~/.pyenv/versions/2.7.12/bin/python2.7 \
-DPYTHON2_INCLUDE_DIR=~/.pyenv/versions/2.7.12/include/python2.7 \
-DPYTHON2_INCLUDE_DIR2=~/.pyenv/versions/2.7.12/include/python2.7 \
-DPYTHON_INCLUDE_DIRS=~/.pyenv/versions/2.7.12/include/ \
-DPYTHON2_LIBRARY=~/.pyenv/versions/2.7.12/lib/libpython2.7.so \
-DPYTHON2_NUMPY_INCLUDE_DIRS=~/.pyenv/versions/2.7.12/lib/python2.7/site-packages/numpy/core/include \
-DPYTHON2_PACKAGES_PATH=~/.pyenv/versions/2.7.12/lib/python2.7/site-packages \
-DINSTALL_PYTHON_EXAMPLES=ON \
-DINSTALL_C_EXAMPLES=OFF \
-DBUILD_EXAMPLES=ON \
-DBUILD_opencv_python2=ON \
../opencv
The library can be installed only for the local user by specifying a local install prefix, e.g. -DCMAKE_INSTALL_PREFIX=~/.local, in which case make install should be run without root priviledges and the last symbolic linking step might not needed.
- After running
cmake, the python2 OpenCV module appears next to 'Unavailable' instead of 'To be built'. Omit the flags that definePYTHON2_EXECUTABLEandPYTHON2_LIBRARYin thecmakecommand and then fix them withccmakeafterwards. - The file
Python.his not found even though it is in the specifiedPYTHON3_INCLUDE_DIR,fatal error: Python.h: No such file or directory. Explicitly expanding the home directory~to${HOME}might solve this. - Installation for python 2 causes the compilation error
error: invalid conversion from ‘const char*’ to ‘Py_ssize_t {aka long int}’. In this case, disable python 2 support with the option-DBUILD_opencv_python2=OFF. - Importing
cv2gives the errorImportError: dynamic module does not define module export function (PyInit_cv2)because it is using the wrongcv2library. Make sure the path for the newly builtcv2package appears first in thePYTHONPATH,export PYTHONPATH=~/.pyenv/versions/3.5.2/lib/python3.5/site-packages:$PYTHONPATH.
In Mac OS X, replace the cmake command with this one:
cmake \
-DWITH_CUDA=OFF \
-DCMAKE_BUILD_TYPE=RELEASE \
-DPYTHON3_EXECUTABLE=~/.pyenv/versions/3.5.2/bin/python3.5 \
-DPYTHON3_INCLUDE_DIR=~/.pyenv/versions/3.5.2/include/python3.5m \
-DPYTHON3_INCLUDE_DIR2=~/.pyenv/versions/3.5.2/include/python3.5m \
-DPYTHON3_LIBRARY=~/.pyenv/versions/3.5.2/lib/libpython3m.dylib \
-DPYTHON3_LIBRARY_DEBUG=~/.pyenv/versions/3.5.2/lib/libpython3m.dylib \
-DPYTHON3_NUMPY_INCLUDE_DIRS=~/.pyenv/versions/3.5.2/lib/python3.5/site-packages/numpy/core/include \
-DPYTHON3_PACKAGES_PATH=~/.pyenv/versions/3.5.2/lib/python3.5/site-packages \
../opencv
The option WITH_CUDA=OFF might be necessary if Caffe is used. See this issue for more information.
- https://gist.github.com/pohmelie/cf4eda5df24303325b16
- http://stackoverflow.com/questions/33250375/compiling-opencv3-with-pyenv-using-python-3-5-0-on-osx
mkdir -p data
python scripts/generate_data.py config/environment/simplequad.yaml config/policy/random_quad_back.yaml -n100 -t100 -o data/simplequad_train_data
python scripts/generate_data.py config/environment/simplequad.yaml config/policy/random_quad_back.yaml -n10 -t100 -o data/simplequad_val_data
python scripts/train.py config/predictor/multiscale_dilated_vgg_local_level1_scales012.yaml config/transformer/transformer_128.yaml config/solver/adam_gamma0.9_level1scales012.yaml config/data/simplequad.yaml
python scripts/learn_visual_servoing.py models/theano/multiscale_dilated_vgg_local_level1_scales012/transformer_128/adam_gamma0.9_level1scales012/simplequad/_iter_10000_model.yaml config/algorithm/fqi_nooptfitbias.yaml