Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture (arxiv)
Katsunori Ohnishi*, Shohei Yamamoto*, Yoshitaka Ushiku, Tatsuya Harada.
In AAAI, 2018
* indicates equal contribution.
Python 2.7
In addition, please pip install the following packages:
matplotlibpillowscipy
You can test FTGAN using pretrained model as follows:
cd src/joint_learning
python demo.py --gpu=0
If you failed to run because of model loading, please install git-lfs and run git lfs clone git@github.com:mil-tokyo/FTGAN.git, or please download models from Google Drive.
-
Download the dataset(Penn Action)and extract optical flow.
-
Resize all frames (76*76) and convert to npy file.
-
Setup dataset directory as follows.
PennAction/
npy_76/
0001.npy
0002.npy
...
2326.npy
npy_flow_76/
0001.npy
0002.npy
...
2326.npy
cd src/FlowGAN
python train.py --gpu=0 --root '/path/to/dataset/'
cd src/TextureGAN
python train.py --gpu=0 --root '/path/to/dataset/'
cd src/joint_learning
python train.py --gpu=0 --root '/path/to/dataset/'
| TextureGAN (from GT Flow and |
FTGAN (from |
|---|---|
 |
 |
If you find FTGAN useful in your research, please consider citing:
@inproceedings{ohnishi2018ftgan,
title={Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture},
author={Ohnishi, Katsunori and Yamamoto, Shohei and Ushiku, Yoshitaka and Harada, Tatsuya},
booktitle={AAAI},
year={2018}
}