TensorFlow implementation of Self-critical Sequence Training for Image Captioning.
General framework for reinforcement learning image caption tasks.
This work is highly based on @yunjey 's implementation of show-attend-and-tell
Pytorch implementation self-critical.pytorch
- python2.7
- tensorflow 0.12
- numpy
- matplotlib
- scipy
- scikit-image
- hickle
- Pillow
- pycocoevalcap
- In this work, we used MSCOCO data set
- Data download & process is the same as @yunjey 's work. Please check show-attend-and-tell "Getting Started" secton for details.
To train the image captioning model, run command below.
$ python train.pyFor reference, the RL model is in ./core/model.py
After 5 epochs, our result has 5% improvement. Note that we are using different data set to test and we used different feature net.