Skip to content

kokookok77/show-attend-and-tell-pytorch

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention in PyTorch

This repository contains PyTorch implementation of Show, Attend and Tell

How to run

To train model form scratch, use following command.

python main.py

To train model following existing checkpoint, use following command.

python main.py --model_path MODEL_PATH

To generate caption of an image, use following command.

python main.py --test --model_path MODEL_PATH --image_path IMAGE_PATH

Lastly, to download required data (Flickr8k and GloVe, for now), use '--download' argument.

Results

Flickr8k dataset

Following examples are generated after training using Google Colaboratory for less than 7 hours. Training captions are lemmatized, and so are generated captions. Thus generated captions are not complete English sentences, but they are still interpretable. (Lemmatization helps training when resource is limited, because it reduces vocabulary size.)

(A lot of examples have dogs, because dogs are cute!)

  • Correct examples

correct1

correct2

correct3

correct4

correct5

correct6

correct7

  • Not 100% correct, but not totally wrong examples

not_correct1

not_correct2

not_correct3

  • Wrong examples

wrong

wrong2

About

PyTorch implementation of "Show, Attend and Tell: Neural Image Caption Generation with Visual Attention"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%