This code comes from the author of the paper: "Graph R-CNN for Scene Graph Generation". It is just used here for my master thesis. The author has also already uploaded their official code finally, please check there own repostory(https://github.com/jwyang/graph-rcnn.pytorch)

This is the pytorch implementation of our graph rcnn model. It completes three tasks: object detection, attribute recognition and relation detection for each image jointly, and obtain the graph representations meanwhile. Besides a good performance on these three seperate tasks, it is also expected to help a bunch of high level (downstream) tasks, such as image captioning, visual question answering, expression reference, etc. It is developed based on the following two projects:

1. jwyang/faster-rcnn.pytorch, a pure, fast and memory efficient pytorch implementation of faster-rcnn.
2. peteanderson80/bottom-up-attention, a more sophisticated visual representation for image captioning and visual question answering based on bottom-up attention model.
3. danfeiX/scene-graph-TF-release, a secene graph detection model based on iterative message passsing.

Download Visual Genome. For this project, you will need:

1. Images (part1, part2)
2. Metadata
3. Object Annotations, Object Alias
4. Attribute Annotations
5. Relation Annotations, Relation Alias
6. Scene Graph

After downloading the above data, unzip all of them and put into a single folder. Then make a soft link to it via:

cd $REPO_ROOT
ln -s PATH/TO/YOUR/DATA_DIR data/vg

Then, prepare the pascal voc like xmls for visual genome using:

cd $REPO_ROOT
python data/genome/setup_vg.py

Create a folder:

cd $REPO_ROOT
mkdir data/pretrained_model

Download pretrained resnet101 and vgg16 models and put them into the folder.

To train a resnet101, run:

CUDA_VISIBLE_DEVICES=0 python trainval_grcnn.py --dataset vg1 --net res101

Alternatively, to train a vgg16, run:

CUDA_VISIBLE_DEVICES=0 python trainval_grcnn.py --dataset vg1 --net vgg16

CUDA_VISIBLE_DEVICES=0 python test_grcnn.py --dataset vg1 --net res101 --checksession 1  --checkepoch 11 --checkpoint 6224 --imdb test --mGPUs True