These are models and scripts for the paper:
Fully Convolutional Models for Semantic Segmentation
Jonathan Long*, Evan Shelhamer*, Trevor Darrell
CVPR 2015
arXiv:1411.4038
and its journal edition in PAMI (to appear).
Note that this is a work in progress and the final, reference version is coming soon. Please ask Caffe and FCN usage questions on the caffe-users mailing list.
These models are compatible with BVLC/caffe:master @ 74cc497 with the merge of PRs BVLC/caffe#3613 and BVLC/caffe#3570.
The code and models here are available under the same license as Caffe (BSD-2) and the Caffe-bundled models (that is, unrestricted use; see the BVLC model license.
PASCAL VOC models: trained online with high momentum for a ~5 point boost in mean intersection-over-union over the original models. These mdoels are trained using extra data from Hariharan et al., but excluding SBD val. FCN-32s is fine-tuned from the ILSVRC-trained VGG-16 model, and the finer striders are then fine-tuned in turn.
- FCN-32s PASCAL: single stream, 32 pixel prediction stride version, scoring 63.6 mIU on seg11valid
- FCN-16s PASCAL: two stream, 16 pixel prediction stride version, scoring 65.0 mIU on seg11valid
- FCN-8s PASCAL: three stream, 8 pixel prediction stride version, scoring 65.5 mIU on seg11valid and 67.2 mIU on seg12test
To reproduce the validation scores, use the seg11valid split defined by the paper in footnote 7. Since SBD train and PASCAL VOC 11 segval intersect, we only evaluate on the non-intersecting set for validation purposes.
The following models have not yet been ported to master and trained with the latest settings. Check back soon.
PASCAL VOC:
- FCN-AlexNet PASCAL: AlexNet (CaffeNet) single stream, 32 pixel prediction stride version
SIFT Flow model (also finetuned from VGG-16):
- FCN-16s SIFT Flow: two stream, 16 pixel prediction stride version
NYUDv2 models (also finetuned from VGG-16, and using HHA features from Gupta et al. https://github.com/s-gupta/rcnn-depth):
- FCN-32s NYUDv2: single stream, 32 pixel prediction stride version
- FCN-16s NYUDv2: two stream, 16 pixel prediction stride version
PASCAL-Context models including architecture definition, solver configuration, and barebones solving script (finetuned from the ILSVRC-trained VGG-16 model):
- FCN-32s PASCAL-Context: single stream, 32 pixel prediction stride version
- FCN-16s PASCAL-Context: two stream, 16 pixel prediction stride version
- FCN-8s PASCAL-Context: three stream, 8 pixel prediction stride version