sh install.sh
network.pyにあるCifar10Classifier_XXXをtrain.pyの下の方に突っ込んで以下のコマンドを実行する．
# Residual Network(32 layers)を訓練させたい場合
python train.py --class_name Cifar10Classifier_ResNet32
output に数値計算結果が出力され，modelsにモデルが生成されます．
| Name | Precision | Memo |
|---|---|---|
| Cifar10Classifier_01 | 83.11% | |
| Cifar10Classifier_02 | 87.00% | |
| Cifar10Classifier_03 | 87.25% | |
| Cifar10Classifier_04 | 87.67% | |
| Cifar10Classifier_05 | 87.17% | |
| Cifar10Classifier_06 | 86.74% | |
| Cifar10Classifier_ResNet20 | 91.07% | [2] |
| Cifar10Classifier_ResNet32 | 92.04% | [2] |
| Cifar10Classifier_ResNet44 | 91.93% | [2] |
| Cifar10Classifier_ResNet56 | 92.38% | [2] |
| Cifar10Classifier_ResNet110 | 92.94% | [2] |
| Name | Description |
|---|---|
| GPU | GeForce GTX TITAN X |
| OS | Ubuntu 16.04 LTS |
| Library | TensorFlow 0.8.0 |
1epochは訓練データ5万枚を一周学習させた回数
| Name | Test Error |
|---|---|
| Original Paper | 8.27% |
| Adadelta(LR 1e-3) | 31.03% |
| Adagrad(LR 1e-2) | 15.90% |
| RMSProp(LR 1e-3) | 10.97% |
注). LRはLearning Rateの意
| Name | Test Error |
|---|---|
| Original Paper | 8.27% |
| BN after addition | 8.89% |
| ReLU before addition | 9.54% |
| ReLU only pre activation | 8.82% |
| ful pre-activation | 10.03% |
| No ReLU | 8.85% |
- [1]. Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." arXiv preprint arXiv:1502.03167 (2015).
Batch Normの仕組みについて記載
- [2]. He, Kaiming, et al. "Deep Residual Learning for Image Recognition." arXiv preprint arXiv:1512.03385 (2015).
ImageNet 2015優勝アルゴリズム.
- [3]. He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE International Conference on Computer Vision. 2015.
ResNetなど多層のネットワークを構築する上で必要な重みの初期化方法が載っている．
- [4]. He, Kaiming, et al. "Identity mappings in deep residual networks." arXiv preprint arXiv:1603.05027 (2016).
Residual Networkの解析が行われている．
- [5]. Lin, Min, Qiang Chen, and Shuicheng Yan. "Network in network." arXiv preprint arXiv:1312.4400 (2013).
ResNet構築に必要なGlobal Average Poolingについて記載されている．
- [?]. Survey - Deep Residual Learning for Image Recognition, 2016/03/01
[2].のサーベイ
- [?]. Survey - Identity Mappings in Deep Residual Networks, 2016/03/30
[4].のサーベイ