This repository is the reproduction of paper "Uncertainty Estimation Using a Single Deep Deterministic Neural Network" by Joost van Amersfoort, Lewis Smith, Yee Whye Teh, Yarin Gal.

All codes for training and experiments are provided. They are as colab notebooks. Additionally, py files are provided for training and testing. Training and models are based on codes provided by the author

All requirements will be self-installed

To install requirements:

pip install -r requirements.txt

All datasets will be downloaded from torchvision except for notMNIST
Download it from "http://yaroslavvb.com/upload/notMNIST/notMNIST_small.mat" and place it in "data" folder:

mkdir -p data && cd data && curl -O "http://yaroslavvb.com/upload/notMNIST/notMNIST_small.mat"

• Open a notebook
• Place the utils folder (only requirement)
• Place the trained models, specify the location of trained model in code (for experiments)
• Run the notebook

• Create a virtual env
• Download the requirements
• Change hyper parameters and Run the python files

Training: will be done on FMnist or CIFAR-10
Testing: Accuracy will be calculated on testset of FMnist or CIFAR-10
Testing: Auroc-ood will be calculated on FMnist+Mnist-ood or CIFAR10+SVHN-ood

Trained models are required to execute the experiment codes
Experiment codes are self explanatory (includes uncertainty Histograms, rejection plots, ROC curve, aleatoric plots, uncertainty maps)

You can download pretrained models here:

• DUQ and DE trained on FashionMNIST, CIFAR10 with paper's default parameters

Performance of the model with our implementation at a glance:

@misc{vanamersfoort2020uncertainty,
      title={Uncertainty Estimation Using a Single Deep Deterministic Neural Network}, 
      author={Joost van Amersfoort and Lewis Smith and Yee Whye Teh and Yarin Gal},
      year={2020},
      eprint={2003.02037},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}