In this repository, I will show you the example code of the scratch implementation of the neural network. The theories about neural network are described in c231n (Stanford Univ.). The details of the backpropagation are described in Neural Networks and Deep Learning.
- numpy 1.14.5
- pandas 0.23.1
- matplotlib 2.2.2
- scikit-learn 0.19.1
- Download MNIST dataset (4 files) and store those files into the "data" directory.
- Required file names are following:
- train-images-idx3-ubyte.gz
- train-labels-idx1-ubyte.gz
- t10k-images-idx3-ubyte.gz
- t10k-labels-idx1-ubyte.gz
What I have created is the network for the digit recognition (MNIST Dataset). The network structure is shown in the following figure.
python train.py
To simplify the execution, hyper parameters of the network have already pre-defined in the code.
Note of the program:
- All the digits data including the test data are normalized by 255.
- Split original training dataset into train & validation data.
- training : validation = 0.8 : 0.2
- Keep training until the validation loss reaches to plateau.
- Save weights and biases as ".npz" format only in case the validation loss improved.
- Load best model and evaluate model using test dataset.