- Python 2.7 (Better on Linux)
- numpy
- theano
- scipy for L-BFGS and CG optimization
http://deepy.readthedocs.org/en/latest/
# MNIST Multi-layer model with dropout.
from deepy.dataset import MnistDataset, MiniBatches
from deepy.networks import NeuralClassifier
from deepy.layers import Dense, Softmax, Dropout
from deepy.trainers import MomentumTrainer, LearningRateAnnealer
model = NeuralClassifier(input_dim=28*28)
model.stack(Dense(256, 'relu'),
Dropout(0.2),
Dense(256, 'relu'),
Dropout(0.2),
Dense(10, 'linear'),
Softmax())
trainer = MomentumTrainer(model)
annealer = LearningRateAnnealer(trainer)
mnist = MiniBatches(MnistDataset(), batch_size=20)
trainer.run(mnist, controllers=[annealer])- CPU
source bin/cpu_env.sh
- GPU
source bin/gpu_env.sh
- Simple MLP
python experiments/mnist/mlp.py
- MLP with dropout
python experiments/mnist/mlp_dropout.py
- MLP with PReLU and dropout
python experiments/mnist/mlp_prelu_dropout.py
- Deep convolution
python experiments/mnist/deep_convolution.py
- Elastic distortion
python experiments/mnist/mlp_elastic_distortion.py
- Recurrent visual attention model
python experiments/attention_models/baseline.py
- Baseline RNNLM (Full-output layer)
python experiments/lm/baseline_rnnlm.py
- Class-based RNNLM
python experiments/lm/class_based_rnnlm.py
- LSTM based LM (Full-output layer)
python experiments/lm/lstm_rnnlm.py
- Char-based LM with LSTM
python experiments/lm/char_lstm.py
- Char-based LM with Deep RNN
python experiments/lm/char_rnn.py
- Start server
pip install Flask-SocketIO
python experiments/deep_qlearning/server.py
- Open this address in browser
http://localhost:5003
- Recurrent NN based auto-encoder
python experiments/auto_encoders/rnn_auto_encoder.py
- Recursive auto-encoder
python experiments/auto_encoders/recursive_auto_encoder.py
- CG
python experiments/scipy_training/mnist_cg.py
- L-BFGS
python experiments/scipy_training/mnist_lbfgs.py
# Train the model
python experiments/draw/mnist_training.py
# Create animation
python experiments/draw/animation.py experiments/draw/mnist1.gz
python experiments/highway_networks/mnist_baseline.py
python experiments/highway_networks/mnist_highway.py
python experiments/initialization_schemes/gaussian.py
python experiments/initialization_schemes/uniform.py
python experiments/initialization_schemes/xavier_glorot.py
python experiments/initialization_schemes/kaiming_he.py
- Auto gradient correction
Sorry for that deepy is not well documented currently, but the framework is designed in the spirit of simplicity and readability. This will be improved if someone requires.
Raphael Shu, 2015