Better Learning

Configure Capacity with Nodes and Layers

Configure Gradient Precision with Batch Size

Configure What to Optimize with Loss Functions

Configure Speed of Learning with Learning Rate

Stabilize Learning with Data Scaling

Fix Vanishing Gradients with ReLU

Fix Exploding Gradients with Gradient Clipping

Accelerate Learning with Batch Normalization

Deeper Models with Greedy Layer-Wise Pretraining

Jump-Start Training with Transfer Learning

Better Generalization

Penalize Large Weights with Weight Regularization

Sparse Representations with Activity Regularization

Force Small Weights with Weight Constraints

Decouple Layers with Dropout

Promote Robustness with Noise

Halt Training at the Right Time with Early Stopping

Better Predictions

Combine Models From Multiple Runs with Model Averaging Ensemble

Contribute Proportional to Trust with Weighted Average Ensemble

Fit Models on Different Samples with Resampling Ensembles

Models from Contiguous Epochs with Horizontal Voting Ensembles

Cyclic Learning Rate and Snapshot Ensembles

Learn to Combine Predictions with Stacked Generalization Ensemble

Combine Model Parameters with Average Model Weights Ensemble