In this project, we tackle model selection by formulating the problem of hyperparameter optimization in the multi-model setting as a two-tiered system. The first tier is a multi-armed Gaussian Bandit problem that selects the model. The second tier is a Gaussian process-based Bayesian optimization technique that selects the optimal hyperparameters. We compare our model selection system to random search and demonstrated superior results. Our system extends on previous work in model selection, expanding model selection to include not only hyperparameter selection but also model family selection. We describe our system in detail in our paper linked below, and discuss future work for developing a more intelligent automated model selection system.

See more in our paper: https://github.com/kashizui/automated-statistician/blob/master/writeup/main.pdf