This is a personal machine learning repository - may not be fit for public use (due to some hard codings, lot of trial and error etc)
Deep learning needs a lot of compute power, better using GPU. In order to reduce the AWS costs, we will try to use the p2.xlarge spot instances. Current spot pricing can be seen at https://aws.amazon.com/ec2/spot/pricing/
setup your aws credentials using
git clone https://github.com/achinta/kaggle.git
cd kaggle/setup/aws
python launch-spot.py --name mlThe IP address and instnace id would be displayed. Then ssh into the instance and setup the software and install the gpu. This can also be given as the startup script when we launch the instance.
#install GPU and setup software
cd ~
git clone https://github.com/fastai/courses.git
sudo bash courses/setup/install-gpu.sh
#install a tensorflow version which detects the gpu - https://anaconda.org/anaconda/tensorflow-gpu
sudo apt-get install libcupti-dev
conda install -c anaconda tensorflow-gpu
#check that the gpu is detected
#generate jupyter notebook configuration (http://jupyter-notebook.readthedocs.io/en/latest/public_server.html#notebook-server-security) and enable it for all ips
jupyter notebook --generate-config