##Training Images: http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
##Training Labels: http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
# Loading Sample Images
#Loading 10K images from MNIST database
images = load_MNIST.load_MNIST_images('../../data/mnist/train-images-idx3-ubyte')
patches = images[:, 0:10000]
patches = patches[:,0:200]
#display_network.display_network(patches[:,0:200])
#sys.exit()
#sys.exit() Use this to stop execution at different parts of the assignment
#Now you will use the display network function to display different sets if the MNIST dataset
#The display is saved in the folder under the name weigths.
#Display 10, 50 and 100 datasets
# Obtain random parameters theta
theta = utils_hw.initialize(hidden_size, visible_size)
#print theta
#sys.exit()
##======================================================================
## STEP 2: Implement sparseAutoencoderCost
#
# You can implement all of the components in the cost function at once, but it may be easier to do
# it step-by-step and run gradient checking (see STEP 3) after each step. We
# suggest implementing the sparseAutoencoderCost function using the following steps:
#
# (a) Implement forward propagation in your neural network, and implement the
# squared error term of the cost function. Implement backpropagation to
# compute the derivatives. Then (using lambda=beta=0), run Gradient Checking
# to verify that the calculations corresponding to the squared error cost
# This loads our training data from the MNIST database files.
train_images = load_MNIST.load_MNIST_images('train-images.idx3-ubyte')
train_labels = load_MNIST.load_MNIST_labels('train-labels.idx1-ubyte')
train_images = train_images[:, 0:10]
train_labels = train_labels[0:10]
# ======================================================================
# STEP 2: Train the first sparse autoencoder
# This trains the first sparse autoencoder on the unlabelled STL training
# images.
# If you've correctly implemented sparseAutoencoderCost.m, you don't need
# to change anything here.
# Randomly initialize the parameters
sae1_theta = utils_hw.initialize(hidden_size_L1, input_size)
J = lambda x: utils_hw.sparse_autoencoder_cost(x, input_size, hidden_size_L1,
lambda_, train_images)
options_ = {'maxiter': 400, 'disp': True}
result = scipy.optimize.minimize(J, sae1_theta, method='L-BFGS-B', jac=True, options=options_)
sae1_opt_theta = result.x
print result
# ======================================================================
# STEP 3: Train the second sparse autoencoder
# This trains the second sparse autoencoder on the first autoencoder
# featurse.
# If you've correctly implemented sparseAutoencoderCost.m, you don't need
# Now you will use the display network function to display different sets if the MNIST dataset # The display is saved in the directory under the name weigths. # Display 10, 50 and 100 datasets ### YOUR CODE HERE ### # display 10 display_network.display_network(patches[:, 0:10], 'weights10.png') # display 50 display_network.display_network(patches[:, 0:50], 'weights50.png') # display 100 display_network.display_network(patches[:, 0:100], 'weights100.png') # Obtain random parameters theta # You need to implement utils_hw.initialize in utils_hw.py theta = utils_hw.initialize(hidden_size, visible_size) # ====================================================================== # STEP 2: Implement sparseAutoencoderCost # # You can implement all of the components in the cost function at once, but it may be easier to do # it step-by-step and run gradient checking (see STEP 3) after each step. We # suggest implementing the sparseAutoencoderCost function using the following steps: # # (a) Implement forward propagation in your neural network, and implement the # squared error term of the cost function. Implement backpropagation to # compute the derivatives. Then (using lambda=beta=0), run Gradient Checking # to verify that the calculations corresponding to the squared error cost # term are correct. # # (b) Add in the weight decay term (in both the cost function and the derivative
