##Training Images: http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz ##Training Labels: http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz # Loading Sample Images #Loading 10K images from MNIST database images = load_MNIST.load_MNIST_images('../../data/mnist/train-images-idx3-ubyte') patches = images[:, 0:10000] patches = patches[:,0:200] #display_network.display_network(patches[:,0:200]) #sys.exit() #sys.exit() Use this to stop execution at different parts of the assignment #Now you will use the display network function to display different sets if the MNIST dataset #The display is saved in the folder under the name weigths. #Display 10, 50 and 100 datasets # Obtain random parameters theta theta = utils_hw.initialize(hidden_size, visible_size) #print theta #sys.exit() ##====================================================================== ## STEP 2: Implement sparseAutoencoderCost # # You can implement all of the components in the cost function at once, but it may be easier to do # it step-by-step and run gradient checking (see STEP 3) after each step. We # suggest implementing the sparseAutoencoderCost function using the following steps: # # (a) Implement forward propagation in your neural network, and implement the # squared error term of the cost function. Implement backpropagation to # compute the derivatives. Then (using lambda=beta=0), run Gradient Checking # to verify that the calculations corresponding to the squared error cost
# This loads our training data from the MNIST database files. train_images = load_MNIST.load_MNIST_images('train-images.idx3-ubyte') train_labels = load_MNIST.load_MNIST_labels('train-labels.idx1-ubyte') train_images = train_images[:, 0:10] train_labels = train_labels[0:10] # ====================================================================== # STEP 2: Train the first sparse autoencoder # This trains the first sparse autoencoder on the unlabelled STL training # images. # If you've correctly implemented sparseAutoencoderCost.m, you don't need # to change anything here. # Randomly initialize the parameters sae1_theta = utils_hw.initialize(hidden_size_L1, input_size) J = lambda x: utils_hw.sparse_autoencoder_cost(x, input_size, hidden_size_L1, lambda_, train_images) options_ = {'maxiter': 400, 'disp': True} result = scipy.optimize.minimize(J, sae1_theta, method='L-BFGS-B', jac=True, options=options_) sae1_opt_theta = result.x print result # ====================================================================== # STEP 3: Train the second sparse autoencoder # This trains the second sparse autoencoder on the first autoencoder # featurse. # If you've correctly implemented sparseAutoencoderCost.m, you don't need
# Now you will use the display network function to display different sets if the MNIST dataset # The display is saved in the directory under the name weigths. # Display 10, 50 and 100 datasets ### YOUR CODE HERE ### # display 10 display_network.display_network(patches[:, 0:10], 'weights10.png') # display 50 display_network.display_network(patches[:, 0:50], 'weights50.png') # display 100 display_network.display_network(patches[:, 0:100], 'weights100.png') # Obtain random parameters theta # You need to implement utils_hw.initialize in utils_hw.py theta = utils_hw.initialize(hidden_size, visible_size) # ====================================================================== # STEP 2: Implement sparseAutoencoderCost # # You can implement all of the components in the cost function at once, but it may be easier to do # it step-by-step and run gradient checking (see STEP 3) after each step. We # suggest implementing the sparseAutoencoderCost function using the following steps: # # (a) Implement forward propagation in your neural network, and implement the # squared error term of the cost function. Implement backpropagation to # compute the derivatives. Then (using lambda=beta=0), run Gradient Checking # to verify that the calculations corresponding to the squared error cost # term are correct. # # (b) Add in the weight decay term (in both the cost function and the derivative