def investigate(): partsPlot1,partsPlot2,extractedFeature = displayParts() for partIndex in range(20): test = [] smallerPart = [] for k in range(1000): x = extractedFeature[4][k] for m in range(8): for n in range(8): if(x[m,n,0] == partIndex): test.append((k,m,n)) smallerPart.append(extractedFeature[2][k,3 * m + 1,3 * n + 1]) number = np.zeros(200) for x in smallerPart: if(x!=-1): number[x]+=1 #plot1 = plot.figure(partIndex) #plot.plot(number) #plot.savefig('frequency %i.png' %partIndex) #plot.close() index = np.where(number > 100)[0] partNew2 = np.ones((index.shape[0] + 1,6,6)) partNew2[0] = partsPlot2[partIndex] for i in range(index.shape[0]): partNew2[i + 1,0:4,0:4] = partsPlot1[index[i],:,:] fileString = 'part%i.png' %partIndex gr.images(partNew2,zero_to_one=False, show=False,vmin = 0, vmax = 1, fileName = fileString)
def main(): # Load the dataset print("Loading data...") X_train, y_train, X_test, y_test = load_data("/X_train.npy", "/Y_train.npy", "/X_test.npy", "/Y_test.npy") input_var = T.ftensor4('inputs') target_var = T.fmatrix('targets') network = lasagne.layers.InputLayer(shape=(None, 1, 28, 28), input_var=input_var, name = 'input_layer') network_reshape = lasagne.layers.ReshapeLayer(network, shape=([0], 784), name = 'reshape_layer') labelInput = lasagne.layers.InputLayer(shape=(None, 10), input_var=target_var) network = lasagne.layers.ConcatLayer( [network_reshape, labelInput], axis = 1) network = lasagne.layers.MultiGaussianMixture(network, num_components = 5, n_classes = 10 , name = 'output_layer') loss = lasagne.layers.get_output(network) loss.name = 'loss' loss_mean = loss.mean() loss_mean.name = 'loss_mean' params = lasagne.layers.get_all_params(network, trainable=True) print(params) #updates = lasagne.updates.nesterov_momentum( # loss_mean, params, learning_rate=0.001, momentum=0.9) gparams = T.grad(loss_mean, params) updates = [ (param, param - 0.1 * gparam) for param, gparam in zip(params, gparams) ] train_fn = theano.function([input_var, target_var], loss_mean, updates=updates) X_train_zero = np.array(X_train[y_train == 0], dtype = np.float32) y_train_zero = np.array(y_train[y_train == 0], dtype = np.float32) for epoch in range(2): train_err = 0 batch_index = 0 for batch in iterate_minibatches(X_train_zero, y_train_zero, 100, 10, shuffle=True): inputs, targets = batch current_result = train_fn(inputs, targets) train_err += current_result print("-------------") print(current_result) print(np.mean(lasagne.layers.get_all_param_values(network)[2][0])) print("-------------") batch_index += 1 print(train_err) learnedWeights = lasagne.layers.get_all_param_values(network) import amitgroup.plot as gr gr.images(lasagne.layers.get_all_param_values(network)[0][0].reshape(5,28,28))
def main(): current_model = "train" print("load data...") X_train, y_train, X_test, y_test = load_data("/X_train.npy", "/Y_train.npy", "/X_test.npy", "/Y_test.npy") print("bulding function...") #extract_function = encoder_extraction(extraction_layer = 6, weights_file = "../data/mnist_autoencoder_params_encoder_linear_decoder_no_bias.npy") extract_function = encoder_extraction() print("feature extraction...") X_train_feature = extract(X_train, extract_function) X_test_feature = extract(X_test, extract_function) print(X_train_feature.shape) print(X_test_feature.shape) print("train llh model...") objectModelLayer = pnet.MixtureClassificationLayer(n_components = 5, min_prob = 0.0001, mixture_type = "gaussian") if current_model == "train": objectModelLayer.train(X_train_feature, y_train) np.save("../data/object_model_rectify_activation_10_class_gaussian_fc.npy", objectModelLayer._models) print("object model classification accuracy: ", np.mean(objectModelLayer.extract(X_test_feature) == y_test)) data = createSampleTest(nSample = 1) gr.images(data[0]) processedData = reprocess_data_for_extraction(data) processedData_reshape = processedData.reshape(-1, 1, 28, 28) processedData_reshape_feature = extract_function(np.array(processedData_reshape,dtype = np.float32)) llh_for_data = objectModelLayer.score(processedData_reshape_feature).reshape(processedData.shape[:3] + (10, )) print llh_for_data.shape print(np.max(llh_for_data, axis = -1)) print(np.argmax(llh_for_data, axis = -1)) print(np.argmax(llh_for_data, axis = -1)[nonMaximalSupress(np.max(llh_for_data, axis = -1), windowSize = 5) != INT_MIN]) print (y_train[:1]) else: model_means = np.load("../data/reconstructModel_mean.npy") model_sigmas = np.load("../data/reconstructModel_sigma.npy") model_weights = np.load("../data/reconstructModel_weights.npy") objectModelLayer._modelinstance = [] for i in range(10): from sklearn.mixture import GMM mm = GMM(n_components = 5, n_iter = 20, n_init = 1, random_state = 0, covariance_type = 'full') mm.covars_ = model_sigmas[i] mm.means_ = model_means[i].reshape(5, -1) mm.weights_ = model_weights[i] objectModelLayer._modelinstance.append(mm) objectModelLayer._models = model_means print("calculating classification accuracy...") print("object model classification accuracy: ", np.mean(objectModelLayer.extract(X_train.reshape(X_train.shape[0], -1)) == y_train)) data = createSampleTest(nSample = 1) gr.images(data[0]) processedData = reprocess_data_for_extraction(data) processedData_reshape = processedData.reshape(-1, 784) llh_for_data = objectModelLayer.score(processedData_reshape)#.reshape(processedData.shape[:3] + (10, )) print llh_for_data.shape
def displayParts(): # load the trained Image #X = np.load('testNew.npy') X = np.load('testNewPooling44.npy') model = X.item() # get the parts Layer numParts1 = model['layers'][1]['num_parts'] numParts2 = model['layers'][3]['num_parts'] net = pnet.PartsNet.load_from_dict(model) allLayer = net.layers print(allLayer) ims,labels = ag.io.load_mnist('training') extractedFeature = [] for i in range(5): extractedFeature.append(extract(ims[0:1000],allLayer[0:i])[0]) extractedParts1 = extractedFeature[2] #extractedParts11 = extract(ims[0:1000],allLayer[0:6])[1] #print(extractedParts11) extractedParts2 = extractedFeature[4] print(extractedParts2.shape) partsPlot1 = np.zeros((numParts1,6,6)) partsCodedNumber1 = np.zeros(numParts1) partsPlot2 = np.zeros((numParts2,12,12)) partsCodedNumber2 = np.zeros(numParts2) for i in range(1000): codeParts1 = extractedParts1[i].reshape(extractedParts1[i].shape[0:2]) codeParts2 = extractedParts2[i].reshape(extractedParts2[i].shape[0:2]) for m in range(23): for n in range(23): if(codeParts1[m,n]!=-1): partsPlot1[codeParts1[m,n]]+=ims[i,m:m+6,n:n+6] partsCodedNumber1[codeParts1[m,n]]+=1 for p in range(23)[3:19]: for q in range(23)[3:19]: if(codeParts2[p,q]!=-1): partsPlot2[codeParts2[p,q]]+=ims[i,p - 3:p + 9,q - 3: q + 9] partsCodedNumber2[codeParts2[p,q]]+=1 #if(codeParts2[p,q,1]!=-1): # partsPlot2[codeParts2[p,q,1]]+=ims[i,p:p+10,q:q+10] # partsCodedNumber2[codeParts2[p,q,1]]+=1 for j in range(numParts1): partsPlot1[j] = partsPlot1[j]/partsCodedNumber1[j] for k in range(numParts2): partsPlot2[k] = partsPlot2[k]/partsCodedNumber2[k] print(partsPlot1.shape) gr.images(partsPlot1,vmin = 0,vmax = 1) gr.images(partsPlot2[0:1000],vmin = 0,vmax = 1) print(partsCodedNumber1) print("-----------------") print(partsCodedNumber2) return partsPlot1,partsPlot2,extractedFeature
def main(): hf = h5py.File("/hdd/Documents/Data/3D-MNIST/full_dataset_vectors.h5", "r") X_train = hf["X_train"][0].reshape(1, 1, 16, 16, 16) X_train = np.rollaxis(X_train, 2, 5) X_train = np.array(X_train, dtype = np.float32) input_var = T.tensor5('inputs') network = build_cnn(input_var, 1) network_output = lasagne.layers.get_output(network) get_rotated = theano.function([input_var], network_output) image_result = get_rotated(X_train) import amitgroup as ag import amitgroup.plot as gr gr.images(np.mean(image_result, axis = 4)[0,0]) return
def train_from_samples(self, patches, original_patches): print(patches.shape) print(original_patches.shape) from pnet.bernoullimm import BernoulliMM print(patches.shape) min_prob = self._settings.get('min_prob', 0.01) parts = np.ones((self._num_parts,) + patches[0].shape) d = np.prod(patches.shape[2:]) flatpatches = patches.reshape((patches.shape[0], -1)) rng = np.random.RandomState(self._settings.get('em_seed',0)) from pnet.latentShiftRotationEM import LatentShiftRotationEM permutation = np.empty((self._num_rot, self._num_rot * d),dtype = np.int_) for a in range(self._num_rot): if a == 0: permutation[a] = np.arange(self._num_rot * d) else: permutation[a] = np.roll(permutation[a-1],d) partsPermutation = np.empty((self._num_rot, self._num_rot * self._part_shape[0] * self._part_shape[1] * 8),dtype = np.int_) for a in range(self._num_rot): if a == 0: partsPermutation[a] = np.arange(self._num_rot * self._part_shape[0] * self._part_shape[1] * 8) else: partsPermutation[a] = np.roll(partsPermutation[a-1],self._part_shape[0] * self._part_shape[1] * 8) result = LatentShiftRotationEM(flatpatches,num_mixture_component = self._num_parts, parts_shape = (self._part_shape[0],self._part_shape[1],8),region_shape = (self._sample_shape[1],self._sample_shape[1],8),shifting_shape = self._shifting_shape,num_rot = self._num_rot, max_num_iteration = 25, loglike_tolerance=1e-3, mu_truncation = (1,1),additional_mu = None, permutation = permutation,partPermutation = partsPermutation, numpy_rng=rng, verbose = True) comps = result[3] parts = result[1].reshape((self._num_parts * self._num_rot,self._part_shape[0], self._part_shape[1],8)) self._bkg_probability = result[4] self._parts = parts allShift = np.zeros(25) for i in comps[:,2]: allShift[i]+=1 print(allShift/np.sum(allShift)) original_patches = np.swapaxes(original_patches,0,1) self._visparts = np.asarray([original_patches[comps[:,0]==k, comps[comps[:,0]==k][:,1],comps[comps[:,0]==k][:,2]].mean(0) for k in range(self._num_parts)]) import amitgroup.plot as gr gr.images(self._visparts, zero_to_one=False, show=False,vmin=0,vmax = 1, fileName = 'moduleShiftingParts1.png') return parts
def test2(): #TODO: NOT RIGHT# X = np.load('testNew.npy') model = X.item() net = pnet.PartsNet.load_from_dict(model) allLayer = net.layers print(allLayer) ims, labels = ag.io.load_mnist('training') extractedParts = extract(ims[0:1000],allLayer[0:2]) #return extractedParts allParts = extractedParts[0] parts_layer = allLayer[1] parts = parts_layer._parts.reshape(100,6,6) #for i in range(200): ims = ims[0:1000] labels = labels[0:1000] #print(ims.shape) classifiedLabel = net.classify(ims) #print out all the misclassified images misclassify = np.nonzero(classifiedLabel!=labels) misclassify = np.append([],np.asarray(misclassify, dtype=np.int)) numMisclassify = len(misclassify) image = np.ones((numMisclassify,25 * 5,25*5)) * 0.5 print(misclassify) for j in range(numMisclassify): i = int(misclassify[j]) print(allParts[i].shape) thisParts = allParts[i].reshape(allParts[i].shape[0:2]) for m in range(25): for n in range(25): if(thisParts[m,n]!=-1): image[j,m*5:m*5+4,n*5:n*5+4] = parts[thisParts[m,n]] else: image[j,m*5:m*5+4,n*5:n*5+4] = 0 gr.images(image)
def trainPOP(): X = np.load("test4.npy") model = X.item() # get num of Parts numParts = model['layers'][1]['num_parts'] net = pnet.PartsNet.load_from_dict(model) allLayer = net.layers ims,labels = ag.io.load_mnist('training') trainingDataNum = 1000 extractedFeature = extract(ims[0:trainingDataNum],allLayer[0:2])[0] print(extractedFeature.shape) extractedFeature = extractedFeature.reshape(extractedFeature.shape[0:3]) partsPlot = np.zeros((numParts,6,6)) partsCodedNumber = np.zeros(numParts) #every list corresponding to the larger region surrounding 10x10 region of the 5*5 region coded by this part imgRegion = [[] for x in range(numParts)] partsRegion = [[] for x in range(numParts)] #Part Visualize# for i in range(trainingDataNum): codeParts = extractedFeature[i] for m in range(23): for n in range(23): if(codeParts[m,n]!=-1): partsPlot[codeParts[m,n]]+=ims[i,m:m+6,n:n+6] partsCodedNumber[codeParts[m,n]]+=1 for j in range(numParts): partsPlot[j] = partsPlot[j]/partsCodedNumber[j] secondLayerCodedNumber = 0 if 1: for i in range(trainingDataNum): codeParts = extractedFeature[i] for m in range(23)[3:20]: for n in range(23)[3:20]: if(codeParts[m,n]!=-1): imgRegion[codeParts[m,n]].append(ims[i,m-3:m+9,n-3:n+9]) secondLayerCodedNumber+=1 partsGrid = partsPool(codeParts[m-3:m+4,n-3:n+4],numParts) partsRegion[codeParts[m,n]].append(partsGrid) for i in range(numParts): print(len(partsRegion[i])) ##Second-Layer Parts numSecondLayerParts = 20 allPartsLayer = [[pnet.PartsLayer(numSecondLayerParts,(1,1),settings=dict(outer_frame=0,threshold=5, sample_per_image=1, max_samples=10000, min_prob=0.005))] for i in range(numParts)] allPartsLayerImg = np.zeros((numParts,numSecondLayerParts,12,12)) allPartsLayerImgNumber = np.zeros((numParts,numSecondLayerParts)) print("====================================================") zeroParts = 0 for i in range(numParts): print("test") allPartsLayer[i][0].train_from_samples(np.array(partsRegion[i]),None) print(np.array(partsRegion[i]).shape) extractedFeaturePart = extract(np.array(partsRegion[i],dtype = np.uint8),allPartsLayer[i])[0] print(extractedFeaturePart.shape) for j in range(len(partsRegion[i])): if(extractedFeaturePart[j,0,0,0]!=-1): partIndex = extractedFeaturePart[j,0,0,0] allPartsLayerImg[i,partIndex]+=imgRegion[i][j] allPartsLayerImgNumber[i,partIndex]+=1 else: zeroParts+=1 for i in range(numParts): for j in range(numSecondLayerParts): allPartsLayerImg[i,j] = allPartsLayerImg[i,j]/allPartsLayerImgNumber[i,j] print(allPartsLayer[i][0]._weights) #print(zeroParts) #print(np.sum(allPartsLayerImgNumber),secondLayerCodedNumber) settings = {'interpolation':'nearest','cmap':plot.cm.gray,} settings['vmin'] = 0 settings['vmax'] = 1 plotData = np.ones((14*100+2,14*(numSecondLayerParts + 1)+2))*0.8 visualShiftParts = 0 if 0: allPartsPlot = np.zeros((20,11,12,12)) gr.images(partsPlot.reshape(numParts,6,6),zero_to_one=False,vmin = 0, vmax = 1) allPartsPlot[:,0] = 0.5 allPartsPlot[:,0,3:9,3:9] = partsPlot[20:40] allPartsPlot[:,1:,:,:] = allPartsLayerImg[20:40] gr.images(allPartsPlot.reshape(220,12,12),zero_to_one=False, vmin = 0, vmax =1) elif 0: for i in range(numSecondLayerParts + 1): for j in range(100): if i == 0: plotData[5 + j * 14:11 + j * 14, 5 + i * 14: 11 + i * 14] = partsPlot[j+visualShiftParts] else: plotData[2 + j * 14:14 + j * 14,2 + i * 14: 14 + i * 14] = allPartsLayerImg[j+visualShiftParts,i-1] plot.figure(figsize=(10,40)) plot.axis('off') plot.imshow(plotData, **settings) plot.savefig('test.pdf',format='pdf',dpi=900) else: pass
def train_from_samples(self, patches, original_patches): # from pnet.latent_bernoulli_mm import LatentBernoulliMM from pnet.bernoullimm import BernoulliMM print(patches.shape) min_prob = self._settings.get("min_prob", 0.01) # num_permutation = self._shifting_shape[0] * self._shifting_shape[1] # parts = np.ones((self._num_true_parts * num_permutation ,) + patches.shape[2:]) parts = np.ones((self._num_parts,) + patches[0].shape) d = np.prod(patches.shape[1:]) # print(d,num_permutation) if 0: # \permutation = np.empty((num_permutation, num_permutation * d),dtype = np.int_) for a in range(num_permutation): if a == 0: permutation[a] = np.arange(num_permutation * d) else: permutation[a] = np.roll(permutation[a - 1], d) flatpatches = patches.reshape((patches.shape[0], -1)) print(flatpatches.shape) if 0: mm = BernoulliMM( n_components=num_parts, n_iter=20, tol=1e-15, n_init=2, random_state=0, min_prob=min_prob, verbose=False ) print(mm.fit(flatpatches)) print("AIC", mm.aic(flatpatches)) print("BIC", mm.bic(flatpatches)) # import pdb; pdb.set_trace() parts = mm.means_.reshape((num_parts,) + patches.shape[1:]) # self._weights = mm.weights_ elif 0: from pnet.bernoulli import em print("before EM") ret = em( flatpatches, self._num_true_parts, 10, mu_truncation=min_prob, permutation=permutation, numpy_rng=self._settings.get("em_seed", 0), verbose=True, ) comps = ret[3] parts = ret[1].reshape((self._num_true_parts * num_permutation,) + patches.shape[2:]) self._weights = np.arange(self._num_parts) else: rng = np.random.RandomState(self._settings.get("em_seed", 0)) from pnet.latentShiftEM import LatentShiftEM # from latentShiftEM import latentShiftEM result = LatentShiftEM( flatpatches, num_mixture_component=self._num_parts, parts_shape=(self._part_shape[0], self._part_shape[1], 8), region_shape=(self._sample_shape[1], self._sample_shape[1], 8), shifting_shape=self._shifting_shape, max_num_iteration=25, loglike_tolerance=1e-3, mu_truncation=(1, 1), additional_mu=None, permutation=None, numpy_rng=rng, verbose=True, ) comps = result[3] print(comps.shape) print(original_patches.shape) print(result[1].shape) parts = result[1].reshape((self._num_parts, self._part_shape[0], self._part_shape[1], 8)) self._bkg_probability = result[4] self._parts = parts print(comps[:50, 0]) print(comps[:50, 1]) self._visparts = np.asarray( [original_patches[comps[:, 0] == k, comps[comps[:, 0] == k][:, 1]].mean(0) for k in range(self._num_parts)] ) print(self._visparts.shape) import amitgroup.plot as gr gr.images(self._visparts, zero_to_one=False, show=False, vmin=0, vmax=1, fileName="moduleShiftingParts1.png") return parts
def train(): """Train CIFAR-10 for a number of steps.""" image_input_var = T.tensor4('original_inputs') target_var = T.imatrix('target_mask') layer_result = build_mask_nn(image_input_var) model_output = lasagne.layers.get_output(layer_result, deterministic=True) all_loss = -T.mean(T.log(model_output) * target_var + T.log(1-model_output) * (1 - target_var), axis = 1) loss = T.mean(all_loss) params = lasagne.layers.get_all_params(layer_result, trainable=True) updates = lasagne.updates.adam(loss, params, learning_rate=0.001) #updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate=0.1, momentum=0.9) train_fn = theano.function([image_input_var, target_var], loss, updates=updates) val_fn = theano.function([image_input_var, target_var], [loss, model_output]) if 0: if os.path.isfile(os.path.join(train_dir, 'latest_model_mask.txt')): weight_file = "" with open(os.path.join(train_dir, 'latest_model_mask.txt'), 'r') as checkpoint_file: weight_file = checkpoint_file.read().replace('\n', '') print("Loading from: ", weight_file) model_weights = np.load(weight_file) current_weights = lasagne.layers.get_all_param_values(cnn_model) for i in range(len(model_weights)): print(model_weights[i].shape) final_weights = [model_weights[i] for i in range(len(model_weights) - 2)] + [current_weights[-2], current_weights[-1]] lasagne.layers.set_all_param_values(layer_result, final_weights) # Get images and labels for CIFAR-10. cifar10_data = cifar10_input.load_cifar10() for epoch in range(max_steps): start_time = time.time() test_image, test_target = cifar10_data.test.next_eval_batch(batch_size) print("Start Evaluating %d" % epoch) total_loss_value = 0 total_count = 0 tmp_predicted = None tmp_target = None while(test_image is not None): loss_value, predicted_target = val_fn(test_image, test_target) tmp_predicted = predicted_target tmp_target = test_target total_count += test_image.shape[0] total_loss_value += loss_value * test_image.shape[0] test_image, test_target = cifar10_data.test.next_eval_batch(batch_size) print("Final Test Loss: %.4f" % (float(total_loss_value / total_count))) print("Start To Train") train_image, train_target, start = cifar10_data.train.next_batch(batch_size) end_time_1 = time.time() - start_time step = 1 total_loss_value = 0 total_count = 0 start = 1 while(start != 0): loss_value = train_fn(train_image, train_target) train_image, train_target, start = cifar10_data.train.next_batch(batch_size) step += 1 assert not np.isnan(loss_value), 'Model diverged with loss = NaN' total_loss_value += loss_value * train_image.shape[0] total_count += train_image.shape[0] print("Epoch Stop, loss_averge", float(total_loss_value) / float(total_count)) duration = time.time() - start_time print("Duration is", duration) if epoch % 100 == 0 or (epoch + 1) == max_steps: checkpoint_path = os.path.join(train_dir, 'model_mask_step%d.npy' % epoch) weightsOfParams = lasagne.layers.get_all_param_values(layer_result) np.save(checkpoint_path, weightsOfParams) latest_model_path = os.path.join(train_dir, 'latest_model_mask.txt') try: os.remove(latest_model_path) except OSError: pass latest_model_file = open(latest_model_path, "w") latest_model_file.write(checkpoint_path) latest_model_file.close() if epoch % 200 == 0: gr.images(tmp_predicted[:50].reshape(50, 32, 32)) gr.images(tmp_target[:50].reshape(50, 32, 32))
def train(image_with_bkg, target_mask_file_name, target_decluttered_file_name): """Train CIFAR-10 for a number of steps.""" image_input_var = T.tensor4('original_inputs') cnn_model = cifar10.build_cnn(image_input_var) model_output_eval = lasagne.layers.get_output(cnn_model, deterministic=True) params = lasagne.layers.get_all_params(cnn_model, trainable=True) val_fn = theano.function([image_input_var], model_output_eval) if os.path.isfile(os.path.join(train_dir, 'latest_model_mask_5x5.txt')): #if os.path.isfile(os.path.join(train_dir, 'latest_model_mask_1x1.txt')): #if os.path.isfile(os.path.join(train_dir, 'latest_model_mask_5x3x3xfully.txt')): weight_file = "" with open(os.path.join(train_dir, 'latest_model_mask_5x5.txt'), 'r') as checkpoint_file: #with open(os.path.join(train_dir, 'latest_model_mask_1x1.txt'), 'r') as checkpoint_file: #with open(os.path.join(train_dir, 'latest_model_mask_5x3x3xfully.txt'), 'r') as checkpoint_file: weight_file = checkpoint_file.read().replace('\n', '') print("Loading from: ", weight_file) model_weights = np.load(weight_file) for i in range(model_weights.shape[0]): print(model_weights[i].shape) print("==========================") current_weights = lasagne.layers.get_all_param_values(cnn_model) for i in range(len(current_weights)): print(current_weights[i].shape) lasagne.layers.set_all_param_values(cnn_model, model_weights) else: print("Weights not found") sys.exit() image_size = image_with_bkg.shape[0] batch_size = 100 image_masks = [] decluttered_images = [] threshold = 0.25 for i in range(image_size // batch_size + 1): test_image = image_with_bkg[i * batch_size : min(i * batch_size + batch_size, image_size)] if test_image.shape[0] == 0: break predicted_target = val_fn(np.rollaxis(test_image, 3, 1)) image_masks.append(predicted_target > threshold) current_image_mask = image_masks[-1].reshape(-1, 32, 32, 1) current_image_mask = np.repeat(current_image_mask, 3, axis = 3) bgcolor = np.array([255, 255, 255]).reshape(1, 1, 1, 3) decluttered_images.append(current_image_mask * test_image + (1 - current_image_mask) * bgcolor) if i == 0: gr.images(predicted_target.reshape(-1, 32, 32)) plt.imshow(test_image.reshape(-1, 32, 32, 3)[0,:,:,:]) plt.show() plt.imshow(decluttered_images[0][0]/255.0) plt.show() image_masks = np.vstack(image_masks) decluttered_images = np.vstack(decluttered_images) np.save(target_mask_file_name, image_masks) np.save(target_decluttered_file_name, decluttered_images)
#np.save("exPartsOriginalJun29.npy",allPartsLayer) if 0: """ Visualize the SuperParts """ settings = {'interpolation':'nearest','cmap':plot.cm.gray,} settings['vmin'] = 0 settings['vmax'] = 1 plotData = np.ones(((2 + secondLayerShape)*100+2,(2+secondLayerShape)*(numSecondLayerParts + 1)+2))*0.8 visualShiftParts = 0 if 0: allPartsPlot = np.zeros((20,numSecondLayerParts + 1,12,12)) gr.images(partsPlot.reshape(numParts,6,6),zero_to_one=False,vmin = 0, vmax = 1) allPartsPlot[:,0] = 0.5 allPartsPlot[:,0,3:9,3:9] = partsPlot[20:40] allPartsPlot[:,1:,:,:] = allPartsLayerImg[20:40] gr.images(allPartsPlot.reshape(20 * (numSecondLayerParts + 1),12,12),zero_to_one=False, vmin = 0, vmax =1) elif 1: for i in range(numSecondLayerParts + 1): for j in range(numParts): if i == 0: plotData[5 + j * (2 + secondLayerShape):5+firstLayerShape + j * (2 + secondLayerShape), 5 + i * (2 + secondLayerShape): 5+firstLayerShape + i * (2 + secondLayerShape)] = partsPlot[j+visualShiftParts] else: plotData[2 + j * (2 + secondLayerShape):2 + secondLayerShape+ j * (2 + secondLayerShape),2 + i * (2 + secondLayerShape): 2+ secondLayerShape + i * (2 + secondLayerShape)] = allPartsLayerImg[j+visualShiftParts,i-1] plot.figure(figsize=(10,40)) plot.axis('off') plot.imshow(plotData, **settings) plot.savefig('originalExParts_2.pdf',format='pdf',dpi=900)
def main(): from dataPreparation import load_data # Load the dataset print("Loading data...") X_train, y_train, X_test, y_test = load_data("/X_train.npy", "/Y_train.npy", "/X_test.npy", "/Y_test.npy") X_train = np.array(X_train, dtype = np.float32) y_train = np.array(y_train, dtype = np.float32) X_test = np.array(X_test, dtype = np.float32) y_test = np.array(y_test, dtype = np.float32) #X_train, y_train, X_test, y_test = load_data("/cluttered_train_x.npy", "/cluttered_train_y.npy", "/cluttered_test_x.npy", "/cluttered_test_y.npy", dataset = "MNIST_CLUTTER") dimension = np.prod(X_train.shape[1:]) # Prepare Theano variables for inputs and targets input_var = T.ftensor4('inputs') target_var = T.fmatrix('targets') target_input_var = input_var.reshape((-1, dimension)) # Create neural network model (depending on first command line parameter) gaussian_output, encoder_output, decoder_output = build_cnn(input_var, target_var) #weightsOfParams = np.load("../data/mnist_autoencoder_params_encoder_linear_decoder.npy") #lasagne.layers.set_all_param_values(fc, weightsOfParams[:4]) # Create a loss expression for training, i.e., a scalar objective we want # to minimize (for our multi-class problem, it is the cross-entropy loss): llh_output = lasagne.layers.get_output(gaussian_output) loss_1 = lasagne.objectives.multi_negative_llh(llh_output, target_var) fc_output = lasagne.layers.get_output(encoder_output) loss_mean_1 = T.mean(loss_1) reconstruction = lasagne.layers.get_output(decoder_output) loss_mean_2 = T.mean(T.mean(lasagne.objectives.squared_error(reconstruction, target_input_var), axis = 1)) alpha = T.scalar('alpha', dtype=theano.config.floatX) combination = T.exp(alpha)/(1 + T.exp(alpha)) loss_mean_burn = combination * loss_mean_1 / 10000 + (1 - combination) * loss_mean_2 # We could add some weight decay as well here, see lasagne.regularization. # Create update expressions for training, i.e., how to modify the # parameters at each training step. Here, we'll use Stochastic Gradient # Descent (SGD) with Nesterov momentum, but Lasagne offers plenty more. #params = list(set(lasagne.layers.get_all_params(decoder_output, trainable=True) + lasagne.layers.get_all_params(gaussian_output, trainable=True))) params = lasagne.layers.get_all_params(decoder_output, trainable=True) gaussianMixtureParameters = lasagne.layers.get_all_params(gaussian_output, trainable = True) gaussianParam = [] for param in gaussianMixtureParameters: if param not in params: gaussianParam.append(param) print(params) print("model built") #updates = lasagne.updates.nesterov_momentum( # loss_mean_2, params, learning_rate=0.1, momentum=0.9) gparams = T.grad(loss_mean_burn, params + gaussianParam) print(gparams) updates = [] for param, gparam in zip(params + gaussianParam, gparams): if param in params: updates.append((param, param - 0.0001 * gparam)) elif param in gaussianParam: updates.append((param, param - 0.01 * gparam)) print(updates) #0.000001 # updates = [(param, param - 0.0000001 * gparam) # for param, gparam in zip(params[:4], gparams[:4])] + [(params[4], params[4] - 0.01 * gparams[4])] #updates = [(param, param - 0.01 * gparam) # for param, gparam in zip(params[:8], gparams[:8])] + [(param, param - 0.01 * gparam) for param, gparam in zip(params[8:], gparams[8:])] #updates = [(param, param - 0.05 * gparam) for param, gparam in zip(params, gparams)] # Create a loss expression for validation/testing. The crucial difference # here is that we do a deterministic forward pass through the network, # disabling dropout layers. test_llh_output = lasagne.layers.get_output(gaussian_output, deterministic=True) test_loss = lasagne.objectives.multi_negative_llh(test_llh_output, target_var) test_loss_mean = T.mean(test_loss) test_reconstruction = lasagne.layers.get_output(decoder_output, deterministic = True) test_loss_reconstruction_mean = T.mean(T.mean(lasagne.objectives.squared_error(test_reconstruction, target_input_var), axis = 1)) test_total_loss = combination * test_loss_mean / 10000 + (1 - combination) * test_loss_reconstruction_mean # As a bonus, also create an expression for the classification accuracy: # Compile a function performing a training step on a mini-batch (by giving # the updates dictionary) and returning the corresponding training loss: # + [update_param for update_param in gparams] train_fn = theano.function([input_var, target_var, alpha], [loss_mean_burn, llh_output, loss_mean_1, loss_mean_2], updates=updates) # Compile a second function computing the validation loss and accuracy: val_fn = theano.function([input_var, target_var, alpha], [test_total_loss, test_llh_output, test_loss_mean, test_loss_reconstruction_mean, test_reconstruction]) # Finally, launch the training loop. print("Starting training...") # We iterate over epochs: loss_combination = 0 num_epochs = 2000 for epoch in range(num_epochs): #loss_combination = loss_combination + 0.02 # In each epoch, we do a full pass over the training data: train_err = 0 train_reconstruction_err = 0 train_llh_err = 0 train_batches = 0 start_time = time.time() batchIndex = 0 for batch in iterate_minibatches(X_train, y_train, 50, 10, shuffle=True): inputs, targets = batch #current_loss_mean, current_loss, fc_output = train_fn(inputs, targets) current_result = train_fn(inputs, targets, loss_combination) # if batchIndex % 2000 == 0: # print(current_result) batchIndex = batchIndex + 1 train_err += current_result[0] train_reconstruction_err += current_result[3] train_llh_err += current_result[2] train_batches += 1 break if epoch % 100 == 0: print(loss_combination) # Then we print the results for this epoch: print("Epoch {} of {} took {:.3f}s".format( epoch + 1, num_epochs, time.time() - start_time)) print(" training loss:\t\t{:.6f}".format(train_err / train_batches)) print(" train llh loss:\t\t{:.6f}".format(train_llh_err / train_batches)) print(" training reconstruction loss:\t\t{:.6f}".format(train_reconstruction_err / train_batches)) #print(lasagne.layers.get_all_param_values(network)[-3]) print("--") # After training, we compute and print the test error: test_err = 0 test_llh_err = 0 test_reconstruction_err = 0 test_batches = 0 accurate = 0 for batch in iterate_minibatches(X_test, y_test, 50, 10, shuffle=False): inputs, targets = batch valuationResult = val_fn(inputs, targets, loss_combination) err = valuationResult[0] test_err += err test_llh_err += valuationResult[2] test_reconstruction_err += valuationResult[3] test_batches += 1 accurate += np.sum(np.argmax(targets, axis = 1) == np.argmax(valuationResult[1], axis = 1)) if epoch % 500 == 0: gr.images(valuationResult[4].reshape(-1, 28, 28), show = False, fileName = "/Users/jiajunshen/Desktop/%d.png"%epoch) print("Final results:") print(" test loss:\t\t\t{:.6f}".format(test_err / test_batches)) print(" test llh loss:\t\t\t{:.6f}".format(test_llh_err / test_batches)) print(" test reconstruction loss:\t\t\t{:.6f}".format(test_reconstruction_err / test_batches)) print("Test Accuracy: ", accurate / 10000.0)
def testInvestigation(ims, labels, net): yhat = net.classify((ims,500)) return np.where(yhat!=labels), yhat #X = np.load("testMay151.npy") #X = np.load("_3_100*6*6_1000*1*1_Jun_16_danny.npy") X = np.load("original6*6 2.npy") #X = np.load("sequential6*6.npy") model = X.item() # get num of Parts numParts = model['layers'][1]['num_parts'] net = pnet.PartsNet.load_from_dict(model) allLayer = net.layers ims,labels = ag.io.load_mnist('training') trainingDataNum = 1000 firstLayerShape = 6 extractedFeature = extract(ims[0:trainingDataNum],allLayer[0:2])[0] print(extractedFeature.shape) extractedFeature = extractedFeature.reshape(extractedFeature.shape[0:3]) partsPlot = np.zeros((numParts,firstLayerShape,firstLayerShape)) partsCodedNumber = np.zeros(numParts) imgRegion= [[] for x in range(numParts)] partsRegion = [[] for x in range(numParts)] for i in range(trainingDataNum): codeParts = extractedFeature[i] for m in range(29 - firstLayerShape): for n in range(29 - firstLayerShape): if(codeParts[m,n]!=-1): partsPlot[codeParts[m,n]]+=ims[i,m:m+firstLayerShape,n:n+firstLayerShape] partsCodedNumber[codeParts[m,n]]+=1 for j in range(numParts): partsPlot[j] = partsPlot[j]/partsCodedNumber[j] secondLayerCodedNumber = 0 secondLayerShape = 12 frame = (secondLayerShape - firstLayerShape)/2 frame = int(frame) totalRange = 29 - firstLayerShape if 1: for i in range(trainingDataNum): codeParts = extractedFeature[i] for m in range(totalRange)[frame:totalRange - frame]: for n in range(totalRange)[frame:totalRange - frame]: if(codeParts[m,n]!=-1): imgRegion[codeParts[m,n]].append(ims[i, m - frame:m + secondLayerShape - frame,n - frame:n + secondLayerShape - frame]) secondLayerCodedNumber+=1 partsGrid = partsPool(codeParts[m-frame:m+frame + 1,n-frame:n+frame + 1],numParts) partsRegion[codeParts[m,n]].append(partsGrid) newPartsRegion = [] for i in range(numParts): newPartsRegion.append(np.asarray(partsRegion[i],dtype = np.uint8)) np.save('/var/tmp/partsRegionOriginalJun29.npy',newPartsRegion) np.save('/var/tmp/imgRegionOriginalJun29.npy',imgRegion) ##second-layer parts numSecondLayerParts = 10 allPartsLayer = [[pnet.PartsLayer(numSecondLayerParts,(1,1), settings=dict(outer_frame = 0, threshold = 5, sample_per_image = 1, max_samples=10000, min_prob = 0.005, #min_llh = -40 ))] for i in range(numParts)] allPartsLayerImg = np.zeros((numParts,numSecondLayerParts,secondLayerShape,secondLayerShape)) allPartsLayerImgNumber = np.zeros((numParts,numSecondLayerParts)) zeroParts = 0 imgRegionPool = [[] for i in range(numParts * numSecondLayerParts)] for i in range(numParts): if(not partsRegion[i]): continue allPartsLayer[i][0].train_from_samples(np.array(partsRegion[i]),None) extractedFeaturePart = extract(np.array(partsRegion[i],dtype = np.uint8),allPartsLayer[i])[0] print(extractedFeaturePart.shape) for j in range(len(partsRegion[i])): if(extractedFeaturePart[j,0,0,0]!=-1): partIndex = extractedFeaturePart[j,0,0,0] allPartsLayerImg[i,partIndex]+=imgRegion[i][j] imgRegionPool[i * numSecondLayerParts + partIndex].append(imgRegion[i][j]) allPartsLayerImgNumber[i,partIndex]+=1 else: zeroParts+=1 for i in range(numParts): for j in range(numSecondLayerParts): if(allPartsLayerImgNumber[i,j]): allPartsLayerImg[i,j] = allPartsLayerImg[i,j]/allPartsLayerImgNumber[i,j] if 1: """ Visualize the SuperParts """ settings = {'interpolation':'nearest','cmap':plot.cm.gray,} settings['vmin'] = 0 settings['vmax'] = 1 plotData = np.ones(((2 + secondLayerShape)*100+2,(2+secondLayerShape)*(numSecondLayerParts + 1)+2))*0.8 visualShiftParts = 0 if 0: allPartsPlot = np.zeros((20,numSecondLayerParts + 1,12,12)) gr.images(partsPlot.reshape(numParts,6,6),zero_to_one=False,vmin = 0, vmax = 1) allPartsPlot[:,0] = 0.5 allPartsPlot[:,0,3:9,3:9] = partsPlot[20:40] allPartsPlot[:,1:,:,:] = allPartsLayerImg[20:40] gr.images(allPartsPlot.reshape(20 * (numSecondLayerParts + 1),12,12),zero_to_one=False, vmin = 0, vmax =1) elif 1: for i in range(numSecondLayerParts + 1): for j in range(numParts): if i == 0: plotData[5 + j * (2 + secondLayerShape):5+firstLayerShape + j * (2 + secondLayerShape), 5 + i * (2 + secondLayerShape): 5+firstLayerShape + i * (2 + secondLayerShape)] = partsPlot[j+visualShiftParts] else: plotData[2 + j * (2 + secondLayerShape):2 + secondLayerShape+ j * (2 + secondLayerShape),2 + i * (2 + secondLayerShape): 2+ secondLayerShape + i * (2 + secondLayerShape)] = allPartsLayerImg[j+visualShiftParts,i-1] plot.figure(figsize=(10,40)) plot.axis('off') plot.imshow(plotData, **settings) else: pass digits = range(10) sup_ims = [] sup_labels = [] classificationTrainingNum = 1000 for d in digits: ims0 = ag.io.load_mnist('training', [d], selection = slice(classificationTrainingNum), return_labels = False) sup_ims.append(ims0) sup_labels.append(d * np.ones(len(ims0),dtype = np.int64)) sup_ims = np.concatenate(sup_ims, axis = 0) sup_labels = np.concatenate(sup_labels,axis = 0) #thirLevelCurx = np.load('./thirdLevelCurx.npy') thirLevelCurx = np.load('./thirdLevelCurx_LargeMatch.npy')[:5000] poolHelper = pnet.PoolingLayer(shape = (4,4),strides = (4,4)) thirLevelCurx = np.array(thirLevelCurx, dtype = np.int64) pooledExtract = poolHelper.extract((thirLevelCurx[:,:,:,np.newaxis],500)) print(pooledExtract.sum(axis = 3)) print(pooledExtract.shape) sup_labels = sup_labels[:5000] sup_ims = sup_ims[:5000] index = np.arange(5000) randomIndex = np.random.shuffle(index) pooledExtract = pooledExtract.reshape(5000,-1) shuffledExtract = pooledExtract[index] shuffledLabel = sup_labels[index] dataSize = shuffledExtract.shape[1] classifier = sgd_optimization_mnist() weights = classifier.W.get_value(borrow=True) bias = classifier.b.get_value(borrow=True) weights = weights.reshape(4,4,500,10) trainingImg_curX = np.load('./thirdLevelCurx_LargeMatch.npy')[:1000] trainingImg_curX = np.array(trainingImg_curX, dtype = np.int64) pooledTrain = poolHelper.extract((trainingImg_curX[:,:,:,np.newaxis],500)) trainImg,trainLabels = ag.io.load_mnist('training') newPooledExtract = np.array(pooledTrain[:1000]).reshape(1000,4,4,500) for p in range(4): for q in range(4): location1 = newPooledExtract[:,p,q,:] data = weights[p,q,:500,:] X = np.array(data.reshape(500,10),dtype=np.double) kmeans = sklearn.cluster.k_means(np.array(X,dtype = np.double),10)[1] skipIndex = np.argmax(np.bincount(kmeans)) #Put in all the array of group index here groupIndexArray = [[] for m in range(10)] for i in range(10): if i == skipIndex: continue testIndex = i indexArray = np.where(kmeans == testIndex)[0] groupIndexArray[testIndex].append(indexArray) poolingIndex = [[] for m in range(500)] for k in np.where(np.max(location1,axis=0)!=0)[0]: if kmeans[k] == skipIndex: continue else: distanceArray = np.array([np.sum((X[m,:]-X[k,:]) * (X[m,:]-X[k,:])) for m in groupIndexArray[kmeans[k]][0]]) #print(distanceArray.shape) numPooling = (distanceArray.shape[0] + 1)//2 # print(numPooling) finalPooling = groupIndexArray[kmeans[k]][0][np.argsort(distanceArray)[:numPooling]] #print(k, finalPooling) poolingIndex[k].append(finalPooling) for r in range(1000): print(r) for m in range(500): if newPooledExtract[r,p,q,m] == 1: if len(poolingIndex[m])==0: continue else: # print(poolingIndex[m][0]) newPooledExtract[r,p,q,:][poolingIndex[m][0]] = 1 testImg_curX = np.load('./thirdLevelCurx_Test.npy') testImg_curX = np.array(testImg_curX, dtype = np.int64) pooledTest = poolHelper.extract((testImg_curX[:,:,:,np.newaxis],500)) testImg,testLabels = ag.io.load_mnist('testing') newPooledExtractTest = np.array(pooledTest[:10000]).reshape(10000,4,4,500) for p in range(4): for q in range(4): location1 = newPooledExtractTest[:,p,q,:] data = weights[p,q,:500,:] X = np.array(data.reshape(500,10),dtype=np.double) kmeans = sklearn.cluster.k_means(np.array(X,dtype = np.double),10)[1] skipIndex = np.argmax(np.bincount(kmeans)) #Put in all the array of group index here groupIndexArray = [[] for m in range(10)] for i in range(10): if i == skipIndex: continue testIndex = i indexArray = np.where(kmeans == testIndex)[0] groupIndexArray[testIndex].append(indexArray) poolingIndex = [[] for m in range(500)] for k in np.where(np.max(location1,axis=0)!=0)[0]: if kmeans[k] == skipIndex: continue else: distanceArray = np.array([np.sum((X[m,:]-X[k,:]) * (X[m,:]-X[k,:])) for m in groupIndexArray[kmeans[k]][0]]) #print(distanceArray.shape) numPooling = (distanceArray.shape[0] + 1)//2 # print(numPooling) finalPooling = groupIndexArray[kmeans[k]][0][np.argsort(distanceArray)[:numPooling]] #print(k, finalPooling) poolingIndex[k].append(finalPooling) for r in range(10000): print(r) for m in range(500): if newPooledExtractTest[r,p,q,m] == 1: if len(poolingIndex[m])==0: continue else: # print(poolingIndex[m][0]) newPooledExtractTest[r,p,q,:][poolingIndex[m][0]] = 1 #Train a class Model# classificationLayers = [pnet.SVMClassificationLayer(C = 1.0)] classificationNet = pnet.PartsNet(classificationLayers) classificationNet.train(np.array(newPooledExtract,dtype = np.int64), trainLabels[:1000])) print("Training Success!") testImg_Input = np.array(newPooledExtractTest, dtype = np.int64) testImg_batches = np.array_split(testImg_Input, 200) testLabels_batches = np.array_split(testLabels, 200) args = [tup + (classificationNet,) for tup in zip(testImg_batches, testLabels_batches)] corrects = 0 total = 0 def format_error_rate(pr): return "{:.2f}%".format(100 * (1-pr)) print("Testing Starting...") for i, res in enumerate(pnet.parallel.starmap_unordered(test,args)): if i !=0 and i % 20 ==0: print("{0:05}/{1:05} Error rate: {2}".format(total, len(ims),format_error_rate(pr))) corrects += res.sum() total += res.size pr = corrects / total print("Final error rate:", format_error_rate(pr))
def printOutWeight(self): gr.images(1000 * self.components_.reshape(self.n_components,16,16),zero_to_one = False, vmin = -2, vmax = 2)
for j in range(numSecondLayerParts): if(allPartsLayerImgNumber[i,j]): allPartsLayerImg[i,j] = allPartsLayerImg[i,j]/allPartsLayerImgNumber[i,j] """ Visualize the SuperParts """ settings = {'interpolation':'nearest','cmap':plot.cm.gray,} settings['vmin'] = 0 settings['vmax'] = 1 plotData = np.ones(((2 + numSecondLayerParts)*100+2,(2+numSecondLayerParts)*(numSecondLayerParts + 1)+2))*0.8 visualShiftParts = 0 if 0: allPartsPlot = np.zeros((20,numSecondLayerParts + 1,12,12)) gr.images(partsPlot.reshape(numParts,6,6),zero_to_one=False,vmin = 0, vmax = 1) allPartsPlot[:,0] = 0.5 allPartsPlot[:,0,3:9,3:9] = partsPlot[20:40] allPartsPlot[:,1:,:,:] = allPartsLayerImg[20:40] gr.images(allPartsPlot.reshape(20 * (numSecondLayerParts + 1),12,12),zero_to_one=False, vmin = 0, vmax =1) elif 1: for i in range(numSecondLayerParts + 1): for j in range(100): if i == 0: plotData[5 + j * (2 + secondLayerShape):5+firstLayerShape + j * (2 + secondLayerShape), 5 + i * (2 + secondLayerShape): 5+firstLayerShape + i * (2 + secondLayerShape)] = partsPlot[j+visualShiftParts] else: plotData[2 + j * (2 + secondLayerShape):2 + secondLayerShape+ j * (2 + secondLayerShape),2 + i * (2 + secondLayerShape): 2+ secondLayerShape + i * (2 + secondLayerShape)] = allPartsLayerImg[j+visualShiftParts,i-1] plot.figure(figsize=(10,40)) plot.axis('off') plot.imshow(plotData, **settings) plot.savefig('test3.pdf',format='pdf',dpi=900)
ims,labels = ag.io.load_mnist('training') extractedFeature = [] extractedFeature = allLayer[0].extract(ims[0:1000])[0] extractedParts1 = extractedFeature print(extractedParts1.shape) partsPlot1 = np.zeros((numParts1,) + patchShape) partsCodedNumber1 = np.zeros(numParts1) for i in range(1000): codeParts1 = extractedParts1[i].reshape(extractedParts1[i].shape[0:2]) for m in range(28 - patchShape[0] + 1): for n in range(28 - patchShape[0] + 1): if(codeParts1[m,n]!=-1): partsPlot1[codeParts1[m,n]]+=ims[i,m:m+patchShape[0],n:n+patchShape[0]] partsCodedNumber1[codeParts1[m,n]]+=1 for j in range(numParts1): partsPlot1[j] = partsPlot1[j]/partsCodedNumber1[j] #fileString = 'firstLayerPartsNonSequencial.png' fileString = 'firstLayeriRotNonSe.png' gr.images(partsPlot1,zero_to_one=False, show=False,vmin = 0, vmax = 1, fileName = fileString) #print(partsPlot1.shape) #gr.images(partsPlot1[0:200],vmin = 0,vmax = 1) #gr.images(partsPlot2,vmin = 0,vmax = 1) #print(partsCodedNumber1) print("-----------------") #print(partsCodedNumber2) #return partsPlot1,partsPlot2,extractedFeature
if(allPartsLayerImgNumber[i,j]): allPartsLayerImg[i,j] = allPartsLayerImg[i,j]/allPartsLayerImgNumber[i,j] #np.save("exPartsOriginalJun29.npy",allPartsLayer) if 1: """ Visualize the SuperParts """ settings = {'interpolation':'nearest','cmap':plot.cm.gray,} settings['vmin'] = 0 settings['vmax'] = 1 plotData = np.ones(((2 + secondLayerShape)*100+2,(2+secondLayerShape)*(numSecondLayerParts + 1)+2))*0.8 visualShiftParts = 0 if 0: allPartsPlot = np.zeros((20,numSecondLayerParts + 1,12,12)) gr.images(partsPlot.reshape(numParts,6,6),zero_to_one=False,vmin = 0, vmax = 1) allPartsPlot[:,0] = 0.5 allPartsPlot[:,0,3:9,3:9] = partsPlot[20:40] allPartsPlot[:,1:,:,:] = allPartsLayerImg[20:40] gr.images(allPartsPlot.reshape(20 * (numSecondLayerParts + 1),12,12),zero_to_one=False, vmin = 0, vmax =1) elif 1: for i in range(numSecondLayerParts + 1): for j in range(numParts): if i == 0: plotData[5 + j * (2 + secondLayerShape):5+firstLayerShape + j * (2 + secondLayerShape), 5 + i * (2 + secondLayerShape): 5+firstLayerShape + i * (2 + secondLayerShape)] = partsPlot[j+visualShiftParts] else: plotData[2 + j * (2 + secondLayerShape):2 + secondLayerShape+ j * (2 + secondLayerShape),2 + i * (2 + secondLayerShape): 2+ secondLayerShape + i * (2 + secondLayerShape)] = allPartsLayerImg[j+visualShiftParts,i-1] plot.figure(figsize=(10,40)) plot.axis('off') plot.imshow(plotData, **settings) plot.savefig('originalExParts_2.pdf',format='pdf',dpi=900)
secondLayerPermutation[a] = np.arange(num_rot * num_rot * numParts) else: secondLayerPermutation[a] = np.roll(secondLayerPermutation[a - 1], num_rot * num_rot * numParts) from pnet.bernoulli import em superParts = [em(np.array(partsRegion[i]).reshape(len(partsRegion[i]),-1),numSecondLayerParts,permutation=secondLayerPermutation,verbose=True) for i in range(numParts)] allVisParts = [] for i in range(numParts): print(superParts[i][3].shape) comps = np.array(superParts[i][3]) raw_originals = np.array(originalPartsList[i]) print(raw_originals.shape, comps.shape) visParts = np.asarray([raw_originals[comps[:,0]==k,(comps[comps[:,0]==k][:,1]-1)%num_rot].mean(0) for k in range(numParts)]) allVisParts.append(visParts) gr.images(np.array(originalPartsList[0])[:20,0],show=False,zero_to_one=False, vmin=0,vmax=1,fileName = 'test.png') print(superParts[0][3][:20,:]) print(shiftRotationLayer._visparts.shape) """ Visualize the SuperParts """ settings = {'interpolation':'nearest','cmap':plot.cm.gray,} settings['vmin'] = 0 settings['vmax'] = 1 plotData = np.ones(((2 + secondLayerShape)*100+2,(2+secondLayerShape)*(numSecondLayerParts + 1)+2))*0.8 visualShiftParts = 0 if 0: allPartsPlot = np.zeros((20,numSecondLayerParts + 1,12,12)) gr.images(partsPlot.reshape(numParts,6,6),zero_to_one=False,vmin = 0, vmax = 1)