def analyzePixelRelevance(): "simplified implementation of paper: Zintgraf et al - Visualizing Deep Neural Network Decisions: Prediction Difference Analysis" # setup model model = Model(open(Constants.fnCharList).read(), DecoderType.BestPath, mustRestore=True) # read image and specify ground-truth text img = cv2.imread(Constants.fnAnalyze, cv2.IMREAD_GRAYSCALE) (w, h) = img.shape assert Model.imgSize[1] == w # compute probability of gt text in original image batch = Batch([Constants.gtText], [preprocess(img, Model.imgSize)]) (_, probs) = model.inferBatch(batch, calcProbability=True, probabilityOfGT=True) origProb = probs[0] grayValues = [0, 63, 127, 191, 255] if Constants.distribution == 'histogram': bins = [0, 31, 95, 159, 223, 255] (hist, _) = np.histogram(img, bins=bins) pixelProb = hist / sum(hist) elif Constants.distribution == 'uniform': pixelProb = [1.0 / len(grayValues) for _ in grayValues] else: raise Exception('unknown value for Constants.distribution') # iterate over all pixels in image pixelRelevance = np.zeros(img.shape, np.float32) for x in range(w): for y in range(h): # try a subset of possible grayvalues of pixel (x,y) imgsMarginalized = [] for g in grayValues: imgChanged = copy.deepcopy(img) imgChanged[x, y] = g imgsMarginalized.append(preprocess(imgChanged, Model.imgSize)) # put them all into one batch batch = Batch([Constants.gtText] * len(imgsMarginalized), imgsMarginalized) # compute probabilities (_, probs) = model.inferBatch(batch, calcProbability=True, probabilityOfGT=True) # marginalize over pixel value (assume uniform distribution) margProb = sum( [probs[i] * pixelProb[i] for i in range(len(grayValues))]) pixelRelevance[x, y] = weightOfEvidence(origProb, margProb) print(x, y, pixelRelevance[x, y], origProb, margProb) np.save(Constants.fnPixelRelevance, pixelRelevance)
def infer(model, fnImg): "recognize text in image provided by file path" #img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) img = preprocess(fnImg, Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) return probability[0], recognized[0]
def infer(model, fnImg): "recognize text in image provided by file path" img = preprocessImage(fnImg, Model.imgSize, binary=True) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('Recognized:', '"' + recognized[0] + '"') print('Probability:', probability[0])
def infer(model, fnImg): "recognize text in image provided by file path" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('Recognized:', '"' + recognized[0] + '"') print('Probability:', probability[0])
def infer(model, fnImg): "recognize text in image provided by file path" '''img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('Recognized:', '"' + recognized[0] + '"') f = open("wordLists.txt","w",encoding='UTF8') f.write(recognized[0]) f.close() print('Probability:', probability[0]) ''' wordList = [] file_list = os.listdir(fnImg) for item in file_list: print(item) imgName = fnImg + "\\" + item img = preprocess(cv2.imread(imgName,cv2.IMREAD_GRAYSCALE),Model.imgSize) batch = Batch(None,[img]) (recognized,probability) = model.inferBatch(batch, True) print('Recognized:', '"' + recognized[0] + '"') wordList.append(recognized[0]) #f = open("C:\\Users\\yea\\.spyder-py3\\answerSheet\\answerwordLists.txt","w",encoding='utf-8') with codecs.open("/Users/hcy/Desktop/GP/answer/answerwordLists.txt","w",encoding='utf8') as f: for i in range(0, len(wordList)): f.write(wordList[i]+"\n") #f.write(recognized[0]) f.close()
def infer(model, fnImg): # if fnImg.shape[1] > 200 and fnImg.shape[0] > 60: # fnImg1 = cv2.resize(fnImg, (100, 32)) # 100, 32 and 85,32 # elif 150 < fnImg.shape[1] < 200 and fnImg.shape[0] > 30: # fnImg1 = cv2.resize(fnImg, (85, 32)) # else: # fnImg1 = cv2.resize(fnImg, (65, 32)) if fnImg.shape[1] > 200: fnImg1 = imutils.resize(fnImg, width=128) else: fnImg1 = imutils.resize(fnImg, width=100) gray = cv2.cvtColor(fnImg1, cv2.COLOR_BGR2GRAY) # thresh = cv2.adaptiveThreshold(gray,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY,115,10) pxmin = np.min(gray) pxmax = np.max(gray) imgContrast = (gray - pxmin) / (pxmax - pxmin) * 255 # increased line width kernel = np.ones((1, 1), np.uint8) imgMorph = cv2.erode(imgContrast, kernel, iterations=1) img = preprocess(imgMorph, Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) word = '' return recognized[0]
def infer(model, fnImg): #fnImg = fnImg total_segments = open(path + '/segmented words/total_segments.txt','r') file = total_segments.read() file = int(file) count = 0 for i in range(1,file+1): segments = open(path + '/segmented words/' + str(i) + '/segments.txt','r') file1 = segments.read() file1 = int(file1) for j in range(file1): fnImg1 = fnImg + str(i) + '/segment_no_%d' % j + '.png' for image_path in glob.glob(fnImg1 + "\\*.png"): image = imageio.imread(image_path) img = preprocess(image, Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) count+=1 file2 = open(path + '/results/output' + str(i) + '.txt','a') file2.write(recognized[0]) file2.write(" ") print('Recognized:', '"' + recognized[0] + '"') print('Probability:', probability[0]) file3 = open(path + '/results/output' + str(i) + '.txt','a') file3.write('\n')
def analyzeTranslationInvariance(): # setup model model = Model(open(Constants.fnCharList).read(), DecoderType.BestPath, mustRestore=True) # read image and specify ground-truth text img = cv2.imread(Constants.fnAnalyze, cv2.IMREAD_GRAYSCALE) (w, h) = img.shape assert Model.imgSize[1] == w imgList = [] for dy in range(Model.imgSize[0] - h + 1): targetImg = np.ones((Model.imgSize[1], Model.imgSize[0])) * 255 targetImg[:, dy:h + dy] = img imgList.append(preprocess(targetImg, Model.imgSize)) # put images and gt texts into batch batch = Batch([Constants.gtText] * len(imgList), imgList) # compute probabilities (texts, probs) = model.inferBatch(batch, calcProbability=True, probabilityOfGT=True) # save results to file f = open(Constants.fnTranslationInvarianceTexts, 'wb') pickle.dump(texts, f) f.close() np.save(Constants.fnTranslationInvariance, probs)
def testing(): imgdirs = os.listdir('../data/') decoderType = DecoderType.BestPath model = Model(open(FilePaths.fnCharList).read(), decoderType, mustRestore=True) output = '' for (j, dire) in enumerate(imgdirs): imgFiles = os.listdir('../data/' + dire) # print(imgFiles) for (i, f) in enumerate(imgFiles): #fnImg=input("Enter the image path in '../data/_____' format\n") #fnImg=FilePaths.fnInfer fnImg = '../data/' + str(dire) + '/' + f print('fn=' + fnImg) img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img] * Model.batchSize ) # fill all batch elements with same input image recognized = model.inferBatch(batch) # recognize text print('Recognized:', '"' + recognized[0] + '"') output = output + recognized[0] + ' ' output = output + '\n' print(output)
def infer(model, input_image): """ recognize input image """ img = preprocess(cv.imread(input_image, cv.IMREAD_GRAYSCALE), Model.image_size) batch = Batch(None, [img]) recognized = model.infer_batch(batch) print('Recognized:', '"' + recognized[0] + '"')
def infer(filePath): "recognize text in image provided by file path" model = Model(open(fnCharList).read(), mustRestore=True) img = preprocess(cv2.imread(fnInfer, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img] * Model.batchSize) recognized = model.inferBatch(batch) print('Recognized:', '"' + recognized[0] + '"')
def make_batches(folder_name, images_list): # Sort the images_list lexicographically # print(images_list) images_list.sort() # print(images_list) max_images_per_batch = 60 quo = len(images_list) // max_images_per_batch rem = len(images_list) % max_images_per_batch batch_range = range(0, max_images_per_batch) batches_list = [] images_filepaths = [] cnt = 0 while (quo > 0): new_images = [] gtTexts = [None for i in range(max_images_per_batch)] imgs = [ preprocess( cv2.imread(folder_name + "/" + images_list[i], cv2.IMREAD_GRAYSCALE), img_size) for i in range(cnt * max_images_per_batch, (cnt + 1) * max_images_per_batch) ] batches_list.append(Batch(gtTexts, imgs)) for i in range(cnt * max_images_per_batch, (cnt + 1) * max_images_per_batch): new_images.append(folder_name + "/" + images_list[i]) cnt += 1 quo -= 1 images_filepaths.append(new_images) if (quo == 0 and rem > 0): new_images = [] gtTexts = [None for i in range(rem)] imgs = [ preprocess( cv2.imread(folder_name + "/" + images_list[i], cv2.IMREAD_GRAYSCALE), img_size) for i in range(cnt * max_images_per_batch, len(images_list)) ] batches_list.append(Batch(gtTexts, imgs)) for i in range(cnt * max_images_per_batch, len(images_list)): new_images.append(folder_name + "/" + images_list[i]) images_filepaths.append(new_images) return batches_list, images_filepaths
def infer(model, fnImg): """recognize text in image provided by file path""" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) # print('Probability:', '"' + str(round(float(probability[0]) * 100, 2)) + '%"') return recognized[0]
def infer(model, fnImg, sentence_list, img_num): "recognize text in image provided by file path" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch( None, [img] * Model.batchSize) # fill all batch elements with same input image recognized = model.inferBatch(batch) # recognize text sentence_list.append((img_num, recognized[0]))
def infer(model, fnImg): "recognize text in image provided by file path" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img] * Model.batchSize) # fill all batch elements with same input image recognized = model.inferBatch(batch) # recognize text # print('Recognized:', '"' + recognized[0] + '"') # all batch elements hold same result # print(recognized[0]) return recognized[0]
def infer(model, fnImg): """ Recognize text in image provided by file path """ img = preprocessor(fnImg, model.imgSize, binary=True) # Fill all batch elements with same input image batch = Batch(None, [img] * Model.batchSize) recognized = model.inferBatch(batch) # recognize text # All batch elements hold same result print('Recognized:', '"' + recognized[0] + '"')
def infer(model, wordImg): "recognize text in image provided by file path" img = preprocess(wordImg, Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) #print('Recognized:', '"' + recognized[0] + '"') #print('Probability:', probability[0]) return recognized[0]
def infer(model, fnImg): "recognize text in image provided by file path" img = cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE) ret, img = cv2.threshold(img, 155, 255, cv2.THRESH_BINARY) img = preprocess(img, Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('Recognized:', '"' + recognized[0] + '"') print('Probability:', probability[0])
def classify(model, img): print(img.shape) img = cv2.resize(img, Model.imgSize) img = preprocess2(img, Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('Recognized:', '""' + recognized[0] + '""') print('Probability: ', probability[0]) return (probability[0], recognized[0])
def infer(model, fnImg): "recognize text in image provided by file path" input_path = glob.glob(fnImg) for file in input_path: img = preprocess(cv2.imread(file, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('Image: ',str(os.path.basename(file))) print('Recognized:', '"' + recognized[0] + '"') print('Probability:', probability[0])
def predict(): if request.method == 'POST': img = np.array( Image.open(io.BytesIO( request.files['image'].read())).convert('L')) img = preprocess(img, Model.imgSize, True) batch = Batch(None, [img]) recognized, probability = model.inferBatch(batch, True) return 'Prediction: {}'.format(recognized[0]) return 'Please upload an image'
def infer_model(): img = preprocess(cv2.imread('data/test.png', cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) # return jsonify({ 'recognized': str(recognized[0]), 'probability': str(probability[0]) })
def infer(model, fnImg): "recognize text in image provided by file path" #image = cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE) # cv2.cvtColor(fnImg, cv2.COLOR_BGR2GRAY) # image = cv2.resize(image,(500,500)) img = preprocess(fnImg, Model.imgSize) batch = Batch(None, [img] * Model.batchSize) recognized = model.inferBatch(batch) print('Recognized:', '"' + recognized[0] + '"') return recognized[0]
def infer(model, fnImg): "recognize text in image provided by file path" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img] * Model.batchSize) # fill all batch elements with same input image recognized = model.inferBatch(batch) # recognize text print('Recognized:', '"' + recognized[0] + '"') # all batch elements hold same result f = open('../dataset/recognized/recognized.txt', 'a+') f.write(recognized[0]+"\n") f.close() wordImg = cv2.imread('../dataset/test.png')
def infer(model, fnImg): "recognize text in image provided by file path" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img] * Model.batchSize) (recognized, probability) = model.inferBatch(batch, True) words = '../data/test.txt' f = open(words, "w+", encoding="utf-8") f.write(recognized[0]) print('Probability:', probability[0]) return recognized[0]
def inferBatch(model): "recognize text in set of images provided by file path" col_names = [ 'Jacard Similarity', 'Cosine similarity', 'Levenstein Similarity', 'Euclidian distance', 'Character Level Accuracy', 'Actual image', 'Predicted Text' ] df = pd.DataFrame(columns=col_names) f = open(FilePaths.fnTrain + 'validation.txt') for line in f: # ignore comment line if not line or line[0] == '#': continue lineSplit = line.strip().split(' ') assert len(lineSplit) >= 9 # filename: part1-part2-part3 --> part1/part1-part2/part1-part2-part3.png fileNameSplit = lineSplit[0].split('-') fileName = FilePaths.fnTrain + 'validation/' + fileNameSplit[ 0] + '/' + fileNameSplit[0] + '-' + fileNameSplit[ 1] + '/' + lineSplit[0] + '.png' # GT text are columns starting at 9 gtText = DataLoader.truncateLabel('self', ' '.join(lineSplit[8:]), Model.maxTextLen) # check if image is not empty if not os.path.getsize(fileName): #bad_samples.append(lineSplit[0] + '.png') continue # put sample into list #self.samples.append(Sample(gtText, fileName)) img = preprocess(cv2.imread(fileName, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) try: (recognized, probability) = model.inferBatch(batch, True) print('Recognized:', '"' + recognized[0] + '"') print('Probability:', probability[0]) jaccard_score, cosine_score, levens_score, euclidean_score = get_AccuracyMetrics( gtText, recognized[0]) acc = getCharacterLevelAccuracy(gtText, recognized[0]) print('Character Accuracy:', acc) df.loc[len(df)] = [ jaccard_score, cosine_score, levens_score, euclidean_score, acc, gtText, recognized[0] ] except: continue df.to_csv(r'C:\Projects\Handwriting recognition\Validation_IAM_HAN.csv')
def infer(model, fnImg): imgs = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize, True, False) print('Recognized: ') text = "" for img in imgs: batch = Batch(None, [img]) recognized = model.inferBatch(batch, True) text = text + " " + recognized[0] print(text) return text
def infer(model, fnImg): "recognize text in image provided by file path" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('1) Model: Recognized:', '"' + recognized[0] + '"' + ' Probability:', probability[0]) output = dictionary_test.input_word(recognized[0], probability[0], fnImg) return output
def infer(model, fnImg): "recognize text in image provided by file path" img = preprocess(cv2.imread(fnImg, cv2.IMREAD_GRAYSCALE), Model.imgSize) batch = Batch(None, [img]) (recognized, probability) = model.inferBatch(batch, True) print('Recognized:', '"' + recognized[0] + '"') print('Probability:', probability[0]) file = open("E:/project ml/SimpleHTR-master/dataextract.txt", "w") file.write(recognized[0] + "\n") file.close() messagebox.showinfo("Information", "SUCCESSFULLY CONVERTED TO DIGITAL TEXT")
def main(): Infilename = sys.argv[1] img = cv2.imread(Infilename, cv2.IMREAD_GRAYSCALE) # increase contrast pxmin = np.min(img) pxmax = np.max(img) imgContrast = (img - pxmin) / (pxmax - pxmin) * 255 # increase line width kernel = np.ones((3, 3), np.uint8) imgMorph = cv2.erode(imgContrast, kernel, iterations = 1) img = prepareImg(imgMorph, 50) res = wordSegmentation(img, kernelSize=25, sigma=11, theta=7, minArea=100) if os.path.isdir(path): shutil.rmtree(path) os.mkdir(path) for (j, w) in enumerate(res): (wordBox, wordImg) = w cv2.imwrite('../SegWords/%d.png'%j, wordImg) files = [] for filename in sorted(os.listdir(path)): files.append(os.path.join(path,filename)) decoderType = DecoderType.WordBeamSearch #decoderType = DecoderType.BeamSearch #decoderType = DecoderType.BestPath model = Model(open('../model/charList.txt').read(), decoderType, mustRestore=True) imgs = [] for fp in files: imgs.append(preprocess(cv2.imread(fp, cv2.IMREAD_GRAYSCALE), Model.imgSize)) batch = Batch(None, imgs) (recognized, probability) = model.inferBatch(batch, True) model = Model(open('../model/charList.txt').read(), decoderType, mustRestore=True) file1 = open("myfile.txt","w") l='' print('The predicted sentence is : ',end="'") for pw in recognized: l += pw l += ' ' print(pw, end=" ") print("'") l += '\n' file1.write(l) file1.close()