Python readFromHDFS Examples

Example #1

File: retrieve_unknowntruth_report.py Project: occ-data/matsu-project

import odg_hsi_analytic_wheel.analytics.utilities.binaryhadoop as binaryhadoop
import sys
import report_unknowntruth
f=binaryhadoop.readFromHDFS('/user/steve/blobspectra-test-Israel/part-00000')
cmrecs = []
for point in f:
  cmrec = point[2].split(',')
  cmrecs.append('%d, %d'%(float(cmrec[0]),float(cmrec[1])))
report_unknowntruth.main(cmrecs, 'SummaryStatistics.json')

Example #2

File: retrieve_confusion_matrix.py Project: occ-data/matsu-project

import odg_hsi_analytic_wheel.analytics.utilities.binaryhadoop as binaryhadoop
import sys
import report
f=binaryhadoop.readFromHDFS('/user/steve/blobspectra-test-Scan_0000/part-00000')
cmrecs = []
for point in f:
  sys.stdout.write('%s\n'%repr(point))
  cmrec = point[2].split(',')
  cmrecs.append('%d, %i, %i, %i, %i, %i'%(float(cmrec[0]),0, int(cmrec[3]), int(cmrec[5]), int(cmrec[4]), int(cmrec[6])))
report.main(cmrecs)