def printinfo(): fnamelist = ["../10"+str(v)+".csv" for v in xrange(22,23)] cnt = 0 found = 0 wholeresult = {} # writefile = open("../cleandata","w") missloc = {} for fname in fnamelist: filereader = FileReader(fname) alarmcode = filereader.getattridx("ALARMCODE") attridx = filereader.getattridx("SUMMARY") locidx= filereader.getattridx("LOCATION") timeidx = filereader.getattridx("ALARMHAPPENTIME") cntidx = 0 while True: tmptran = filereader.readtransection() cntidx += 1 # print cntidx if tmptran is None: filereader.close() break summary = tmptran[attridx] location = tmptran[locidx] if location.startswith("SU6095-SU2551"): print "SUMMARY:" print summary print "--------------" print "LOCATION" print location
def printidentifier(): filereader = FileReader("../1022.csv") identifieridx = filereader.getattridx("NEIDENTIFIER") while True: tmptran = filereader.readtransection() identifier = tmptran[identifieridx] if tmptran is None: filereader.close() break print identifier raw_input()
class FileReaderTestCase(unittest.TestCase): lines = ('1st line\n', '2nd line\n', '3rd line\n') def setUp(self): self.tmpfn = tempfile.mktemp("filereadertest") fp = open(self.tmpfn, 'w') for line in self.lines: fp.write(line) fp.close() self.f = FileReader(self.tmpfn) def tearDown(self): self.f.close() os.unlink(self.tmpfn)
def testwrongfile(): filereader = FileReader("../wrongdocfile") alarmcodeidx = filereader.getattridx("ALARMCODE") attridx = filereader.getattridx("SUMMARY") locidx = filereader.getattridx("LOCATION") timeidx = filereader.getattridx("ALARMHAPPENTIME") print "idxdata:", timeidx, alarmcodeidx, locidx, attridx print filereader.m_header print filereader.m_headerlen while True: tmptran = filereader.readtransection() if tmptran is None: filereader.close() break summary = tmptran[attridx] location = tmptran[locidx] alarmcode = tmptran[alarmcodeidx] timestr = tmptran[timeidx] print tmptran raw_input()
def testfound(self): # fnamelist = ["../wrongdocfile",] # fnamelist = ["../10"+str(v)+".csv" for v in xrange(22,23)] # fnamelist = ["../1125.csv",] fnamelist = ["../10"+str(v)+".csv" for v in xrange(22,32)] + \ ["../110"+str(v)+".csv" for v in xrange(01,10)] + \ ["../11"+str(v)+".csv" for v in xrange(10,31)] + \ ["../120"+str(v)+".csv" for v in xrange(01,10)] + \ ["../12"+str(v)+".csv" for v in xrange(10,23)] cnt = 0 found = 0 wholeresult = {} writeflag = True if writeflag: writefile = open("../cleandata", "w") wrongdocflag = False if wrongdocflag: wrongdocfile = open("../wrongdocfile", "w") wrongdocfile.write(",".join([ '"ALARMHAPPENTIME"', '"ALARMCODE"', '"LOCATION"', '"SUMMARY"' ]) + "\n") missloc = {} print "||||||||||||||||" fnamecnt = open("fnamecnt", "w") for fname in fnamelist: print fname filereader = FileReader(fname) alarmcodeidx = filereader.getattridx("ALARMCODE") attridx = filereader.getattridx("SUMMARY") locidx = filereader.getattridx("LOCATION") timeidx = filereader.getattridx("ALARMHAPPENTIME") identifieridx = filereader.getattridx("NEIDENTIFIER") cntidx = 0 while True: tmptran = filereader.readtransection() cntidx += 1 # print cntidx if tmptran is None: filereader.close() break summary = tmptran[attridx] location = tmptran[locidx] alarmcode = tmptran[alarmcodeidx] identifier = tmptran[identifieridx] warn = Warning(summary, location) if warn.m_type == NOTP4: continue ftword = warn.getfirstword() if ftword not in wholeresult: wholeresult[ftword] = {"cnt": 0, "good": 0} wholeresult[ftword]["cnt"] += 1 cnt += 1 loc = self.m_topo.getnebyidentifier(identifier) if loc is None: loc = warn.fetchloc(self.m_topo) if loc is None: locstr = warn.fetchlocstr() if warn.m_type != NOTP5 and warn.m_type != TP9: if locstr not in missloc: missloc[locstr] = 0 print "===============================================" print warn.m_summary print "----------------------------------" print warn.m_location print "----------------------------------" print identifier print "locstr:", warn.m_type, locstr missloc[locstr] += 1 if wrongdocflag: wrongdocfile.write(",".join(['\"'+v+'\"' for v in \ [tmptran[timeidx],tmptran[alarmcodeidx],tmptran[locidx],tmptran[attridx],]])+"\r\n") continue wholeresult[ftword]["good"] += 1 found += 1 summary = summary.replace("\n", "_") if writeflag: writefile.write(alarmcode + "\t" + loc.m_name + "\t" + summary + "\t" + tmptran[timeidx] + "\n") print fname, "\t", cntidx fnamecnt.write(fname + "\t" + str(cntidx) + "\n") fnamecnt.close() if writeflag: writefile.close() if wrongdocflag: wrongdocfile.close() print "result:" print "cnt:", cnt print "found:", found print "pcg:", found * 1.0 / cnt for v in wholeresult.keys(): if wholeresult[v]["good"] == wholeresult[v]["cnt"]: del wholeresult[v] else: wholeresult[v]["pcg"] = wholeresult[v][ "good"] * 1.0 / wholeresult[v]["cnt"] import pprint pprint.pprint(wholeresult) print "-----------------------" pprint.pprint(missloc) print "missloclen:", len(missloc) json.dump(wholeresult, open("tmpwholeresult", "w")) json.dump(missloc, open("missloc", "w"))