def toScreenConfTrg(trgconfdata,iresults=[],isverbose=False): ''' input:{run:[datasource,normbitname,[allbits]]} ''' if isverbose: labels=[('Run','source','bitnames','normbit')] else: labels=[('Run','source','bitnames')] result=[] for r in iresults: result.append(r) for run in sorted(trgconfdata): if trgconfdata[run] is None: ll=[str(run),'n/a','n/a'] if isverbose: ll.append('n/a') result.append(ll) continue source=trgconfdata[run][0] source=source.split('/')[-1] normbit=trgconfdata[run][1] allbits=trgconfdata[run][2] bitnames=', '.join(allbits) if isverbose: result.append([str(run),source,bitnames,normbit]) else: result.append([str(run),source,bitnames]) print ' == = ' print tablePrinter.indent (labels+result, hasHeader = True, separateRows = False, prefix = '| ', postfix = ' |', justify = 'left', delim = ' | ', wrapfunc = lambda x: wrap_onspace_strict(x,60) )
def printRecordedLumi(self,lumidata,isVerbose=False,hltpath=''): datatoprint=[] totalrow=[] labels=[('Run','HLT path','Recorded'+u' (/\u03bcb)'.encode('utf-8'))] lastrowlabels=[('Selected LS','Recorded'+u' (/\u03bcb)'.encode('utf-8'))] if len(hltpath)!=0 and hltpath!='all': lastrowlabels=[('Selected LS','Recorded'+u' (/\u03bcb)'.encode('utf-8'),'Effective '+u'(/\u03bcb) '.encode('utf-8')+hltpath)] if isVerbose: labels=[('Run','HLT-path','L1-bit','L1-presc','HLT-presc','Recorded'+u' (/\u03bcb)'.encode('utf-8'))] totalSelectedLS=0 totalRecorded=0.0 totalRecordedInPath=0.0 for dataperRun in lumidata: runnum=dataperRun[0] if len(dataperRun[1])==0: rowdata=[] rowdata+=[str(runnum)]+2*['N/A'] datatoprint.append(rowdata) continue perlsdata=dataperRun[2] totalSelectedLS=totalSelectedLS+len(perlsdata) recordedLumi=0.0 #norbits=perlsdata.values()[0][3] recordedLumi=self.calculateTotalRecorded(perlsdata) totalRecorded=totalRecorded+recordedLumi trgdict=dataperRun[1] effective=self.calculateEffective(trgdict,recordedLumi) if trgdict.has_key(hltpath) and effective.has_key(hltpath): rowdata=[] l1bit=trgdict[hltpath][0] if len(trgdict[hltpath]) != 3: if not isVerbose: rowdata+=[str(runnum),hltpath,'N/A'] else: rowdata+=[str(runnum),hltpath,l1bit,'N/A','N/A','N/A'] else: if not isVerbose: rowdata+=[str(runnum),hltpath,'%.3f'%(effective[hltpath])] else: hltprescale=trgdict[hltpath][1] l1prescale=trgdict[hltpath][2] rowdata+=[str(runnum),hltpath,l1bit,str(l1prescale),str(hltprescale),'%.3f'%(effective[hltpath])] totalRecordedInPath=totalRecordedInPath+effective[hltpath] datatoprint.append(rowdata) continue for trg,trgdata in trgdict.items(): #print trg,trgdata rowdata=[] if trg==trgdict.keys()[0]: rowdata+=[str(runnum)] else: rowdata+=[''] l1bit=trgdata[0] if len(trgdata)==3: if not isVerbose: rowdata+=[trg,'%.3f'%(effective[trg])] else: hltprescale=trgdata[1] l1prescale=trgdata[2] rowdata+=[trg,l1bit,str(l1prescale),str(hltprescale),'%.3f'%(effective[trg])] else: if not isVerbose: rowdata+=[trg,'N/A'] else: rowdata+=[trg,l1bit,'N/A','N/A','%.3f'%(effective[trg])] datatoprint.append(rowdata) #print datatoprint print '===' print tablePrinter.indent(labels+datatoprint,hasHeader=True,separateRows=False,prefix='| ',postfix=' |',justify='right',delim=' | ',wrapfunc=lambda x: wrap_onspace_strict(x,22)) if len(hltpath)!=0 and hltpath!='all': totalrow.append([str(totalSelectedLS),'%.3f'%(totalRecorded),'%.3f'%(totalRecordedInPath)]) else: totalrow.append([str(totalSelectedLS),'%.3f'%(totalRecorded)]) print '=== Total : ' print tablePrinter.indent(lastrowlabels+totalrow,hasHeader=True,separateRows=False,prefix='| ',postfix=' |',justify='right',delim=' | ',wrapfunc=lambda x: wrap_onspace(x,20)) if isVerbose: deadtoprint=[] deadtimelabels=[('Run','Lumi section : Dead fraction')] for dataperRun in lumidata: runnum=dataperRun[0] if len(dataperRun[1])==0: deadtoprint.append([str(runnum),'N/A']) continue perlsdata=dataperRun[2] #print 'perlsdata 2 : ',perlsdata deadT=self.getDeadfractions(perlsdata) t='' for myls,de in deadT.items(): if de<0: t+=str(myls)+':nobeam ' else: t+=str(myls)+':'+'%.5f'%(de)+' ' deadtoprint.append([str(runnum),t]) print '===' print tablePrinter.indent(deadtimelabels+deadtoprint,hasHeader=True,separateRows=True,prefix='| ',postfix=' |',justify='right',delim=' | ',wrapfunc=lambda x: wrap_onspace(x,80))
def printPerLSLumi(self,lumidata,isVerbose=False,hltpath=''): ''' input lumidata [['runnumber','trgtable{}','deadtable{}']] deadtable {lsnum:[deadtime,instlumi,bit_0,norbits]} ''' datatoprint=[] totalrow=[] labels=[('Run','LS','Delivered','Recorded'+u' (/\u03bcb)'.encode('utf-8'))] lastrowlabels=[('Selected LS','Delivered'+u' (/\u03bcb)'.encode('utf-8'),'Recorded'+u' (/\u03bcb)'.encode('utf-8'))] totalDeliveredLS=0 totalSelectedLS=0 totalDelivered=0.0 totalRecorded=0.0 for perrundata in lumidata: runnumber=perrundata[0] deadtable=perrundata[2] lumiresult=self.lsBylsLumi(deadtable) totalSelectedLS=totalSelectedLS+len(deadtable) for lsnum,dataperls in lumiresult.items(): rowdata=[] if len(dataperls)==0: rowdata+=[str(runnumber),str(lsnum),'N/A','N/A'] else: rowdata+=[str(runnumber),str(lsnum),'%.3f'%(dataperls[0]),'%.3f'%(dataperls[1])] totalDelivered=totalDelivered+dataperls[0] totalRecorded=totalRecorded+dataperls[1] datatoprint.append(rowdata) totalrow.append([str(totalSelectedLS),'%.3f'%(totalDelivered),'%.3f'%(totalRecorded)]) print '===' print tablePrinter.indent(labels+datatoprint,hasHeader=True,separateRows=False,prefix='| ',postfix=' |',justify='right',delim=' | ',wrapfunc=lambda x: wrap_onspace_strict(x,22)) print '=== Total : ' print tablePrinter.indent(lastrowlabels+totalrow,hasHeader=True,separateRows=False,prefix='| ',postfix=' |',justify='right',delim=' | ',wrapfunc=lambda x: wrap_onspace(x,20))
def toScreenTotEffective(lumidata,resultlines,scalefactor,isverbose): ''' input: {run:[lumilsnum(0),triggeredls(1),timestamp(2),beamstatus(3),beamenergy(4),deliveredlumi(5),recordedlumi(6),calibratedlumierror(7),{hltpath:[l1name,l1prescale,hltprescale,efflumi]},bxdata,beamdata](8)} screen Run,SelectedLS,Recorded,HLTPath,L1Bit,Effective ''' result=[]#[run,selectedlsStr,recorded,hltpath,l1bit,efflumi] totdict={}#{hltpath:[nls,toteff]} selectedcmsls=[] alltotrecorded=0.0 alleffective=0.0 totOldSelectedLS=0 totOldRecorded=0.0 #totOldEffective=0.0 for rline in resultlines: myls=rline[1] mypath=rline[3] if mypath!='n/a': mypath=mypath.split('(')[0] if not totdict.has_key(mypath): totdict[mypath]=[0,0.0] if myls!='n/a': listcomp=myls.split(', ') for lstr in listcomp: enddigs=lstr[1:-1].split('-') lsmin=int(enddigs[0]) lsmax=int(enddigs[1]) rls=lsmax-lsmin+1 totOldSelectedLS+=rls totdict[mypath][0]+=rls myrecorded=rline[2] if myrecorded!='n/a': totOldRecorded+=float(myrecorded) (rr,lumiu)=CommonUtil.guessUnit(float(myrecorded)) rline[2]='%.3f'%(rr)+' ('+lumiu+')' myeff=rline[5] if myeff!='n/a': reff=float(myeff) (rr,lumiu)=CommonUtil.guessUnit(float(reff)) rline[5]='%.3f'%(rr)+' ('+lumiu+')' totdict[mypath][1]+=reff result.append(rline) for run in sorted(lumidata):#loop over runs hprescdict={} lprescdict={} rundata=lumidata[run] if rundata is None: result.append([str(run),'n/a','n/a','n/a','n/a','n/a']) continue selectedcmsls=[x[1] for x in rundata if x[1]!=0] totefflumiDict={} totrecorded=0.0 toteffective=0.0 pathmap={}#{hltpathname:1lname} for lsdata in rundata: cmslsnum=lsdata[1] efflumiDict=lsdata[8]# this ls has no such path? if not efflumiDict: if cmslsnum in selectedcmsls: selectedcmsls.remove(cmslsnum) continue for hltpathname,pathdata in efflumiDict.items(): if not totefflumiDict.has_key(hltpathname): totefflumiDict[hltpathname]=0.0 pathmap[hltpathname]='n/a' l1name=pathdata[0] l1presc=pathdata[1] hltpresc=pathdata[2] lumival=pathdata[3] if not totdict.has_key(hltpathname): totdict[hltpathname]=[0,0.0] if l1presc is None or hltpresc is None:#if found all null prescales and if it is in the selectedcmsls, remove it because incomplete if cmslsnum in selectedcmsls: selectedcmsls.remove(cmslsnum) else: recordedlumi=lsdata[6] totrecorded+=recordedlumi alltotrecorded+=recordedlumi if not hprescdict.has_key(hltpathname): hprescdict[hltpathname]=[] hprescdict[hltpathname].append(hltpresc) if not lprescdict.has_key(l1name): lprescdict[l1name]=[] lprescdict[l1name].append(l1presc) totdict[hltpathname][0]+=1 if lumival: totdict[hltpathname][1]+=lumival totefflumiDict[hltpathname]+=lumival pathmap[hltpathname]=l1name if len(selectedcmsls)==0: selectedlsStr='n/a' else: selectedlsStr = CommonUtil.splitlistToRangeString(selectedcmsls) for name in sorted(totefflumiDict): lname=pathmap[name] if lname=='n/a': continue (efflumival,efflumiunit)=CommonUtil.guessUnit(totefflumiDict[name]) (totrecval,totrecunit)=CommonUtil.guessUnit(totrecorded) hprescs=list(set(hprescdict[hltpathname])) lprescs=list(set(lprescdict[lname])) hprescStr='('+','.join(['%d'%(x) for x in hprescs])+')' lprescStr='('+','.join(['%d'%(x) for x in lprescs])+')' #print 'efflumival , efflumiunit ',efflumival,efflumiunit result.append([str(run),selectedlsStr,'%.3f'%(totrecval*scalefactor)+'('+totrecunit+')',name+hprescStr,lname+lprescStr,'%.3f'%(efflumival*scalefactor)+'('+efflumiunit+')']) labels = [('Run','SelectedLS','Recorded','HLTpath','L1bit','Effective')] print ' == = ' print tablePrinter.indent (labels+result, hasHeader = True, separateRows = False, prefix = '| ', postfix = ' |', justify = 'right', delim = ' | ', wrapfunc = lambda x: wrap_onspace_strict(x,22) ) print ' == = Total : ' lastrowlabels=[('HLTPath','SelectedLS','Recorded','Effective')] totresult=[] (alltotrecval,alltotrecunit)=CommonUtil.guessUnit(alltotrecorded+totOldRecorded) for hname in sorted(totdict): hdata=totdict[hname] totnls=hdata[0] (toteffval,toteffunit)=CommonUtil.guessUnit(hdata[1]) totresult.append([hname,str(totnls),'%.3f'%(alltotrecval*scalefactor)+'('+alltotrecunit+')','%.3f'%(toteffval*scalefactor)+'('+toteffunit+')']) print tablePrinter.indent (lastrowlabels+totresult, hasHeader = True, separateRows = False, prefix = '| ', postfix = ' |', justify = 'right', delim = ' | ', wrapfunc = lambda x: wrap_onspace (x,20) )
def toScreenLSEffective(lumidata,resultlines,scalefactor,isverbose): ''' input: {run:[lumilsnum(0),cmslsnum(1),timestamp(2),beamstatus(3),beamenergy(4),deliveredlumi(5),recordedlumi(6),calibratedlumierror(7),{hltpath:[l1name,l1prescale,hltprescale,efflumi]},bxdata,beamdata]} ''' result=[]#[run,ls,hltpath,l1bitname,hltpresc,l1presc,efflumi] totalrow=[] totSelectedLS=0 totRecorded=0.0 recordedlumiunit='/ub' totEffective=0 efflumiunit='/ub' totOldSelectedLS=0 totOldRecorded=0.0 totOldEffective=0.0 for rline in resultlines: myls=rline[1] if myls!='n/a': totOldSelectedLS+=1 myrecorded=rline[6] if myrecorded!='n/a': totOldRecorded+=float(myrecorded) rline[6]='%.2f'%float(myrecorded) myeff=rline[7] if myeff!='n/a': totOldEffective+=float(myeff) rline[7]='%.2f'%float(myeff) result.append(rline) totrecordedlumi=0.0 totefflumi=0.0 for run in sorted(lumidata):#loop over runs rundata=lumidata[run] if rundata is None: result.append([str(run),'n/a','n/a','n/a','n/a','n/a','n/a','n/a']) continue for lsdata in rundata: efflumiDict=lsdata[8]# this ls has no such path? if not efflumiDict: continue cmslsnum=lsdata[1] recorded=lsdata[6] totSelectedLS+=1 if not recorded: recorded=0.0 totRecorded+=recorded for hltpathname in sorted(efflumiDict): pathdata=efflumiDict[hltpathname] l1name=pathdata[0] if l1name is None: l1name='n/a' else: l1name=l1name.replace('"','') l1prescale=pathdata[1] hltprescale=pathdata[2] lumival=pathdata[3] if lumival is not None: result.append([str(run),str(cmslsnum),hltpathname,l1name,str(hltprescale),str(l1prescale),'%.2f'%(recorded*scalefactor),'%.2f'%(lumival*scalefactor)]) totEffective+=lumival else: result.append([str(run),str(cmslsnum),hltpathname,l1name,str(hltprescale),str(l1prescale),'%.2f'%(recorded*scalefactor),'n/a']) (totrecordedlumi,recordedlumiunit)=CommonUtil.guessUnit(totRecorded+totOldRecorded) (totefflumi,efflumiunit)=CommonUtil.guessUnit(totEffective+totOldEffective) labels = [('Run','LS','HLTpath','L1bit','HLTpresc','L1presc','Recorded(/ub)','Effective(/ub)')] print ' == = ' print tablePrinter.indent (labels+result, hasHeader = True, separateRows = False, prefix = '| ', postfix = ' |', justify = 'right', delim = ' | ', wrapfunc = lambda x: wrap_onspace_strict(x,25) ) totalrow.append([str(totSelectedLS+totOldSelectedLS),'%.3f'%(totrecordedlumi*scalefactor),'%.3f'%(totefflumi*scalefactor)]) lastrowlabels = [ ('Selected LS','Recorded('+recordedlumiunit+')','Effective('+efflumiunit+')')] print ' == = Total : ' print tablePrinter.indent (lastrowlabels+totalrow, hasHeader = True, separateRows = False, prefix = '| ', postfix = ' |', justify = 'right', delim = ' | ', wrapfunc = lambda x: wrap_onspace (x, 20))
def toScreenLumiByLS(lumidata,resultlines,scalefactor,isverbose): ''' input: lumidata {run:[lumilsnum,cmslsnum,timestamp,beamstatus,beamenergy,deliveredlumi,recordedlumi,calibratedlumierror,(bxidx,bxvalues,bxerrs),(bxidx,b1intensities,b2intensities)]} resultlines [[resultrow1],[resultrow2],...,] existing result row ''' result=[] labels = [ ('Run','LS','UTCTime','Beam Status','E(GeV)','Delivered(/ub)','Recorded(/ub)')] totalrow = [] totalDeliveredLS = 0 totalSelectedLS = 0 totalDelivered = 0.0 totalRecorded = 0.0 totOldDeliveredLS = 0 totOldSelectedLS = 0 totOldDelivered = 0.0 totOldRecorded = 0.0 for rline in resultlines: myls=rline[1] if myls!='n/a': [luls,cmls]=myls.split(':') totOldDeliveredLS+=1 if cmls!='0': totOldSelectedLS+=1 dl=0.0 if rline[5]!='n/a': dl=float(rline[5])#delivered in /ub rline[5]='%.2f'%(dl) totOldDelivered+=dl if rline[6]!='n/a': rl=float(rline[6])#recorded in /ub rline[6]='%.2f'%(rl) totOldRecorded+=rl result.append(rline) for run in lumidata.keys(): rundata=lumidata[run] if rundata is None: result.append([str(run),'n/a','n/a','n/a','n/a','n/a']) continue for lsdata in rundata: lumilsnum=lsdata[0] cmslsnum=lsdata[1] ts=lsdata[2] bs=lsdata[3] begev=lsdata[4] deliveredlumi=lsdata[5] recordedlumi=lsdata[6] result.append([str(run),str(lumilsnum)+':'+str(cmslsnum),ts.strftime('%m/%d/%y %H:%M:%S'),bs,'%.1f'%begev,'%.2f'%(deliveredlumi*scalefactor),'%.2f'%(recordedlumi*scalefactor)]) totalDelivered+=deliveredlumi totalRecorded+=recordedlumi totalDeliveredLS+=1 if(cmslsnum!=0): totalSelectedLS+=1 totdeliveredlumi=0.0 deliveredlumiunit='/ub' #if (totalDelivered+totOldDelivered)!=0: (totdeliveredlumi,deliveredlumiunit)=CommonUtil.guessUnit(totalDelivered+totOldDelivered) totrecordedlumi=0.0 recordedlumiunit='/ub' #if (totalRecorded+totOldRecorded)!=0: (totrecordedlumi,recordedlumiunit)=CommonUtil.guessUnit(totalRecorded+totOldRecorded) lastrowlabels = [ ('Delivered LS','Selected LS', 'Delivered('+deliveredlumiunit+')', 'Recorded('+recordedlumiunit+')')] totalrow.append ([str(totalDeliveredLS+totOldDeliveredLS),str(totalSelectedLS+totOldSelectedLS),'%.3f'%(totdeliveredlumi*scalefactor),'%.3f'%(totrecordedlumi*scalefactor)]) sortedresult=sorted(result,key=lambda x : int(x[0])) print ' == = ' print tablePrinter.indent (labels+sortedresult, hasHeader = True, separateRows = False, prefix = '| ', postfix = ' |', justify = 'right', delim = ' | ', wrapfunc = lambda x: wrap_onspace_strict (x, 22)) print ' == = Total : ' print tablePrinter.indent (lastrowlabels+totalrow, hasHeader = True, separateRows = False, prefix = '| ', postfix = ' |', justify = 'right', delim = ' | ', wrapfunc = lambda x: wrap_onspace (x, 20))