def main(): """main functionf for testing""" from WMCore.DataStructs.Job import Job from WMCore.DataStructs.File import File from WMCore.DataStructs.Run import Run from WMCore.DataStructs.JobPackage import JobPackage from WMCore.Services.UUID import makeUUID from WMCore.WMSpec.Makers.TaskMaker import TaskMaker factory = HarvestingWorkloadFactory() workload = factory("derp", getTestArguments()) task = workload.getTask('Harvesting') job = Job("SampleJob") job["id"] = makeUUID() job["task"] = task.getPathName() job["workflow"] = workload.name() file = File(lfn="/store/relval/CMSSW_3_8_2/RelValMinBias/GEN-SIM-RECO/MC_38Y_V9-v1/0019/FEC5BB4D-BFAF-DF11-A52A-001A92810AD2.root") job.addFile(file) jpackage = JobPackage() jpackage[1] = job import pickle handle = open("%s/JobPackage.pkl" % os.getcwd(), 'w') pickle.dump(jpackage, handle) handle.close() taskMaker = TaskMaker(workload, os.getcwd()) taskMaker.skipSubscription = True taskMaker.processWorkload() task.build(os.getcwd())
def createTestJob(self): """ _createTestJob_ Create a test job that has parents for each input file. """ newJob = Job(name = "TestJob") newJob.addFile(File(lfn = "/some/file/one", parents = set([File(lfn = "/some/parent/one")]))) newJob.addFile(File(lfn = "/some/file/two", parents = set([File(lfn = "/some/parent/two")]))) return newJob
def createTestJob(self): """ Create a test job to pass to the DashboardInterface """ job = Job(name = "ThisIsASillyName") testFileA = File(lfn = "/this/is/a/lfnA", size = 1024, events = 10) testFileA.addRun(Run(1, *[45])) testFileB = File(lfn = "/this/is/a/lfnB", size = 1024, events = 10) testFileB.addRun(Run(1, *[46])) job.addFile(testFileA) job.addFile(testFileB) job['id'] = 1 return job
class JobTest(unittest.TestCase): """ _JobTest_ Testcase for the Job class Instantiate a dummy Job object with a dummy Subscription and a dummy Fileset full of random files as input """ def setUp(self): """ _setUp_ Initial Setup for the Job Testcase """ self.inputFiles = [] for i in range(1, 1000): lfn = "/store/data/%s/%s/file.root" % (random.randint( 1000, 9999), random.randint(1000, 9999)) size = random.randint(1000, 2000) events = 1000 run = random.randint(0, 2000) lumi = random.randint(0, 8) file = File(lfn=lfn, size=size, events=events, checksums={"cksum": "1"}) file.addRun(Run(run, *[lumi])) self.inputFiles.append(file) self.dummyJob = Job(files=self.inputFiles) return def tearDown(self): """ No tearDown method for this Testcase """ pass def testGetFilesList(self): """ _testGetFilesList_ Verify that the Job::getFiles(type = "list") method returns the same files in the same order that they were passed in. """ assert self.dummyJob.getFiles() == self.inputFiles, \ "ERROR: Initial fileset does not match Job fileset" return def testGetFilesSet(self): """ _testGetFilesSet_ Verify that the Job::getFiles(type = "set") method returns the correct input files in the form of a set. """ assert self.dummyJob.getFiles(type = "set") == set(self.inputFiles), \ "ERROR: getFiles(type = 'set') does not work correctly." return def testGetFilesLFN(self): """ _testGetFilesLFN_ Verify that the Job::getFiles(type = "lfn") method returns the same files in the same order that they were passed in. """ jobLFNs = self.dummyJob.getFiles(type="lfn") goldenLFNs = [] for file in self.inputFiles: goldenLFNs.append(file["lfn"]) assert len(goldenLFNs) == len(jobLFNs), \ "ERROR: Job has different number of files than input" for jobLFN in jobLFNs: assert jobLFN in goldenLFNs, \ "ERROR: LFN missing from job." return def testAddFile(self): """ _testAddFile_ Verify that the Job::addFile() method works properly. """ dummyFileAddFile = File("/tmp/dummyFileAddFileTest", 1234, 1, 2) self.dummyJob.addFile(dummyFileAddFile) assert dummyFileAddFile in self.dummyJob.getFiles(), \ "ERROR: Couldn't add file to Job - addFile method error" return def testChangeState(self): """ _testChangeState_ Verify that the Job::changeState() method updates the state and the state time. """ currentTime = time.time() self.dummyJob.changeState("created") assert self.dummyJob["state_time"] > currentTime - 1 and \ self.dummyJob["state_time"] < currentTime + 1, \ "ERROR: State time not updated on state change" assert self.dummyJob["state"] == "created", \ "ERROR: Couldn't change Job state - changeState method error" def testChangeOutcome(self): """ _testChangeOutcome_ Verify that the Job::changeOutcome() method changes the final outcome of the job. """ self.dummyJob.changeOutcome("success") assert self.dummyJob["outcome"] == "success", \ "ERROR: Job outcome failed to update." return def testGetBaggage(self): """ test that setting/accessing the Job Baggage ConfigSection works """ setattr(self.dummyJob.baggage, "baggageContents", {"key": "value"}) try: baggage = self.dummyJob.getBaggage() except Exception, ex: msg = "Error calling Job.getBaggage()\n" msg += str(ex) self.fail(msg) self.failUnless(hasattr(baggage, "baggageContents"))
class JobTest(unittest.TestCase): """ _JobTest_ Testcase for the Job class Instantiate a dummy Job object with a dummy Subscription and a dummy Fileset full of random files as input """ def setUp(self): """ _setUp_ Initial Setup for the Job Testcase """ self.inputFiles = [] for i in range(1,1000): lfn = "/store/data/%s/%s/file.root" % (random.randint(1000, 9999), random.randint(1000, 9999)) size = random.randint(1000, 2000) events = 1000 run = random.randint(0, 2000) lumi = random.randint(0, 8) file = File(lfn = lfn, size = size, events = events, checksums = {"cksum": "1"}) file.addRun(Run(run, *[lumi])) self.inputFiles.append(file) self.dummyJob = Job(files = self.inputFiles) return def tearDown(self): """ No tearDown method for this Testcase """ pass def testGetFilesList(self): """ _testGetFilesList_ Verify that the Job::getFiles(type = "list") method returns the same files in the same order that they were passed in. """ assert self.dummyJob.getFiles() == self.inputFiles, \ "ERROR: Initial fileset does not match Job fileset" return def testGetFilesSet(self): """ _testGetFilesSet_ Verify that the Job::getFiles(type = "set") method returns the correct input files in the form of a set. """ assert self.dummyJob.getFiles(type = "set") == set(self.inputFiles), \ "ERROR: getFiles(type = 'set') does not work correctly." return def testGetFilesLFN(self): """ _testGetFilesLFN_ Verify that the Job::getFiles(type = "lfn") method returns the same files in the same order that they were passed in. """ jobLFNs = self.dummyJob.getFiles(type = "lfn") goldenLFNs = [] for file in self.inputFiles: goldenLFNs.append(file["lfn"]) assert len(goldenLFNs) == len(jobLFNs), \ "ERROR: Job has different number of files than input" for jobLFN in jobLFNs: assert jobLFN in goldenLFNs, \ "ERROR: LFN missing from job." return def testAddFile(self): """ _testAddFile_ Verify that the Job::addFile() method works properly. """ dummyFileAddFile = File("/tmp/dummyFileAddFileTest", 1234, 1, 2) self.dummyJob.addFile(dummyFileAddFile) assert dummyFileAddFile in self.dummyJob.getFiles(), \ "ERROR: Couldn't add file to Job - addFile method error" return def testChangeState(self): """ _testChangeState_ Verify that the Job::changeState() method updates the state and the state time. """ currentTime = time.time() self.dummyJob.changeState("created") assert self.dummyJob["state_time"] > currentTime - 1 and \ self.dummyJob["state_time"] < currentTime + 1, \ "ERROR: State time not updated on state change" assert self.dummyJob["state"] == "created", \ "ERROR: Couldn't change Job state - changeState method error" def testChangeOutcome(self): """ _testChangeOutcome_ Verify that the Job::changeOutcome() method changes the final outcome of the job. """ self.dummyJob.changeOutcome("success") assert self.dummyJob["outcome"] == "success", \ "ERROR: Job outcome failed to update." return def testGetBaggage(self): """ test that setting/accessing the Job Baggage ConfigSection works """ self.dummyJob.addBaggageParameter("baggageContents", True) self.dummyJob.addBaggageParameter("trustPUSitelists", False) self.dummyJob.addBaggageParameter("skipPileupEvents", 20000) baggage = self.dummyJob.getBaggage() self.assertTrue(getattr(baggage, "baggageContents")) self.assertFalse(getattr(baggage, "trustPUSitelists")) self.assertEqual(getattr(baggage, "skipPileupEvents"), 20000) self.assertFalse(hasattr(baggage, "IDontExist"))