declare the outputs; the process will still be executed correctly.
however, this strategy is trying to improve the readability of the code.
a person just take your code may be not familiar with the flow. the declared
outputs will be listed in the generated document and help the folk to catch
the key concepts of the job.
"""
"""
same as previous tutorial
"""
# ==
j = deepcopy(j_temp)
j.id = "hello Serious"
j.desc = "say hello to Serious"
j.need_input("name", "Serious")
wrapper.add_sub_job(j)
# ==
j = deepcopy(j_temp)
j.id = "hello Kidding"
j.desc = "say hello to Kidding"
j.need_input("name", "Kidding")
wrapper.add_sub_job(j)
"""
and we add the two avatars.
they receive the output of the first job, msg_to_[name], as its input
and response individually
"""
j = JobNode(id="Serious", desc="I'm Serious")
j.need_input("msg_to_Serious")
j.set_callback(Serious_avatar)
wrapper = JobBlock(
'entry job', '''
this job demonstrate how to use delegatees, say DFS or Pig
''')
wrapper.add_plan(Job.INIT_JOB, Job.START, 'hadoop delegatee')
wrapper.add_plan('hadoop delegatee', Job.DONE, 'wrong command')
wrapper.add_plan('wrong command', Job.DONE, Job.LAST_JOB)
'''
prepare the jobs
'''
j = JobNode(id='hadoop delegatee',
desc='''
cat some file on the dfs (to run this tutorial, you have to prepare
your own data on the dfs)
''')
j.set_callback(delegated_job)
wrapper.add_sub_job(j)
# ==
j = JobNode(id='wrong command',
desc='''
execute some error command
''')
j.set_callback(failed_delegated_job)
wrapper.add_sub_job(j)
'''
run this tutorial on the Hadoop system
'''
# ==
job_id, state = wrapper.execute()
#raw_input()
wrapper = JobBlock('entry job', '''
this job demonstrate how to organize your plan by JobBlocks
''')
wrapper.add_plan(Job.INIT_JOB, Job.START, 'block1')
wrapper.add_plan('block1', Job.DONE, Job.LAST_JOB)
'''
first, with the top-down design strategy, we define JobBlock, which is like
wrapper with its own plan.
'''
# ==
j = JobBlock(id='block1', desc='block1')
j.add_plan(Job.INIT_JOB, Job.START, 'job0')
j.add_plan('job0', Job.DONE, 'job1')
j.add_plan('job1', Job.DONE, Job.LAST_JOB)
wrapper.add_sub_job(j)
'''
then, we define the inner JobNodes (same as previous tutorial)
'''
# ==
j_sub = JobNode(id='job0',desc='desc0')
j_sub.set_callback(normal_job)
j.add_sub_job(j_sub)
# ==
j_sub = JobNode(id='job1',desc='desc1')
j_sub.set_callback(normal_job)
j.add_sub_job(j_sub)
# ==
'''
