Python scripts for the daily tasks in particle physics (Run 2) - Daily Python Scripts (aka DPS)
Quick installation recipe:
# get the code from the repository
git clone https://github.com/BristolTopGroup/DailyPythonScripts
cd DailyPythonScripts
# for Run 1 code, please checkout the run1 branch
# git checkout run1
# get submodules:
git submodule init && git submodule update
# setup run:
source setup_conda.sh
# setup environment (using virtualenv for python):
source environment_conda.sh
# make sure matplotlib is up to date (should return 1.3.1 or above):
python -c 'import matplotlib; print matplotlib.__version__'
If working on soolin (or anywhere where dependencies like ROOT/latex/etc are not available), run it within CMSSW:
# install CMSSW and setup environment:
export SCRAM_ARCH=slc6_amd64_gcc491
scram p -n CMSSW_7_4_7_DPS_ CMSSW_7_4_7
cd CMSSW_7_4_7_DPS/src/
cmsenv
# This version comes with ROOT 6.02/05
# then install DailyPythonScripts according to the recipe above
ROOT ≥ 6.02
matplotlib ≥ 1.3
All plots/histograms provided by this package are based on either toy MC or simulated events from the CMS experiment. Real data is not included at any point.
config/* - files to save presets for available modules
data/* - for input/output of ROOT & JSON files (will be created by some scripts)
examples/* - generic examples for available modules
external/* - external projects (virtualenv, RooUnfold, etc)
plots/* - default output folder for plots (will be created by some scripts)
src/* - specific use of available modules
test/* - unit tests for available modules
tools/* - available modules
- Run
experimental/BLTUnfold/merge_unfolding_jobs.sh
to merge BLT unfolding CRAB output files (each sample needs to be merged into one file, cannot be split over several files) - Move merged files to e.g.:
/hdfs/TopQuarkGroup/mc/7TeV
or8TeV/v11/NoSkimUnfolding/BLT/
- run
produceUnfoldingJobs.py
with finebinning option turned on, on central sample only (run locally on soolin):python produceUnfoldingJobs.py -c=7 -f --sample=central
- Move fine binned unfolding file to
/hdfs/TopQuarkGroup/results/histogramfiles/AN-14-071_6th_draft/7TeV
or8TeV/unfolding/
- Run the
src/cross_section_measurement/00_pick_bins.py
script to find new binning. - Modify
config/variable_binning
(and TTbar_plus_X_analyser.cpp in AnalysisSoftware) with new binning
- Run
python experimental/BLTUnfold/runJobsCrab.py
with the last few lines commented out and uncommenting the lineprint len(jobs)
to print the number of jobs. - Update
queue
inexperimental/BLTUnfold/submitBLTUnfold.description
with the outputted number of jobs cd
up to the folder containing DailyPythonScripts andtar --exclude='external/vpython' --exclude='any other large/unnecessary folders in DailyPythonScripts' -cf dps.tar DailyPythonScripts
(tar file should be <= 100MB)- Run
experimental/BLTUnfold/produceUnfoldingHistogram.py
script on merged files using HTCondor:condor_submit submitBLTUnfold.description
to convert unfolding files to our binning. Check progress usingcondor_q your_username
- Once all jobs have finished, untar output files:
tar -xf *.tar
- Output root files should be in a folder called
unfolding
. Move these new files to/hdfs/TopQuarkGroup/results/histogramfiles/AN-14-071_6th_draft/7TeV
or8TeV/unfolding/
- After running the Analysis Software, move the output files to
/hdfs/TopQuarkGroup/results/histogramfiles/AN-14-071_7th_draft/7TeV
or8TeV
using ```python experimental/move_BAT_output_files_to - Find out how many merging jobs are needed using
python experimental/merge_samples_onDICE.py -c 7(or 8) -n 1 --listJobs
- Modify the following lines in experimental/submitMerge.description:
centre of mass energy:
arguments = $(process) 7
orarguments = $(process) 8
number of jobs: enter the number of merging jobs for the centre of mass energy in question here e.g.queue 65
cd
up to the folder containing DailyPythonScripts andtar --exclude='external/vpython' --exclude='any other large/unnecessary folders in DailyPythonScripts' -cf dps.tar DailyPythonScripts
(tar file should be <= 100MB)- Merge the required BAT output files (e.g. SingleTop, QCD etc.) using
condor_submit DailyPythonScripts/experimental/submitMerge.description
x_01_all_vars
x_02_all_vars
x_03_all_vars
x_04_all_vars
x_05_all_vars
x_98_all_vars
x_99_QCD_cross_checks
x_make_binning_plots
x_make_control_plots
x_make_fit_variable_plots
(script AN-14-071 runs all of these scripts automatically if you are confident everything will run smoothly(!))