A pipeline prototype for the Cherenkov Telescope Array.

• Code: https://github.com/cta-observatory/protopipe
• Docs: https://cta-observatory.github.io/protopipe/

These instructions are for users only, i.e. you want to use the latest version of the code but do not intend to make any change to it. If your wish is to develop the package then follow the instruction below entitled "Instructions for developpers".

Get the source code and create the required basic conda environment:

      git clone https://github.com/cta-observatory/protopipe.git
      cd protopipe
      conda env create -f environment.yml
      conda activate protopipe

In case you are updating your environment from a previously existing one, named e.g. 'myenv', use:

      conda env update -n myenv -f environment.yml

This environment contains the bare minimum in order to run the scripts and build the documentation.

It doesn't take into account any additional tool you could use later on (it is suggested to install jupyter and vitables, especially if you want to contribute to the code).

Next you need to install protopipe itself (the main folder will be named protopipe for the development version, or protopipe-X.Y.Z for a released tagged version) :

      cd protopipe

To install a released version with no further development, use:

      python setup.py install

To use and develop the latest development version, use:

      python setup.py develop

The second command will let you make changes to your local git repository without the need to update your environment every time.

Remember that the environment needs to be activated in order for protopipe to work. This procedure has been successfully tested on macOS (10.10.5 & 10.14.6) and on Scientific Linux 7.

From the main folder go down to the documentation repository and create the documentation :

      cd docs
      make html

If you are (or plan to be) a developer and you get some warnings when you build the documention, before pushing a Pull Request please try to fix them.

The initial page is stored in _ build/html/index.html, which you can open using your favorite internet browser.

Before starting to use protopipe, be sure to be inside the relevant conda environment (should be named protopipe by default) and try to print the help documentation for one of the scripts, e.g.,

      cd protopipe/scripts
      python write_dl1.py -h

The following instructions refer to local use of protopipe, which doesn't involve the use of the DIRAC computing grid (you can ignore the grid.yaml configuration file). Typical analysis steps are the following :

1. create an analysis parent folder with the auxiliary script create_dir_structure.py
2. prepare the configuration files
   1. copy the example YAML configuration files in the relative subfolders
   2. edit them for the particular needs of your analysis
3. build a model for energy estimation
   1. Create tables for gamma-rays using write_dl1.py and analysis.yaml
   2. Merge them with the auxiliary script merge.sh
   3. create the model with build_model.py and regressor.yaml
   4. check it's performance with model_diagnostic.py and regressor.yaml
4. build a model for gamma/hadron separation
   1. set 'estimate_energy' to True when using write_dl1.py so the reconstructed energy can be estimated and further used as a discriminant parameter.
   2. same process as for the energy estimation but using classifier.yaml
5. produce DL2-level data
   1. create tables for gamma-rays, protons and electrons using write_dl2.py and analysis.yaml
   2. merge them with the auxiliary script merge.sh
6. Estimate the final performance (IRFs) with make_performance.py and performance.yaml

Note: DL1/DL2 scripts take as input only 1 file at the time.

1. Fork the master protopipe remote repository as explained here
2. Follow the installation instructions above, but using your remote repository (we'll call origin yours and upstream the official one)
3. if you want to develop something new:
   1. update local master branch (git pull upstream master)
   2. create a new branch from your local master branch
   3. develop inside it
   4. push it to origin
   5. continue to develop and push until you feel ready
4. start a pull request from origin/your_branch to upstream/master
   1. wait for an outcome
   2. if necessary, you can update or fix things in your branch because now everything is traced (local/your_branch --> origin/your_branch --> pull request)

Note: if your developments take a relatively long time,

1. update periodically your local master branch,
2. if updates have been made, go to your local development branch (git checkout your_branch)
3. if there are no conflicts, move the beginning of your branch at the end of the updated master (git rebase master)
4. push your branch to your remote