User Guide

• Columnflow Structure
  • General introduction
  • Tasks in columnflow
  • Important note on required parameters
  • Important modules and configs
    • Analysis, Campaign and Config
• v0.2 → v0.3 Transition
  • Restructured Task Array Functions
    • Example
    • Update Instructions
  • Multi-config Tasks
    • Update Instructions
  • Reducers
    • Example
    • Update Instructions
  • Histogram Producers
    • Example
    • Update Instructions
  • Inference Model Updates
    • Example
    • Update Instructions
  • Changed Plotting Task Names
    • Update Instructions
  • Miscellaneous smaller updates
• Task Array Functions
  • Introduction
  • TAF Types
    • Calibrator
    • Selector
    • Reducer
    • Producer
    • Histogram producer
  • Simple Example
  • TAF Interface
  • Full example
• Building Blocks
  • Calibrations
  • Selections
    • Introduction
    • Create an instance of the Selector class
    • Running the SelectEvents task
  • Event and object reduction
    • Introduction
    • Usage
  • Production of columns
    • Introduction
    • Usage
    • ProduceColumns task
  • Categories
    • Create a category
    • Creation of nested categories
    • Extend your categorization as part of a Producer
    • Helper functions for categories
    • Key points (TL; DR)
  • Histogram Producers
    • Introduction
    • Usage
  • Config Objects
    • Generalities
    • Parameters from the order package (required)
    • Auxiliary Parameters (optional)
• Sandboxes and Their Use in Columnflow
  • What needs to be defined
  • Where to Store the Corresponding Files
  • The Setup File
  • The requirement file
  • How to Use the Namespace
• Machine Learning
  • How training happens in Columnflow: K-fold cross validation
  • Configure your custom machine learning class:
  • After the configuration
  • ABC functions
    • sandbox:
    • datasets:
    • produces:
    • uses:
    • output:
    • open_model:
    • train:
    • evaluate:
  • Commands to start training and evaluation
  • Optional useful functions
    • Separate training and evaluation configuration for configs, calibrators, selector and producers
    • setup
  • Adjusting the ML model settings via the command line
• Systematic Uncertainties
  • Uncertainties that only modify the yield
  • Weight-based uncertainties
  • Uncertainties that modify the selection efficiency
  • Uncertainties with dedicated datasets
• Plotting
  • Creating your first plot
  • Customization of plots
  • Creating 2D plots
  • Creating cutflow plots
  • Creating plots for different shifts
  • Directly displaying plots in the terminal
  • Using your own plotting function
  • Applying a selection to a variable
• Creating your own task
  • Writing new tasks
    • Example: Task Class
    • The “super” class
    • The sandbox
    • The parameters
    • The requirements
    • RunOnceTask
    • Add the task to the analysis
    • Executing the Task
  • Custom law.cfg file for users without grid certificate (CMS specific)
• Debugging
  • Debugging outputs
    • Debugging outputs of supported extensions (ROOT, Parquet, JSON and Pickle)
    • Debugging histograms
  • FAQ
    • Troubleshooting:
• Best practices
  • Selecting output locations
  • Controlling versions of upstream tasks
    • Task family specific parameters
    • Pinned versions in the analysis config or law.cfg file
  • Dynamic variation of used columns using DeferredColumn
  • Columnflow convenience tools
  • General advices
  • Using python scripts removed from the standard workflow
• Law introduction
  • Tasks and parameters
  • Law Config
  • Running remote
• Special Usecases
  • Mechanism to hook into histograms (e.g. for ABCD methods)
    • Hist hooks
  • Setting default Calibrator, Selector, Producer, Ml-Model (CSPM) via function
• CMS specializations
  • Tasks
    • CreatePileUpWeights