columnflow.inference

`columnflow.inference`#

Basic objects for defining statistical inference models.

Classes:

`ParameterType`(value)	Parameter type flag.
`ParameterTransformation`(value)	Flags denoting transformations to be applied on parameters.
`ParameterTransformations`(transformations)	Container around a sequence of `ParameterTransformation`'s with a few convenience methods.
`FlowStrategy`(value)	Strategy to handle over- and underflow bin contents.
`InferenceModel`(config_insts)	Interface to statistical inference models with connections to config objects (such as py:class:order.Config or `order.Dataset`).

Functions:

inference_model([func, bases])

Decorator for creating a new InferenceModel subclass with additional, optional bases and attaching the decorated function to it as init_func.

class ParameterType(value)[source]#

Bases: Enum

Parameter type flag.

Variables:

rate_gauss – Gaussian rate parameter.
rate_uniform – Uniform rate parameter.
rate_unconstrained – Unconstrained rate parameter.
shape – Shape parameter.

Attributes:

`is_rate`	Checks if the parameter type is a rate type.
`is_shape`	Checks if the parameter type is a shape type.

property is_rate: bool#

Checks if the parameter type is a rate type.

Returns:: True if the parameter type is a rate type, False otherwise.

property is_shape: bool#

Checks if the parameter type is a shape type.

Returns:: True if the parameter type is a shape type, False otherwise.

class ParameterTransformation(value)[source]#

Bases: Enum

Flags denoting transformations to be applied on parameters.

Variables:

none – No transformation.
centralize – Centralize the parameter.
symmetrize – Symmetrize the parameter.
asymmetrize – Asymmetrize the parameter.
asymmetrize_if_large – Asymmetrize the parameter if it is large.
normalize – Normalize the parameter.
effect_from_shape – Derive effect from shape.
effect_from_rate – Derive effect from rate.

Attributes:

`from_shape`	Checks if the transformation is derived from shape.
`from_rate`	Checks if the transformation is derived from rate.

property from_shape: bool#

Checks if the transformation is derived from shape.

Returns:: True if the transformation is derived from shape, False otherwise.

property from_rate: bool#

Checks if the transformation is derived from rate.

Returns:: True if the transformation is derived from rate, False otherwise.

class ParameterTransformations(transformations: Sequence[ParameterTransformation | str])[source]#

Bases: tuple

Container around a sequence of ParameterTransformation’s with a few convenience methods.

Parameters:: transformations – A sequence of ParameterTransformation or their string names.

Attributes:

`any_from_shape`	Checks if any transformation is derived from shape.
`any_from_rate`	Checks if any transformation is derived from rate.

property any_from_shape: bool#

Checks if any transformation is derived from shape.

Returns:: True if any transformation is derived from shape, False otherwise.

property any_from_rate: bool#

Checks if any transformation is derived from rate.

Returns:: True if any transformation is derived from rate, False otherwise.

class FlowStrategy(value)[source]#

Bases: Enum

Strategy to handle over- and underflow bin contents.

Variables:

ignore – Ignore over- and underflow bins.
warn – Issue a warning when over- and underflow bins are encountered.
remove – Remove over- and underflow bins.
move – Move over- and underflow bins to the first and last bin, respectively.

class InferenceModel(config_insts)[source]#

Bases: Derivable

Interface to statistical inference models with connections to config objects (such as py:class:order.Config or order.Dataset).

The internal structure to describe a model looks as follows (in yaml style) and is accessible through model as well as property access to its top-level objects.

categories:
  - name: cat1
    postfix: null
    config_data:
      22pre_v14:
        category: 1e
        variable: ht
        data_datasets: [data_mu_a]
    data_from_processes: []
    mc_stats: 10
    flow_strategy: warn
    rate_precision: 5
    empty_bin_value: 1e-05
    processes:
      - name: HH
        is_signal: True
        config_data:
          22pre_v14:
            process: hh
            mc_datasets: [hh_ggf]
        scale: 1.0
        is_dynamic: False
        parameters:
          - name: lumi
            type: rate_gauss
            effect: 1.02
            effect_precision: 4
            config_data: {}
            transformations: []
          - name: pu
            type: rate_gauss
            effect: [0.97, 1.02]
            effect_precision: 4
            config_data: {}
            transformations: []
          - name: pileup
            type: shape
            effect: 1.0
            effect_precision: 4
            config_data:
              22pre_v14:
                shift_source: minbias_xs
            transformations: []
      - name: tt
        is_signal: False
        config_data:
          22pre_v14:
            process: tt
            mc_datasets: [tt_sl, tt_dl, tt_fh]
        scale: 1.0
        is_dynamic: False
        parameters:
          - name: lumi
            type: rate_gauss
            effect: 1.02
            effect_precision: 4
            config_data: {}
            transformations: []

  - name: cat2
    ...

parameter_groups:
  - name: rates
    parameters_names: [lumi, pu]
  - ...

name#

type: str

The unique name of this model.

config_insts#

type: list[order.Config]

Reference to order.Config objects.

model#

type: DotDict

The internal data structure representing the model, see InferenceModel.model_spec().

Classes:

YamlDumper(*args, **kwargs)

YAML dumper for statistical inference models with ammended representers to serialize internal, structured objects as safe, standard objects.

Methods:

`inference_model`([func, bases])	Decorator for creating a new `InferenceModel` subclass with additional, optional bases and attaching the decorated function to it as `init_func`.
`used_datasets`(config_inst)	Used datasets for which the upstream_task_cls.resolve_instances will be called.
`model_spec`()	Returns a dictionary representing the top-level structure of the model.
`category_spec`(name[, config_data, ...])	Returns a dictionary representing a category (interchangeably called bin or channel in other tools), forwarding all arguments.
`process_spec`(name[, is_signal, config_data, ...])	Returns a dictionary representing a process, forwarding all arguments.
`parameter_spec`(name, type[, ...])	Returns a dictionary representing a (nuisance) parameter, forwarding all arguments.
`parameter_group_spec`(name[, parameter_names])	Returns a dictionary representing a group of parameter names.
`category_config_spec`([category, variable, ...])	Returns a dictionary representing configuration specific data, forwarding all arguments.
`process_config_spec`([process, mc_datasets])	Returns a dictionary representing configuration specific data, forwarding all arguments.
`parameter_config_spec`([shift_source])	Returns a dictionary representing configuration specific data, forwarding all arguments.
`require_shapes_for_parameter`(param_obj)	Function to check if for a certain parameter object param_obj varied shapes are needed.
`to_yaml`([stream])	Writes the content of the `model` into a file-like object stream when given, and returns a string representation otherwise.
`pprint`()	Pretty-prints the content of the `model` in yaml-style.
`get_categories`([category, only_names, ...])	Returns a list of categories whose name match category.
`get_category`(category[, only_name, ...])	Returns a single category whose name matches category.
`has_category`(category[, match_mode])	Returns True if a category whose name matches category is existing, and False otherwise.
`add_category`(args, *kwargs)	Adds a new category with all args and kwargs used to create the structured category dictionary via `category_spec()`.
`remove_category`(category[, match_mode])	Removes one or more categories whose names match category.
`get_processes`([process, category, ...])	Returns a dictionary of processes whose names match process, mapped to the name of the category they belong to.
`get_process`(process[, category, only_name, ...])	Returns a single process whose name matches process, and optionally, whose category's name matches category.
`has_process`(process[, category, match_mode, ...])	Returns True if a process whose name matches process, and optionally whose category's name matches category, exists, and False otherwise.
`add_process`(*args[, category, ...])	Adds a new process to all categories whose names match category, with all args and kwargs used to create the structured process dictionary via `process_spec()`.
`remove_process`(process[, category, ...])	Removes one or more processes whose names match process, and optionally whose category's name matches category.
`get_parameters`([parameter, process, ...])	Returns a dictionary of parameters whose names match parameter, mapped twice to the name of the category and the name of the process they belong to.
`get_parameter`(parameter[, process, ...])	Returns a single parameter whose name matches parameter, and optionally, whose category's and process' name matches category and process.
`has_parameter`(parameter[, process, ...])	Returns True if a parameter whose name matches parameter, and optionally whose category's and process' name match category and process, exists, and False otherwise.
`add_parameter`(*args[, process, category, ...])	Adds a parameter to all categories and processes whose names match category and process, with all args and kwargs used to create the structured parameter dictionary via `parameter_spec()`.
`remove_parameter`(parameter[, process, ...])	Removes one or more parameters whose names match parameter, and optionally whose category's and process' name match category and process.
`get_parameter_groups`([group, match_mode, ...])	Returns a list of parameter groups whose names match group.
`get_parameter_group`(group[, match_mode, ...])	Returns a single parameter group whose name matches group.
`has_parameter_group`(group[, match_mode])	Returns True if a parameter group whose name matches group exists, and False otherwise.
`add_parameter_group`(args, *kwargs)	Adds a new parameter group with all args and kwargs used to create the structured parameter group dictionary via `parameter_group_spec()`.
`remove_parameter_group`(group[, match_mode])	Removes one or more parameter groups whose names match group.
`add_parameter_to_group`(parameter, group[, ...])	Adds a parameter named parameter to one or multiple parameter groups whose names match group.
`remove_parameter_from_groups`(parameter[, ...])	Removes all parameters matching parameter from parameter groups whose names match group.
`get_categories_with_process`(process[, ...])	Returns a flat list of category names that contain processes matching process.
`get_processes_with_parameter`(parameter[, ...])	Returns a dictionary of names of processes that contain a parameter whose names match parameter, mapped to category names.
`get_categories_with_parameter`(parameter[, ...])	Returns a dictionary of category names mapping to process names that contain parameters whose names match parameter.
`get_groups_with_parameter`(parameter[, ...])	Returns a list of names of parameter groups that contain a parameter whose name matches parameter.
`cleanup`([keep_parameters])	Cleans the internal model structure by removing empty and dangling objects by calling `remove_empty_categories()`, `remove_dangling_parameters_from_groups()` (receiving keep_parameters), and `remove_empty_parameter_groups()` in that order.
`remove_empty_categories`()	Removes all categories that contain no processes.
`remove_dangling_parameters_from_groups`([...])	Removes names of parameters from parameter groups that are not assigned to any process in any category.
`remove_empty_parameter_groups`()	Removes parameter groups that contain no parameter names.
`iter_processes`([process, category, ...])	Generator that iteratively yields all processes whose names match process, optionally in all categories whose names match category.
`iter_parameters`([parameter, process, ...])	Generator that iteratively yields all parameters whose names match parameter, optionally in all processes and categories whose names match process and category.
`scale_process`(scale[, process, category, ...])	Sets the scale attribute of all processes whose names match process, optionally in all categories whose names match category, to scale.

class YamlDumper(*args, **kwargs)[source]#

Bases: SafeDumper

YAML dumper for statistical inference models with ammended representers to serialize internal, structured objects as safe, standard objects.

classmethod inference_model(func=None, bases=(), **kwargs)[source]#

Decorator for creating a new InferenceModel subclass with additional, optional bases and attaching the decorated function to it as init_func. All additional kwargs are added as class members of the new subclass.

Parameters:

func (Callable | None, default: None) – The function to be decorated and attached as init_func.
bases (tuple[type], default: ()) – Optional tuple of base classes for the new subclass.

Return type:

DerivableMeta | Callable

Returns:

The new subclass or a decorator function.

classmethod used_datasets(config_inst)[source]#

Used datasets for which the upstream_task_cls.resolve_instances will be called. Defaults to the default dataset.

Return type:: list[str]

classmethod model_spec()[source]#

Returns a dictionary representing the top-level structure of the model. :rtype: DotDict

categories: List of category_spec() objects.

parameter_groups: List of paramter_group_spec() objects.

classmethod category_spec(name, config_data=None, data_from_processes=None, flow_strategy=FlowStrategy.warn, mc_stats=None, postfix=None, empty_bin_value=1e-05, rate_precision=5)[source]#

Returns a dictionary representing a category (interchangeably called bin or channel in other tools), forwarding all arguments.

Parameters:

name (str) – The name of the category in the model.
config_data (dict[str, DotDict] | None, default: None) – Dictionary mapping names of order.Config objects to dictionaries following the category_config_spec() that wrap settings like category, variable and real datasets in that config.
data_from_processes (Sequence[str] | None, default: None) – Optional list of names of process_spec() objects that, when config_data_datasets is not defined, make up a fake data contribution.
flow_strategy (FlowStrategy | str, default: <FlowStrategy.warn: 'warn'>) – A FlowStrategy instance describing the strategy to handle over- and underflow bin contents.
mc_stats (int | float | tuple | None, default: None) – Either None to disable MC stat uncertainties, or an integer, a float or a tuple thereof to control the options of MC stat options.
postfix (str | None, default: None) – A postfix that is appended to (e.g.) file names created for this model.
empty_bin_value (float, default: 1e-05) – When bins have no content, they are filled with this value.
rate_precision (int, default: 5) – The precision of reported rates.

Return type:

DotDict

Returns:

A dictionary representing the category.

classmethod process_spec(name, is_signal=False, config_data=None, scale=1.0, is_dynamic=False)[source]#

Returns a dictionary representing a process, forwarding all arguments.

Parameters:

name (str) – The name of the process in the model.
is_signal (bool, default: False) – A boolean flag deciding whether this process describes signal.
config_data (dict[str, DotDict] | None, default: None) – Dictionary mapping names of order.Config objects to dictionaries following the process_config_spec() that wrap settings like process and mc datasets in that config.
scale (float | int, default: 1.0) – A float value to scale the process, defaulting to 1.0.
is_dynamic (bool, default: False) – A boolean flag deciding whether this process is dynamic, i.e., whether it is created on-the-fly.

Return type:

DotDict

Returns:

A dictionary representing the process.

classmethod parameter_spec(name, type, transformations=(), config_data=None, effect=1.0, effect_precision=4)[source]#

Returns a dictionary representing a (nuisance) parameter, forwarding all arguments.

Parameters:

name (str) – The name of the parameter in the model.
type (ParameterType | str) – A ParameterType instance describing the type of this parameter.
transformations (Sequence[ParameterTransformation | str], default: ()) – A sequence of ParameterTransformation instances describing transformations to be applied to the effect of this parameter.
config_data (dict[str, DotDict] | None, default: None) – Dictionary mapping names of order.Config objects to dictionaries following the parameter_config_spec() that wrap settings like corresponding shift source in that config.
effect (Any | None, default: 1.0) – An arbitrary object describing the effect of the parameter (e.g. float for symmetric rate effects, 2-tuple for down/up variation, etc).
effect_precision (int, default: 4) – The precision of reported effects.

Return type:

DotDict

Returns:

A dictionary representing the parameter.

classmethod parameter_group_spec(name, parameter_names=None)[source]#

Returns a dictionary representing a group of parameter names.

Parameters:

name (str) – The name of the parameter group in the model.
parameter_names (Sequence[str] | None, default: None) – Names of parameter objects this group contains.

Return type:

DotDict

Returns:

A dictionary representing the group of parameter names.

classmethod category_config_spec(category=None, variable=None, data_datasets=None)[source]#

Returns a dictionary representing configuration specific data, forwarding all arguments.

Parameters:

category (str | None, default: None) – The name of the source category in the config to use.
variable (str | None, default: None) – The name of the variable in the config to use.
data_datasets (Sequence[str] | None, default: None) – List of names or patterns of datasets in the config to use for real data.

Return type:

DotDict

Returns:

A dictionary representing category specific settings.

classmethod process_config_spec(process=None, mc_datasets=None)[source]#

Returns a dictionary representing configuration specific data, forwarding all arguments.

Parameters:

process (str | None, default: None) – The name of the process in the config to use.
mc_datasets (Sequence[str] | None, default: None) – List of names or patterns of datasets in the config to use for mc.

Return type:

DotDict

Returns:

A dictionary representing process specific settings.

classmethod parameter_config_spec(shift_source=None)[source]#

Returns a dictionary representing configuration specific data, forwarding all arguments.

Parameters:: shift_source (str | None, default: None) – The name of a systematic shift source in the config.
Return type:: DotDict
Returns:: A dictionary representing parameter specific settings.

classmethod require_shapes_for_parameter(param_obj)[source]#

Function to check if for a certain parameter object param_obj varied shapes are needed.

Parameters:: param_obj (dict) – The parameter object to check.
Return type:: bool
Returns:: True if varied shapes are needed, False otherwise.

to_yaml(stream=None)[source]#

Writes the content of the model into a file-like object stream when given, and returns a string representation otherwise.

Parameters:: stream (TextIO | None, default: None) – A file-like object to write the model content into.
Return type:: str | None
Returns:: A string representation of the model content if stream is not provided.

pprint()[source]#

Pretty-prints the content of the model in yaml-style.

Return type:: None

get_categories(category=None, only_names=False, match_mode=<built-in function any>)[source]#

Returns a list of categories whose name match category. category can be a string, a pattern, or sequence of them. When only_names is True, only names of categories are returned rather than structured dictionaries.

Parameters:

category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
only_names (bool, default: False) – A boolean flag to return only names of categories if set to True.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).

Return type:

list[DotDict | str]

Returns:

A list of matching categories or their names.

get_category(category, only_name=False, match_mode=<built-in function any>, silent=False)[source]#

Returns a single category whose name matches category. category can be a string, a pattern, or sequence of them. An exception is raised if no or more than one category is found, unless silent is True in which case None is returned. When only_name is True, only the name of the category is returned rather than a structured dictionary.

Parameters:

category (str | Sequence[str]) – A string, pattern, or sequence of them to match category names.
only_name (bool, default: False) – A boolean flag to return only the name of the category if set to True.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
silent (bool, default: False) – A boolean flag to return None instead of raising an exception if no or more than one category is found.

Return type:

DotDict | str

Returns:

A single matching category or its name.

has_category(category, match_mode=<built-in function any>)[source]#

Returns True if a category whose name matches category is existing, and False otherwise. category can be a string, a pattern, or sequence of them.

Parameters:

category (str | Sequence[str]) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if a matching category exists, False otherwise.

add_category(*args, **kwargs)[source]#

Adds a new category with all args and kwargs used to create the structured category dictionary via category_spec(). If a category with the same name already exists, an exception is raised.

Raises:: ValueError – If a category with the same name already exists.
Return type:: None

remove_category(category, match_mode=<built-in function any>)[source]#

Removes one or more categories whose names match category.

Parameters:

category (str | Sequence[str]) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if at least one category was removed, False otherwise.

get_processes(process=None, category=None, only_names=False, flat=False, match_mode=<built-in function any>, category_match_mode=<built-in function any>)[source]#

Returns a dictionary of processes whose names match process, mapped to the name of the category they belong to. Categories can optionally be filtered through category. Both process and category can be a string, a pattern, or sequence of them.

When only_names is True, only names of processes are returned rather than structured dictionaries. When flat is True, a flat, unique list of process names is returned.

Parameters:

process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to filter categories.
only_names (bool, default: False) – A boolean flag to return only names of processes if set to True.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
flat (bool, default: False) – A boolean flag to return a flat, unique list of process names if set to True.

Return type:

dict[str, DotDict | str] | list[str]

Returns:

A dictionary of processes mapped to the category name, or a list of process names.

get_process(process, category=None, only_name=False, match_mode=<built-in function any>, category_match_mode=<built-in function any>, silent=False)[source]#

Returns a single process whose name matches process, and optionally, whose category’s name matches category. Both process and category can be a string, a pattern, or sequence of them.

An exception is raised if no or more than one process is found, unless silent is True in which case None is returned. When only_name is True, only the name of the process is returned rather than a structured dictionary.

Parameters:

process (str | Sequence[str]) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
only_name (bool, default: False) – A boolean flag to return only the name of the process if set to True.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
silent (bool, default: False) – A boolean flag to return None instead of raising an exception if no or more than one process is found.

Return type:

DotDict | str

Returns:

A single matching process or its name.

Raises:

ValueError – If no process or more than one process is found and silent is False.

has_process(process, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>)[source]#

Returns True if a process whose name matches process, and optionally whose category’s name matches category, exists, and False otherwise. Both process and category can be a string, a pattern, or sequence of them.

Parameters:

process (str | Sequence[str]) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if a matching process exists, False otherwise.

add_process(*args, category=None, category_match_mode=<built-in function any>, silent=False, **kwargs)[source]#

Adds a new process to all categories whose names match category, with all args and kwargs used to create the structured process dictionary via process_spec(). category can be a string, a pattern, or sequence of them.

If a process with the same name already exists in one of the categories, an exception is raised unless silent is True.

Parameters:

args – Positional arguments used to create the process.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
silent (bool, default: False) – A boolean flag to suppress exceptions if a process with the same name already exists.
kwargs – Keyword arguments used to create the process.

Raises:

ValueError – If a process with the same name already exists in one of the categories and silent is False.

Return type:

None

remove_process(process, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>)[source]#

Removes one or more processes whose names match process, and optionally whose category’s name matches category. Both process and category can be a string, a pattern, or sequence of them. Returns True if at least one process was removed, and False otherwise.

Parameters:

process (str | Sequence[str]) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if at least one process was removed, False otherwise.

get_parameters(parameter=None, process=None, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>, process_match_mode=<built-in function any>, only_names=False, flat=False)[source]#

Returns a dictionary of parameters whose names match parameter, mapped twice to the name of the category and the name of the process they belong to. Categories and processes can optionally be filtered through category and process. All three, parameter, process and category can be a string, a pattern, or sequence of them.

When only_names is True, only names of parameters are returned rather than structured dictionaries. When flat is True, a flat, unique list of parameter names is returned.

Parameters:

parameter (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match parameter names.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
process_match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
only_names (bool, default: False) – A boolean flag to return only names of parameters if set to True.
flat (bool, default: False) – A boolean flag to return a flat, unique list of parameter names if set to True.

Return type:

dict[str, dict[str, DotDict | str]] | list[str]

Returns:

A dictionary of parameters mapped to category and process names, or a list of parameter names.

get_parameter(parameter, process=None, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>, process_match_mode=<built-in function any>, only_name=False, silent=False)[source]#

Returns a single parameter whose name matches parameter, and optionally, whose category’s and process’ name matches category and process. All three, parameter, process and category can be a string, a pattern, or sequence of them.

An exception is raised if no or more than one parameter is found, unless silent is True in which case None is returned. When only_name is True, only the name of the parameter is returned rather than a structured dictionary.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
process_match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
only_name (bool, default: False) – A boolean flag to return only the name of the parameter if set to True.
silent (bool, default: False) – A boolean flag to return None instead of raising an exception if no or more than one parameter is found.

Return type:

DotDict | str

Returns:

A single matching parameter or its name.

Raises:

ValueError – If no parameter or more than one parameter is found and silent is False.

has_parameter(parameter, process=None, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>, process_match_mode=<built-in function any>)[source]#

Returns True if a parameter whose name matches parameter, and optionally whose category’s and process’ name match category and process, exists, and False otherwise. All three, parameter, process and category can be a string, a pattern, or sequence of them.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
process_match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if a matching parameter exists, False otherwise.

add_parameter(*args, process=None, category=None, category_match_mode=<built-in function any>, process_match_mode=<built-in function any>, group=None, **kwargs)[source]#

Adds a parameter to all categories and processes whose names match category and process, with all args and kwargs used to create the structured parameter dictionary via parameter_spec(). Both process and category can be a string, a pattern, or sequence of them.

If a parameter with the same name already exists in one of the processes throughout the categories, an exception is raised.

When group is given and the parameter is new, it is added to one or more parameter groups via add_parameter_to_group().

Parameters:

args – Positional arguments used to create the parameter.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
process_match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
group (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to specify parameter groups.
kwargs – Keyword arguments used to create the parameter.

Return type:

DotDict

Returns:

The created parameter.

Raises:

ValueError – If a parameter with the same name already exists in one of the processes throughout the categories.

remove_parameter(parameter, process=None, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>, process_match_mode=<built-in function any>)[source]#

Removes one or more parameters whose names match parameter, and optionally whose category’s and process’ name match category and process. All three, parameter, process and category can be a string, a pattern, or sequence of them.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
process_match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if at least one parameter was removed, False otherwise.

get_parameter_groups(group=None, match_mode=<built-in function any>, only_names=False)[source]#

Returns a list of parameter groups whose names match group. group can be a string, a pattern, or sequence of them.

When only_names is True, only names of parameter groups are returned rather than structured dictionaries.

Parameters:

group (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match group names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter group matching behavior (see pattern_matcher()).
only_names (bool, default: False) – A boolean flag to return only names of parameter groups if set to True.

Return type:

list[DotDict | str]

Returns:

A list of parameter groups or their names.

get_parameter_group(group, match_mode=<built-in function any>, only_name=False)[source]#

Returns a single parameter group whose name matches group. group can be a string, a pattern, or sequence of them.

An exception is raised in case no or more than one parameter group is found. When only_name is True, only the name of the parameter group is returned rather than a structured dictionary.

Parameters:

group (str | Sequence[str]) – A string, pattern, or sequence of them to match group names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter group matching behavior (see pattern_matcher()).
only_name (bool, default: False) – A boolean flag to return only the name of the parameter group if set to True.

Return type:

DotDict | str

Returns:

A single matching parameter group or its name.

Raises:

ValueError – If no parameter group or more than one parameter group is found.

has_parameter_group(group, match_mode=<built-in function any>)[source]#

Returns True if a parameter group whose name matches group exists, and False otherwise. group can be a string, a pattern, or sequence of them.

Parameters:

group (str | Sequence[str]) – A string, pattern, or sequence of them to match group names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter group matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if a matching parameter group exists, False otherwise.

add_parameter_group(*args, **kwargs)[source]#

Adds a new parameter group with all args and kwargs used to create the structured parameter group dictionary via parameter_group_spec(). If a group with the same name already exists, an exception is raised.

Parameters:

args – Positional arguments used to create the parameter group.
kwargs – Keyword arguments used to create the parameter group.

Raises:

ValueError – If a parameter group with the same name already exists.

Return type:

None

remove_parameter_group(group, match_mode=<built-in function any>)[source]#

Removes one or more parameter groups whose names match group. group can be a string, a pattern, or sequence of them. Returns True if at least one group was removed, and False otherwise.

Parameters:

group (str | Sequence[str]) – A string, pattern, or sequence of them to match group names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter group matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if at least one group was removed, False otherwise.

add_parameter_to_group(parameter, group, match_mode=<built-in function any>, parameter_match_mode=<built-in function any>)[source]#

Adds a parameter named parameter to one or multiple parameter groups whose names match group. group can be a string, a pattern, or sequence of them. When parameter is a pattern or regular expression, all previously added, matching parameters are added. Otherwise, parameter is added as is. If a parameter was added to at least one group, True is returned and False otherwise.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
group (str | Sequence[str]) – A string, pattern, or sequence of them to match group names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter group matching behavior (see pattern_matcher()).
parameter_match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if at least one parameter was added to a group, False otherwise.

remove_parameter_from_groups(parameter, group=None, match_mode=<built-in function any>, parameter_match_mode=<built-in function any>)[source]#

Removes all parameters matching parameter from parameter groups whose names match group. Both parameter and group can be a string, a pattern, or sequence of them. Returns True if at least one parameter was removed, and False otherwise.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
group (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match group names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter group matching behavior (see pattern_matcher()).
parameter_match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if at least one parameter was removed, False otherwise.

get_categories_with_process(process, match_mode=<built-in function any>)[source]#

Returns a flat list of category names that contain processes matching process. process can be a string, a pattern, or sequence of them.

Parameters:

process (str | Sequence[str]) – A string, pattern, or sequence of them to match process names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).

Return type:

list[str]

Returns:

A list of category names containing matching processes.

get_processes_with_parameter(parameter, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>, flat=True)[source]#

Returns a dictionary of names of processes that contain a parameter whose names match parameter, mapped to category names. Categories can optionally be filtered through category. Both parameter and category can be a string, a pattern, or sequence of them.

When flat is True, a flat, unique list of process names is returned.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
flat (bool, default: True) – A boolean flag to return a flat, unique list of process names if set to True.

Return type:

list[str] | dict[str, list[str]]

Returns:

A dictionary of process names mapped to category names, or a flat list of process names.

get_categories_with_parameter(parameter, process=None, match_mode=<built-in function any>, process_match_mode=<built-in function any>, flat=True)[source]#

Returns a dictionary of category names mapping to process names that contain parameters whose names match parameter. Processes can optionally be filtered through process. Both parameter and process can be a string, a pattern, or sequence of them.

When flat is True, a flat, unique list of category names is returned.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).
process_match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
flat (bool, default: True) – A boolean flag to return a flat, unique list of category names if set to True.

Return type:

list[str] | dict[str, list[str]]

Returns:

A dictionary of category names mapped to process names, or a flat list of category names.

get_groups_with_parameter(parameter, match_mode=<built-in function any>)[source]#

Returns a list of names of parameter groups that contain a parameter whose name matches parameter. parameter can be a string, a pattern, or sequence of them.

Parameters:

parameter (str | Sequence[str]) – A string, pattern, or sequence of them to match parameter names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).

Return type:

list[str]

Returns:

A list of names of parameter groups containing the matching parameter.

cleanup(keep_parameters=None)[source]#

Cleans the internal model structure by removing empty and dangling objects by calling remove_empty_categories(), remove_dangling_parameters_from_groups() (receiving keep_parameters), and remove_empty_parameter_groups() in that order.

Parameters:: keep_parameters (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to specify parameters to keep.
Return type:: None

remove_empty_categories()[source]#

Removes all categories that contain no processes.

Return type:: None

remove_dangling_parameters_from_groups(keep_parameters=None, match_mode=<built-in function any>)[source]#

Removes names of parameters from parameter groups that are not assigned to any process in any category.

Parameters:

keep_parameters (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to specify parameters to keep.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).

Return type:

None

remove_empty_parameter_groups()[source]#

Removes parameter groups that contain no parameter names.

Return type:: None

iter_processes(process=None, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>)[source]#

Generator that iteratively yields all processes whose names match process, optionally in all categories whose names match category. The yielded value is a 2-tuple containing the category name and the process object.

Parameters:

process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).

Return type:

Generator[tuple[DotDict, DotDict], None, None]

Returns:

A generator yielding 2-tuples of category name and process object.

iter_parameters(parameter=None, process=None, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>, process_match_mode=<built-in function any>)[source]#

Generator that iteratively yields all parameters whose names match parameter, optionally in all processes and categories whose names match process and category. The yielded value is a 3-tuple containing the category name, the process name, and the parameter object.

Parameters:

parameter (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match parameter names.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the parameter matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).
process_match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).

Return type:

Generator[tuple[DotDict, DotDict, DotDict], None, None]

Returns:

A generator yielding 3-tuples of category name, process name, and parameter object.

scale_process(scale, process=None, category=None, match_mode=<built-in function any>, category_match_mode=<built-in function any>)[source]#

Sets the scale attribute of all processes whose names match process, optionally in all categories whose names match category, to scale.

Parameters:

scale (int | float) – The scale value to set for the matching processes.
process (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match process names.
category (str | Sequence[str] | None, default: None) – A string, pattern, or sequence of them to match category names.
match_mode (Callable, default: <built-in function any>) – Either any or all to control the process matching behavior (see pattern_matcher()).
category_match_mode (Callable, default: <built-in function any>) – Either any or all to control the category matching behavior (see pattern_matcher()).

Return type:

bool

Returns:

True if at least one process was found and scaled, False otherwise.

inference_model(func=None, bases=(), **kwargs)#

Decorator for creating a new InferenceModel subclass with additional, optional bases and attaching the decorated function to it as init_func. All additional kwargs are added as class members of the new subclass.

Parameters:

func (Callable | None, default: None) – The function to be decorated and attached as init_func.
bases (tuple[type], default: ()) – Optional tuple of base classes for the new subclass.

Return type:

DerivableMeta | Callable

Returns:

The new subclass or a decorator function.

columnflow.inference

Contents

columnflow.inference#

`columnflow.inference`#