columnflow.util
#
Collection of general helpers and utilities.
Summary#
Placeholder for an unset value. |
|
Boolean denoting whether the environment is in a remote job (based on |
|
Boolean denoting whether the environment is used for development (based on |
|
List of the first 100 primes. |
|
|
Calls importlib.import_module internally and returns the module if it exists, or otherwise a |
Lazily imports and configures matplotlib pyplot. |
|
Lazily imports and configures ROOT. |
|
|
Loads the content of a python file located at path and returns its package content as a dictionary. |
Returns a random string based on UUID v4. |
|
|
Takes path fragments, joins them and recursively expands all contained environment variables. |
|
Takes path fragments and returns the real, absolute location with all variables expanded. |
|
Ensures that a directory at path (and its subdirectories) exists and returns the full, expanded path. |
|
Downloads a file from a remote src to a local destination dst, creating intermediate directories when needed. |
|
Execute a function func in a thread and aborts the call when timeout is reached. |
|
Execute a function func in a process and aborts the call when timeout is reached. |
|
Law task decorator that checks whether either a voms or arc proxy is existing before calling the decorated method. |
|
Takes a sandbox key sandbox and adds or removes the substring "_dev" right before the file extension (if any), depending on whether the current environment is used for development (see |
|
Returns a divided by b if b is not zero, and zero otherwise. |
|
Tests whether a value f can be converted to a float. |
|
Returns True if a string s contains pattern characters such as "*" or "?", and False otherwise. |
|
Returns True if a string s is a regular expression starting with "^" and ending with "$", and False otherwise. |
|
Takes a string pattern which might be an actual pattern for fnmatching, a regular expressions or just a plain string and returns a function that can be used to test of a string matches that pattern. |
|
Adds key-value pair to dictionary, but only if it does not change an existing value; Raises KeyError otherwise. |
|
Returns the source code of any object obj as a string. |
Subclass of OrderedDict that provides read and write access to items via attributes by implementing |
|
|
Mockup object that resembles a module with arbitrarily deep structure such that, e.g., |
|
Leight-weight utility class that wraps all passed args and kwargs and allows to invoke different functions with them. |
|
Generic descriptor class that is used by |
|
Propety decorator for class-level methods. |
|
Meta class for |
Derivable base class with features provided by the meta |
Attributes#
UNSET
#
- util.UNSET = <object object>#
env_is_remote
#
- util.env_is_remote = False#
env_is_dev
#
- util.env_is_dev = False#
primes
#
- util.primes = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199, 211, 223, 227, 229, 233, 239, 241, 251, 257, 263, 269, 271, 277, 281, 283, 293, 307, 311, 313, 317, 331, 337, 347, 349, 353, 359, 367, 373, 379, 383, 389, 397, 401, 409, 419, 421, 431, 433, 439, 443, 449, 457, 461, 463, 467, 479, 487, 491, 499, 503, 509, 521, 523, 541]#
Functions#
maybe_import
#
- maybe_import(name, package=None, force=False)[source]#
Calls importlib.import_module internally and returns the module if it exists, or otherwise a
MockModule
instance with the same name. When force is True and the import fails, an ImportError is raised.- Return type:
ModuleType | MockModule
import_plt
#
import_ROOT
#
import_file
#
- import_file(path, attr=None)[source]#
Loads the content of a python file located at path and returns its package content as a dictionary. When attr is set, only the attribute with that name is returned.
The file is not required to be importable as its content is loaded directly into the interpreter. While this approach is not necessarily clean, it can be useful in places where custom code must be loaded.
create_random_name
#
expand_path
#
real_path
#
ensure_dir
#
wget
#
call_thread
#
call_proc
#
ensure_proxy
#
dev_sandbox
#
- dev_sandbox(sandbox, add=True, remove=True)[source]#
Takes a sandbox key sandbox and adds or removes the substring “_dev” right before the file extension (if any), depending on whether the current environment is used for development (see
env_is_dev
) and the add and remove flags.If sandbox does not contain the “_dev” postfix and both
env_is_dev
and add are True, the postfix is appended.If sandbox does (!) contain the “_dev” postfix,
env_is_dev
is False and remove is True, the postfix is removed.In any other case, sandbox is returned unchanged.
Examples:
# if env_is_dev and /path/to/script_dev.sh exists dev_sandbox("bash::/path/to/script.sh") # -> "bash::/path/to/script_dev.sh" # otherwise dev_sandbox("bash::/path/to/script.sh") # -> "bash::/path/to/script.sh"
- Return type:
safe_div
#
test_float
#
is_pattern
#
is_regex
#
pattern_matcher
#
- pattern_matcher(pattern, mode=<built-in function any>)[source]#
Takes a string pattern which might be an actual pattern for fnmatching, a regular expressions or just a plain string and returns a function that can be used to test of a string matches that pattern.
When pattern is a sequence, all its patterns are compared the same way and the result is the combination given a mode which typically should be any or all.
Example:
matcher = pattern_matcher("foo*") matcher("foo123") # -> True matcher("bar123") # -> False matcher = pattern_matcher(r"^foo\d+.*$") matcher("foox") # -> False matcher("foo1") # -> True matcher = pattern_matcher(("foo*", "*bar"), mode=any) matcher("foo123") # -> True matcher("123bar") # -> True matcher = pattern_matcher(("foo*", "*bar"), mode=all) matcher("foo123") # -> False matcher("123bar") # -> False matcher("foo123bar") # -> True
dict_add_strict
#
get_source_code
#
classproperty
#
Classes#
DotDict
#
- class DotDict[source]#
Bases:
OrderedDict
Subclass of OrderedDict that provides read and write access to items via attributes by implementing
__getattr__
and__setattr__
. In case a item is accessed via attribute and it does not exist, an AttriuteError is raised rather than a KeyError. Example:d = DotDict() d["foo"] = 1 print(d["foo"]) # => 1 print(d.foo) # => 1 print(d["bar"]) # => KeyError print(d.bar) # => AttributeError d.bar = 123 print(d.bar) # => 123 # use wrap() to convert a nested dict d = DotDict({"foo": {"bar": 1}}) print(d.foo.bar) # => 1
MockModule
#
- class MockModule(name)[source]#
Bases:
object
Mockup object that resembles a module with arbitrarily deep structure such that, e.g.,
coffea = MockModule("coffea") print(coffea.nanoevents.NanoEventsArray) # -> "<MockupModule 'coffea' at 0x981jald1>"
will always succeed at declaration, but most likely fail at execution time. In fact, each attribute access will return the mock object again. This might only be useful in places where a module is potentially not existing (e.g. due to sandboxing) but one wants to import it either way a) to perform only one top-level import as opposed to imports in all functions of a package, or b) to provide type hints for documentation purposes.
- _name#
- type: str
The name of the mock module.
FunctionArgs
#
ClassPropertyDescriptor
#
DerivableMeta
#
- class DerivableMeta(cls_name: str, bases: tuple, cls_dict: dict)[source]#
Bases:
ABCMeta
Meta class for
Derivable
objects providing class-level features such as improved tracing and lookup of subclasses, and single-line subclassing for partial-like overwriting of class-level attributes.- has_cls(cls_name, deep=True)[source]#
Returns True if this class has a subclass named cls_name and False otherwise. When deep is True, the lookup is recursive through all levels of subclasses.
- Return type:
- get_cls(cls_name, deep=True, silent=False)[source]#
Returns a previously created subclass named cls_name. When deep is True, the lookup is recursive through all levels of subclasses. When no such subclass was found an exception is raised, unless silent is True in which case None is returned.
- Return type:
DerivableMeta | None