5.1.1.5. recipe_system.utils package¶
5.1.1.5.1. Subpackages¶
5.1.1.5.2. Submodules¶
5.1.1.5.3. recipe_system.utils.decorators module¶
5.1.1.5.4. recipe_system.utils.errors module¶
5.1.1.5.5. recipe_system.utils.findexe module¶
5.1.1.5.6. recipe_system.utils.mapper_utils module¶
Utility functions for Mappers.
find_user_recipe – searches for a user specified recipe, if any. dotpath() – build a python import path for dr packages.
-
recipe_system.utils.mapper_utils.dotpath(*args)[source]¶ Build an import path from args.
Parameters: args (<list>, implied by *) – a set of arguments of arbitrary length Returns: a dot path to an importable module Return type: <str>
-
recipe_system.utils.mapper_utils.find_user_recipe(dashr)[source]¶ Function receives the value of the reduce [-r, –recipe] flag, if passed. This will be a path to a recipe file and a dotted recipe name, which exists as a function in the recipe file. A properly specified user recipe shall contain one, and only one, dot operator.
If the recipefile.recipename cannot be found on the path, whether specified or implied (as cwd), then None is returned.
The string value of dashr will look like,
-r ‘/path/to/users/recipes/recipefile.recipe_function’
-r ‘recipefile.recipe_function’ – recipe file in cwd.
A recipe name with no dot operator implies a recipe name in the system recipe library.
Parameters: dashr (<str>) – a path to a recipe file dotted with a recipe function name. Returns: imported recipe function OR None Return type: <type ‘function’> or None
5.1.1.5.7. recipe_system.utils.md5 module¶
-
recipe_system.utils.md5.md5sum(filename)[source]¶ Generates the md5sum of the data in filename, returns the hex string.
-
recipe_system.utils.md5.md5sum_size_fp(fobj)[source]¶ Generates the md5sum and size of the data returned by the file-like object fobj, returns a tuple containing the hex string md5 and the size in bytes. f must be open. It will not be closed. We will read from it until we encounter EOF. No seeks will be done, fobj will be left at eof
5.1.1.5.8. recipe_system.utils.reduceActions module¶
This module provides a number “action” classes, subclassed from the argparse.Action class. These classes only override the __call__() method. This actions class library supplies ad hoc functionality to DPDG requirements on the reduce command line interface.
Action classes provided:
PosArgAction - positional argument BooleanAction - optional switches UnitaryArgumentAction - single value options ParameterAction - user parameters (-p, –param) CalibrationAction - user calibration services (–user_cal)
Becuase of requirements on the reduce interface, any new reduce options should specify one of these actions in the add_argument() call. But only one (1) PosArgAction should occur in a given parser.
These actions may be used in the add_argument() method call, such as,
- parser.add_argument(‘-f’, ‘–foo’, action=BooleanAction,
- help=”Switch on foo.”)
-
class
recipe_system.utils.reduceActions.BooleanAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]¶ Bases:
argparse.Action
-
class
recipe_system.utils.reduceActions.CalibrationAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]¶ Bases:
argparse.Action
-
class
recipe_system.utils.reduceActions.ParameterAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]¶ Bases:
argparse.Action
-
class
recipe_system.utils.reduceActions.PosArgAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]¶ Bases:
argparse.Action
-
class
recipe_system.utils.reduceActions.UnitaryArgumentAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]¶ Bases:
argparse.Action
5.1.1.5.9. recipe_system.utils.reduce_utils module¶
-
class
recipe_system.utils.reduce_utils.ReduceArgumentParser(prog=None, usage=None, description=None, epilog=None, parents=[], formatter_class=<class 'argparse.HelpFormatter'>, prefix_chars='-', fromfile_prefix_chars=None, argument_default=None, conflict_handler='error', add_help=True, allow_abbrev=True, exit_on_error=True)[source]¶ Bases:
argparse.ArgumentParserConverts an argument line from a user param file into an actual argument, yields to the calling parser.
-
class
recipe_system.utils.reduce_utils.ReduceHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)[source]¶ Bases:
argparse.RawDescriptionHelpFormatterReduceHelpFormatter class overrides default help formatting on customized reduce actions.
-
recipe_system.utils.reduce_utils.normalize_args(args)[source]¶ Convert argparse argument lists to single string values.
Parameters: args (<Namespace>) – argparse Namespace object or equivalent Returns: Same with converted types. Return type: <Namespace>
-
recipe_system.utils.reduce_utils.normalize_ucals(cals)[source]¶ When a user passes a –user_cal argument of the form,
–user_cal processed_bias:/path/to/foo.fits
The parser produces a user calibrations list like,
[‘processed_bias:/path/to/foo.fits’]
This list would pass to the Reduce __init__ as such, but, this function will translate into a dict and confirm that the provided file exists and is of the correct type.
{‘processed_bias’: ‘/path/to/foo.fits’}
User calibrations always take precedence over nominal calibration retrieval. User calibrations are not cached because they are not retrieved from fitsstore and are presumably on disk.
- cals : list
- A list of strings like, ‘caltype:calfilepath’.
- normalz : dict
- a dictionary of the cal types applied to input files.
-
recipe_system.utils.reduce_utils.normalize_upload(upload)[source]¶ For Recipe System v2.0, upload shall now be a list of things to send to fitsstore. E.g., $ reduce –upload metrics <file.fits> <file2.fits> $ reduce –upload metrics, calibs <file.fits> <file2.fits> $ reduce –upload metrics, calibs, science <file.fits> <file2.fits>
Result in
upload == [‘metrics’] upload == [‘metrics’, ‘calibs’] upload == [‘metrics’, ‘calibs’, ‘science’]
Parameters: upload (<list>) – upload argument received by the reduce command line. Returns: list of coerced or defaulted upload instructions. Return type: <list>
-
recipe_system.utils.reduce_utils.set_btypes(userparams)[source]¶ All cmd line args are delivered as strings. Find any user parameters that should be other python types and set them to those actual corresponding types.
I.e.,
‘None’ –> None ‘True’ –> True ‘False’ –> FalseParameters userparams: user parameters (if any) passed on the command line. Returns: A tuple of same parameters with converted None and boolean types. preserved with any specified primitive name. E.g., [(‘foo’,’bar’), (‘tileArrays:par1’,’val1’)] Return type: <list> of tuples.
5.1.1.5.10. recipe_system.utils.rs_utilities module¶
Set of functions in support of the recipe_system.
-
recipe_system.utils.rs_utilities.makedrpkg(pkgname, instruments, modes=None)[source]¶ Create the basic structure for a data reduction package that the recipe system will recognize.
- pkgname: str
- Name of the new dr-package.
- instruments: list of str
- Name of the instrument(s) for which to create a directory structure.
- modes: list of str
- Name of the recipe modes that will be supported. Eg. modes = [‘sq’, ‘qa’] Default: [‘sq’] (‘sq’=science quality, ‘qa’=quality assessement)