astrodata package

This package add another abstraction layer to astronomical data by parsing the information contained in the headers as attributes. To do so, one must subclass astrodata.AstroData and add parse methods accordingly to the TagSet received.

class astrodata.AstroData(provider)[source]

Bases: object

Base class for the AstroData software package. It provides an interface to manipulate astronomical data sets.

Parameters:provider (DataProvider) – The data that will be manipulated through the AstroData instance.
add(oper)

Implements the augmented arithmetic assignment +=.

Parameters:oper (number or object) – The operand to be added to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
append(extension, name=None, *args, **kw)[source]

Adds a new top-level extension to the provider. Please, read the the concrete DataProvider documentation that is being used to know the exact behavior and additional accepted arguments.

Parameters:
  • extension (array, Table, or other) – The contents for the new extension. Usually the underlying DataProvider will understand how to deal with regular NumPy arrays and with AstroData Table instances, but it may also accept other types.
  • name (string, optional) – A DataProvider will usually require a name for extensions. If the name cannot be derived from the metadata associated to extension, you will have to provider one.
  • args (optional) – The DataProvider may accept additional arguments. Please, refer to its documentation.
  • kw (optional) – The DataProvider may accept additional arguments. Please, refer to its documentation.
Returns:

  • The instance that has been added internally (potentially *not the same that*
  • was passed as *extension)*

Raises:
  • TypeError – Will be raised if the DataProvider doesn’t know how to deal with the data that has been passed.
  • ValueError – Raised if the extension is of a proper type, but its value is illegal somehow.
descriptors

Returns a sequence of names for the methods that have been decorated as descriptors.

Returns:
Return type:A tuple of str
divide(oper)

Implements the augmented arithmetic assignment /=.

Parameters:oper (number or other) – The operand to be divided to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
info()[source]

Prints out information about the contents of this instance. Implemented by the derived classes.

load(source)[source]

Class method that returns an instance of this same class, properly initialized with a DataProvider that can deal with the object passed as source

This method is abstract and has to be implemented by derived classes.

multiply(oper)

Implements the augmented arithmetic assignment *=.

Parameters:oper (number or object) – The operand to be multiplied to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
operate(operator, *args, **kwargs)[source]

Applies a function to the main data array on each extension, replacing the data with the result. The data will be passed as the first argument to the function.

It will be applied to the mask and variance of each extension, too, if they exist.

This is a convenience method, which is equivalent to:

for ext in ad:
    ad.ext.data = operator(ad.ext.data, *args, **kwargs)
    ad.ext.mask = operator(ad.ext.mask, *args, **kwargs) if ad.ext.mask is not None else None
    ad.ext.variance = operator(ad.ext.variance, *args, **kwargs) if ad.ext.variance is not None else None

with the additional advantage that it will work on single slices, too.

Parameters:
  • operator (function, or bound method) – A function that takes an array (and, maybe, other arguments) and returns an array
  • args (optional) – Additional arguments to be passed positionally to the operator
  • kwargs (optional) – Additional arguments to be passed by name to the operator

Examples

>>> import numpy as np
>>> ad.operate(np.squeeze)  # doctest: +SKIP
reset(data, mask=-23, variance=-23, check=True)[source]

Sets the .data, and optionally .mask and .variance attributes of a single-extension AstroData slice. This function will optionally check whether these attributes have the same shape.

Parameters:
  • data (ndarray) – The array to assign to the .data attribute (“SCI”)
  • mask (ndarray, optional) – The array to assign to the .mask attribute (“DQ”)
  • variance (ndarray, optional) – The array to assign to the .variance attribute (“VAR”)
  • check (bool) – If set, then the function will check that the mask and variance arrays have the same shape as the data array
Raises:
  • TypeError – if an attempt is made to set the .mask or .variance attributes with something other than an array
  • ValueError – if the .mask or .variance attributes don’t have the same shape as .data, OR if this is called on an AD instance that isn’t a single extension slice
subtract(oper)

Implements the augmented arithmetic assignment -=.

Parameters:oper (number or object) – The operand to be subtracted to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
tags

A set of strings that represent the tags defining this instance

exception astrodata.AstroDataError[source]

Bases: Exception

class astrodata.TagSet(add=None, remove=None, blocked_by=None, blocks=None, if_present=None)[source]

Bases: astrodata.core.TagSet

Named tuple that is used by tag methods to return which actions should be performed on a tag set. All the attributes are optional, and any combination of them can be used, allowing to create complex tag structures. Read the documentation on the tag-generating algorithm if you want to better understand the interactions.

The simplest TagSet, though, tends to just add tags to the global set.

It can be initialized by position, like any other tuple (the order of the arguments is the one in which the attributes are listed below). It can also be initialized by name.

add

Tags to be added to the global set

Type:set of str, or None
remove

Tags to be removed from the global set

Type:set of str, or None
blocked_by

Tags that will prevent this TagSet from being applied

Type:set of str, or None
blocks

Other TagSets containing these won’t be applied

Type:set of str, or None
if_present

This TagSet will be applied only all of these tags are present

Type:set of str, or None

Examples

>>> TagSet()
TagSet(add=set(), remove=set(), blocked_by=set(), blocks=set(), if_present=set())
>>> TagSet({'BIAS', 'CAL'})
TagSet(add={'BIAS', 'CAL'}, remove=set(), blocked_by=set(), blocks=set(), if_present=set())
>>> TagSet(remove={'BIAS', 'CAL'})
TagSet(add=set(), remove={'BIAS', 'CAL'}, blocked_by=set(), blocks=set(), if_present=set())
class astrodata.NDAstroData(data, uncertainty=None, mask=None, wcs=None, meta=None, unit=None, copy=False, window=None)[source]

Bases: astropy.nddata.mixins.ndarithmetic.NDArithmeticMixin, astropy.nddata.mixins.ndslicing.NDSlicingMixin, astropy.nddata.nddata.NDData

Implements NDData with all Mixins, plus some AstroData specifics.

This class implements an NDData-like container that supports reading and writing as implemented in the astropy.io.registry and also slicing (indexing) and simple arithmetics (add, subtract, divide and multiply).

A very important difference between NDAstroData and NDData is that the former attempts to load all its data lazily. There are also some important differences in the interface (eg. .data lets you reset its contents after initialization).

Documentation is provided where our class differs.

See also

NDData, NDArithmeticMixin, NDSlicingMixin

Examples

The mixins allow operation that are not possible with NDData or NDDataBase, i.e. simple arithmetics:

>>> from astropy.nddata import StdDevUncertainty
>>> import numpy as np
>>> data = np.ones((3,3), dtype=np.float)
>>> ndd1 = NDAstroData(data, uncertainty=StdDevUncertainty(data))
>>> ndd2 = NDAstroData(data, uncertainty=StdDevUncertainty(data))
>>> ndd3 = ndd1.add(ndd2)
>>> ndd3.data
array([[2., 2., 2.],
    [2., 2., 2.],
    [2., 2., 2.]])
>>> ndd3.uncertainty.array
array([[1.41421356, 1.41421356, 1.41421356],
    [1.41421356, 1.41421356, 1.41421356],
    [1.41421356, 1.41421356, 1.41421356]])

see NDArithmeticMixin for a complete list of all supported arithmetic operations.

But also slicing (indexing) is possible:

>>> ndd4 = ndd3[1,:]
>>> ndd4.data
array([2., 2., 2.])
>>> ndd4.uncertainty.array
array([1.41421356, 1.41421356, 1.41421356])

See NDSlicingMixin for a description how slicing works (which attributes) are sliced.

T
data

An array representing the raw data stored in this instance. It implements a setter.

mask

Mask for the dataset, if any.

Masks should follow the numpy convention that valid data points are marked by False and invalid ones with True.

Type:any type
set_section(section, input)[source]

Sets only a section of the data. This method is meant to prevent fragmentation in the Python heap, by reusing the internal structures instead of replacing them with new ones.

Parameters:
  • section (slice) – The area that will be replaced
  • input (NDData-like instance) – This object needs to implement at least data, uncertainty, and mask. Their entire contents will replace the data in the area defined by section.

Examples

>>> sec = NDData(np.zeros((100,100)))  # doctest: +SKIP
>>> ad[0].nddata.set_section((slice(None,100),slice(None,100)), sec)  # doctest: +SKIP
shape
transpose()[source]
uncertainty

Uncertainty in the dataset, if any.

Should have an attribute uncertainty_type that defines what kind of uncertainty is stored, such as 'std' for standard deviation or 'var' for variance. A metaclass defining such an interface is ~astropy.nddata.NDUncertainty but isn’t mandatory.

Type:any type
variance

A convenience property to access the contents of uncertainty, squared (as the uncertainty data is stored as standard deviation).

wcs

A world coordinate system (WCS) for the dataset, if any.

Type:any type
window

Interface to access a section of the data, using lazy access whenever possible.

Returns:
  • An instance of NDWindowing, which provides __getitem__, to allow the use
  • of square brackets when specifying the window. Ultimately, an
  • NDWindowingAstrodata instance is returned

Examples

>>> ad[0].nddata.window[100:200, 100:200]  # doctest: +SKIP
<NDWindowingAstrodata .....>
astrodata.astro_data_descriptor(fn)[source]

Decorator that will mark a class method as an AstroData descriptor. Useful to produce list of descriptors, for example.

If used in combination with other decorators, this one must be the one on the top (ie. the last one applying). It doesn’t modify the method in any other way.

Parameters:fn (method) – The method to be decorated
Returns:
Return type:The tagged method (not a wrapper)
astrodata.astro_data_tag(fn)[source]

Decorator that marks methods of an AstroData derived class as part of the tag-producing system.

It wraps the method around a function that will ensure a consistent return value: the wrapped method can return any sequence of sequences of strings, and they will be converted to a TagSet. If the wrapped method returns None, it will be turned into an empty TagSet.

Parameters:fn (method) – The method to be decorated
Returns:
Return type:A wrapper function
astrodata.keyword

alias of astrodata.fits.KeywordCallableWrapper

astrodata.version(short=False, tag='')[source]

Returns DRAGONS’s version based on the api, feature and bug numbers.

Returns:str
Return type:formatted version

Submodules

astrodata.core module

class astrodata.core.AstroData(provider)[source]

Bases: object

Base class for the AstroData software package. It provides an interface to manipulate astronomical data sets.

Parameters:provider (DataProvider) – The data that will be manipulated through the AstroData instance.
add(oper)

Implements the augmented arithmetic assignment +=.

Parameters:oper (number or object) – The operand to be added to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
append(extension, name=None, *args, **kw)[source]

Adds a new top-level extension to the provider. Please, read the the concrete DataProvider documentation that is being used to know the exact behavior and additional accepted arguments.

Parameters:
  • extension (array, Table, or other) – The contents for the new extension. Usually the underlying DataProvider will understand how to deal with regular NumPy arrays and with AstroData Table instances, but it may also accept other types.
  • name (string, optional) – A DataProvider will usually require a name for extensions. If the name cannot be derived from the metadata associated to extension, you will have to provider one.
  • args (optional) – The DataProvider may accept additional arguments. Please, refer to its documentation.
  • kw (optional) – The DataProvider may accept additional arguments. Please, refer to its documentation.
Returns:

  • The instance that has been added internally (potentially *not the same that*
  • was passed as *extension)*

Raises:
  • TypeError – Will be raised if the DataProvider doesn’t know how to deal with the data that has been passed.
  • ValueError – Raised if the extension is of a proper type, but its value is illegal somehow.
descriptors

Returns a sequence of names for the methods that have been decorated as descriptors.

Returns:
Return type:A tuple of str
divide(oper)

Implements the augmented arithmetic assignment /=.

Parameters:oper (number or other) – The operand to be divided to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
info()[source]

Prints out information about the contents of this instance. Implemented by the derived classes.

load(source)[source]

Class method that returns an instance of this same class, properly initialized with a DataProvider that can deal with the object passed as source

This method is abstract and has to be implemented by derived classes.

multiply(oper)

Implements the augmented arithmetic assignment *=.

Parameters:oper (number or object) – The operand to be multiplied to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
operate(operator, *args, **kwargs)[source]

Applies a function to the main data array on each extension, replacing the data with the result. The data will be passed as the first argument to the function.

It will be applied to the mask and variance of each extension, too, if they exist.

This is a convenience method, which is equivalent to:

for ext in ad:
    ad.ext.data = operator(ad.ext.data, *args, **kwargs)
    ad.ext.mask = operator(ad.ext.mask, *args, **kwargs) if ad.ext.mask is not None else None
    ad.ext.variance = operator(ad.ext.variance, *args, **kwargs) if ad.ext.variance is not None else None

with the additional advantage that it will work on single slices, too.

Parameters:
  • operator (function, or bound method) – A function that takes an array (and, maybe, other arguments) and returns an array
  • args (optional) – Additional arguments to be passed positionally to the operator
  • kwargs (optional) – Additional arguments to be passed by name to the operator

Examples

>>> import numpy as np
>>> ad.operate(np.squeeze)  # doctest: +SKIP
reset(data, mask=-23, variance=-23, check=True)[source]

Sets the .data, and optionally .mask and .variance attributes of a single-extension AstroData slice. This function will optionally check whether these attributes have the same shape.

Parameters:
  • data (ndarray) – The array to assign to the .data attribute (“SCI”)
  • mask (ndarray, optional) – The array to assign to the .mask attribute (“DQ”)
  • variance (ndarray, optional) – The array to assign to the .variance attribute (“VAR”)
  • check (bool) – If set, then the function will check that the mask and variance arrays have the same shape as the data array
Raises:
  • TypeError – if an attempt is made to set the .mask or .variance attributes with something other than an array
  • ValueError – if the .mask or .variance attributes don’t have the same shape as .data, OR if this is called on an AD instance that isn’t a single extension slice
subtract(oper)

Implements the augmented arithmetic assignment -=.

Parameters:oper (number or object) – The operand to be subtracted to this instance. The accepted types depend on the DataProvider.
Returns:
Return type:self
tags

A set of strings that represent the tags defining this instance

exception astrodata.core.AstroDataError[source]

Bases: Exception

class astrodata.core.DataProvider[source]

Bases: object

Abstract class describing the minimal interface that DataProvider derivative classes need to implement.

append(ext, name=None)[source]

Adds a new component to the provider. Objects appended to a single slice will actually be made hierarchically dependent of the science object represented by that slice. If appended to the provider as a whole, the new member will be independent (eg. global table, new science object).

Parameters:
  • ext (array, NDData, Table, etc) – The component to be added. The exact accepted types depend on the class implementing this interface. Implementations specific to certain data formats may accept specialized types (eg. a FITS provider will accept an ImageHDU and extract the array out of it)
  • name (str, optional) –

    A name that may be used to access the new object, as an attribute of the provider. The name is typically ignored for top-level (global) objects, and required for the others.

    It can consist in a combination of numbers and letters, with the restriction that the letters have to be all capital, and the first character cannot be a number (“[A-Z][A-Z0-9]*”).

Returns:

  • The same object, or a new one, if it was necessary to convert it to a more
  • suitable format for internal use.

Raises:
  • TypeError – If adding the object in an invalid situation (eg. name is None when adding to a single slice)
  • ValueError – If adding an object that is not acceptable
data

A list of the the arrays (or single array, if this is a single slice) corresponding to the science data attached to each extension, in loading/appending order.

exposed

A collection of strings with the names of objects that can be accessed directly by name as attributes of this instance, and that are not part of its standard interface (ie. data objects that have been added dynamically).

Examples

>>> ad[0].exposed  # doctest: +SKIP
set(['OBJMASK', 'OBJCAT'])
is_settable(attribute)[source]

Predicate that can be used to figure out if certain attribute of the DataProvider is meant to be modified by an external object.

This is used mostly by AstroData, which acts as a proxy exposing attributes of its assigned provider, to decide if it should set a value on the provider or on itself.

Parameters:attribute (str) –
Returns:
Return type:A boolean
is_single

If this data provider represents a single slice out of a whole dataset, return True. Otherwise, return False.

Returns:
Return type:A boolean
is_sliced

If this data provider instance represents the whole dataset, return False. If it represents a slice out of the whole, return True.

Returns:
Return type:A boolean
mask

A list of the mask arrays (or a single array, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

For objects that miss a mask, None will be provided instead.

uncertainty

A list of the uncertainty objects (or a single object, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

The objects are instances of AstroPy’s NDUncertainty, or None where no information is available.

See also

variance
The actual array supporting the uncertainty object
variance

A list of the variance arrays (or a single array, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

For objects that miss uncertainty information, None will be provided instead.

See also

uncertainty
The NDUncertainty object used under the hood to propagate uncertainty when

operating

class astrodata.core.TagSet(add=None, remove=None, blocked_by=None, blocks=None, if_present=None)[source]

Bases: astrodata.core.TagSet

Named tuple that is used by tag methods to return which actions should be performed on a tag set. All the attributes are optional, and any combination of them can be used, allowing to create complex tag structures. Read the documentation on the tag-generating algorithm if you want to better understand the interactions.

The simplest TagSet, though, tends to just add tags to the global set.

It can be initialized by position, like any other tuple (the order of the arguments is the one in which the attributes are listed below). It can also be initialized by name.

add

Tags to be added to the global set

Type:set of str, or None
remove

Tags to be removed from the global set

Type:set of str, or None
blocked_by

Tags that will prevent this TagSet from being applied

Type:set of str, or None
blocks

Other TagSets containing these won’t be applied

Type:set of str, or None
if_present

This TagSet will be applied only all of these tags are present

Type:set of str, or None

Examples

>>> TagSet()
TagSet(add=set(), remove=set(), blocked_by=set(), blocks=set(), if_present=set())
>>> TagSet({'BIAS', 'CAL'})
TagSet(add={'BIAS', 'CAL'}, remove=set(), blocked_by=set(), blocks=set(), if_present=set())
>>> TagSet(remove={'BIAS', 'CAL'})
TagSet(add=set(), remove={'BIAS', 'CAL'}, blocked_by=set(), blocks=set(), if_present=set())
astrodata.core.astro_data_descriptor(fn)[source]

Decorator that will mark a class method as an AstroData descriptor. Useful to produce list of descriptors, for example.

If used in combination with other decorators, this one must be the one on the top (ie. the last one applying). It doesn’t modify the method in any other way.

Parameters:fn (method) – The method to be decorated
Returns:
Return type:The tagged method (not a wrapper)
astrodata.core.astro_data_tag(fn)[source]

Decorator that marks methods of an AstroData derived class as part of the tag-producing system.

It wraps the method around a function that will ensure a consistent return value: the wrapped method can return any sequence of sequences of strings, and they will be converted to a TagSet. If the wrapped method returns None, it will be turned into an empty TagSet.

Parameters:fn (method) – The method to be decorated
Returns:
Return type:A wrapper function
astrodata.core.returns_list(fn)[source]

Decorator to ensure that descriptors that should return a list (of one value per extension) only returns single values when operating on single slices; and vice versa.

This is a common case, and you can use the decorator to simplify the logic of your descriptors.

Parameters:fn (method) – The method to be decorated
Returns:
Return type:A function

astrodata.factory module

class astrodata.factory.AstroDataFactory[source]

Bases: object

addClass(cls)[source]

Add a new class to the AstroDataFactory registry. It will be used when instantiating an AstroData class for a FITS file.

createFromScratch(phu, extensions=None)[source]

Creates an AstroData object from a collection of objects.

Parameters:
  • phu (fits.PrimaryHDU or fits.Header or dict or list) – FITS primary HDU or header, or something that can be used to create a fits.Header (a dict, a list of “cards”).
  • extensions (list of HDUs) – List of HDU objects.
getAstroData(source)[source]

Takes either a string (with the path to a file) or an HDUList as input, and tries to return an AstroData instance.

It will raise exceptions if the file is not found, or if there is no match for the HDUList, among the registered AstroData classes.

Returns an instantiated object, or raises AstroDataError if it was not possible to find a match

astrodata.factory.fits_opener(source)[source]

astrodata.fits module

class astrodata.fits.AstroDataFits(provider)[source]

Bases: astrodata.core.AstroData

extver(ver)[source]

Get an extension using its EXTVER instead of the positional index in this object.

Parameters:ver (int) – The EXTVER for the desired extension
Returns:
Return type:A sliced object containing the desired extension
Raises:IndexError – If the provided EXTVER doesn’t exist
hdr
info()[source]

Prints out information about the contents of this instance. Implemented by the derived classes.

instrument()[source]

Returns the name of the instrument making the observation

Returns:instrument name
Return type:str
classmethod load(source)[source]

Implementation of the abstract method load.

It takes an HDUList and returns a fully instantiated AstroData instance.

object()[source]

Returns the name of the object being observed

Returns:object name
Return type:str
phu
telescope()[source]

Returns the name of the telescope

Returns:name of the telescope
Return type:str
update_filename(prefix=None, suffix=None, strip=False)[source]

This method updates the “filename” attribute of the AstroData object. A prefix and/or suffix can be specified. If strip=True, these will replace the existing prefix/suffix; if strip=False, they will simply be prepended/appended.

The current filename is broken down into its existing prefix, root, and suffix using the ORIGNAME phu keyword, if it exists and is contained within the current filename. Otherwise, the filename is split at the last underscore and the part before is assigned as the root and the underscore and part after the suffix. No prefix is assigned.

Note that, if strip=True, a prefix or suffix will only be stripped if ‘’ is specified.

Parameters:
  • prefix (str/None) – new prefix (None => leave alone)
  • suffix (str/None) – new suffix (None => leave alone)
  • strip (bool) – Strip existing prefixes and suffixes if new ones are given?
write(filename=None, overwrite=False)[source]
exception astrodata.fits.AstroDataFitsDeprecationWarning[source]

Bases: DeprecationWarning

class astrodata.fits.FitsHeaderCollection(headers)[source]

Bases: object

This class provides group access to a list of PyFITS Header-like objects. It exposes a number of methods (set, get, etc.) that operate over all the headers at the same time.

It can also be iterated.

get(key, default=None)[source]
get_comment(key)[source]
remove(key)[source]
set(key, value=None, comment=None)[source]
set_comment(key, comment)[source]
class astrodata.fits.FitsLazyLoadable(obj)[source]

Bases: object

data
dtype

Need to to some overriding of astropy.io.fits since it doesn’t know about BITPIX=8

header
shape
class astrodata.fits.FitsLoader(cls=<class 'astrodata.fits.FitsProvider'>)[source]

Bases: object

load(source, extname_parser=None)[source]

Takes either a string (with the path to a file) or an HDUList as input, and tries to return a populated FitsProvider (or descendant) instance.

It will raise exceptions if the file is not found, or if there is no match for the HDUList, among the registered AstroData classes.

class astrodata.fits.FitsProvider[source]

Bases: astrodata.core.DataProvider

append(ext, name=None, header=None, reset_ver=True, add_to=None)[source]

Adds a new component to the provider. Objects appended to a single slice will actually be made hierarchically dependent of the science object represented by that slice. If appended to the provider as a whole, the new member will be independent (eg. global table, new science object).

Parameters:
  • ext (array, NDData, Table, etc) – The component to be added. The exact accepted types depend on the class implementing this interface. Implementations specific to certain data formats may accept specialized types (eg. a FITS provider will accept an ImageHDU and extract the array out of it)
  • name (str, optional) –

    A name that may be used to access the new object, as an attribute of the provider. The name is typically ignored for top-level (global) objects, and required for the others.

    It can consist in a combination of numbers and letters, with the restriction that the letters have to be all capital, and the first character cannot be a number (“[A-Z][A-Z0-9]*”).

Returns:

  • The same object, or a new one, if it was necessary to convert it to a more
  • suitable format for internal use.

Raises:
  • TypeError – If adding the object in an invalid situation (eg. name is None when adding to a single slice)
  • ValueError – If adding an object that is not acceptable
crop(x1, y1, x2, y2)[source]
data

A list of the the arrays (or single array, if this is a single slice) corresponding to the science data attached to each extension, in loading/appending order.

default_extension = 'SCI'
exposed

A collection of strings with the names of objects that can be accessed directly by name as attributes of this instance, and that are not part of its standard interface (ie. data objects that have been added dynamically).

Examples

>>> ad[0].exposed  # doctest: +SKIP
set(['OBJMASK', 'OBJCAT'])
extver_map()[source]

Provide a mapping between the FITS EXTVER of an extension and the index that will be used to access it within this object.

Returns:A dictionary `{EXTVER
Return type:index, ..}`
Raises:ValueError – If used against a single slice. It is of no use in that situation.
filename
hdr()[source]
header
info(tags, indices=None)[source]
is_settable(attr)[source]

Predicate that can be used to figure out if certain attribute of the DataProvider is meant to be modified by an external object.

This is used mostly by AstroData, which acts as a proxy exposing attributes of its assigned provider, to decide if it should set a value on the provider or on itself.

Parameters:attribute (str) –
Returns:
Return type:A boolean
mask

A list of the mask arrays (or a single array, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

For objects that miss a mask, None will be provided instead.

nddata
orig_filename
path
phu()[source]
set_phu(phu)[source]
shape
table()[source]
tables
to_hdulist()[source]
uncertainty

A list of the uncertainty objects (or a single object, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

The objects are instances of AstroPy’s NDUncertainty, or None where no information is available.

See also

variance
The actual array supporting the uncertainty object
variance

A list of the variance arrays (or a single array, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

For objects that miss uncertainty information, None will be provided instead.

See also

uncertainty
The NDUncertainty object used under the hood to propagate uncertainty when

operating

class astrodata.fits.FitsProviderProxy(provider, mapping, single)[source]

Bases: astrodata.core.DataProvider

append(ext, name)[source]

Adds a new component to the provider. Objects appended to a single slice will actually be made hierarchically dependent of the science object represented by that slice. If appended to the provider as a whole, the new member will be independent (eg. global table, new science object).

Parameters:
  • ext (array, NDData, Table, etc) – The component to be added. The exact accepted types depend on the class implementing this interface. Implementations specific to certain data formats may accept specialized types (eg. a FITS provider will accept an ImageHDU and extract the array out of it)
  • name (str, optional) –

    A name that may be used to access the new object, as an attribute of the provider. The name is typically ignored for top-level (global) objects, and required for the others.

    It can consist in a combination of numbers and letters, with the restriction that the letters have to be all capital, and the first character cannot be a number (“[A-Z][A-Z0-9]*”).

Returns:

  • The same object, or a new one, if it was necessary to convert it to a more
  • suitable format for internal use.

Raises:
  • TypeError – If adding the object in an invalid situation (eg. name is None when adding to a single slice)
  • ValueError – If adding an object that is not acceptable
crop(x1, y1, x2, y2)[source]
data

A list of the the arrays (or single array, if this is a single slice) corresponding to the science data attached to each extension, in loading/appending order.

exposed

A collection of strings with the names of objects that can be accessed directly by name as attributes of this instance, and that are not part of its standard interface (ie. data objects that have been added dynamically).

Examples

>>> ad[0].exposed  # doctest: +SKIP
set(['OBJMASK', 'OBJCAT'])
extver_map()[source]

Provide a mapping between the FITS EXTVER of an extension and the index that will be used to access it within this object.

Returns:A dictionary `{EXTVER
Return type:index, ..}`
Raises:ValueError – If used against a single slice. It is of no use in that situation.
hdr()[source]
header
info(tags)[source]
is_settable(attr)[source]

Predicate that can be used to figure out if certain attribute of the DataProvider is meant to be modified by an external object.

This is used mostly by AstroData, which acts as a proxy exposing attributes of its assigned provider, to decide if it should set a value on the provider or on itself.

Parameters:attribute (str) –
Returns:
Return type:A boolean
is_single

If this data provider represents a single slice out of a whole dataset, return True. Otherwise, return False.

Returns:
Return type:A boolean
is_sliced

If this data provider instance represents the whole dataset, return False. If it represents a slice out of the whole, return True.

Returns:
Return type:A boolean
mask

A list of the mask arrays (or a single array, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

For objects that miss a mask, None will be provided instead.

nddata
shape
uncertainty

A list of the uncertainty objects (or a single object, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

The objects are instances of AstroPy’s NDUncertainty, or None where no information is available.

See also

variance
The actual array supporting the uncertainty object
variance

A list of the variance arrays (or a single array, if this is a single slice) attached to the science data, for each extension, in loading/appending order.

For objects that miss uncertainty information, None will be provided instead.

See also

uncertainty
The NDUncertainty object used under the hood to propagate uncertainty when

operating

wcs
class astrodata.fits.KeywordCallableWrapper(keyword, default=<object object>, on_ext=False, coerce_with=None)[source]

Bases: object

astrodata.fits.add_header_to_table(table)[source]
astrodata.fits.asdftablehdu_to_wcs(hdu)[source]

Recreate a gWCS object from its serialization in a FITS table extension.

Returns None (issuing a warning) if the extension cannot be parsed, so the rest of the file can still be read.

astrodata.fits.card_filter(cards, include=None, exclude=None)[source]
astrodata.fits.deprecated(reason)[source]
astrodata.fits.fits_ext_comp_key(ext)[source]

Returns a pair (integer, string) that will be used to sort extensions

astrodata.fits.header_for_table(table)[source]
astrodata.fits.new_imagehdu(data, header, name=None)[source]
astrodata.fits.normalize_indices(slc, nitems)[source]
astrodata.fits.table_to_bintablehdu(table, extname=None)[source]

Convert an astropy Table object to a BinTableHDU before writing to disk.

Parameters:
  • table (astropy.table.Table instance) – the table to be converted to a BinTableHDU
  • extname (str) – name to go in the EXTNAME field of the FITS header
Returns:

Return type:

BinTableHDU

astrodata.fits.update_header(headera, headerb)[source]
astrodata.fits.wcs_to_asdftablehdu(wcs, extver=None)[source]

Serialize a gWCS object as a FITS TableHDU (ASCII) extension.

The ASCII table is actually a mini ASDF file. The constituent AstroPy models must have associated ASDF “tags” that specify how to serialize them.

In the event that serialization as pure ASCII fails (this should not happen), a binary table representation will be used as a fallback.

astrodata.fits.windowedOp(func, sequence, kernel, shape=None, dtype=None, with_uncertainty=False, with_mask=False, **kwargs)[source]

Apply function on a NDData obbjects, splitting the data in chunks to limit memory usage.

Parameters:
  • func (callable) – The function to apply.
  • sequence (list of NDData) – List of NDData objects.
  • kernel (tuple of int) – Shape of the blocks.
  • shape (tuple of int) – Shape of inputs. Defaults to sequence[0].shape.
  • dtype (str or dtype) – Type of the output array. Defaults to sequence[0].dtype.
  • with_uncertainty (bool) – Compute uncertainty?
  • with_mask (bool) – Compute mask?
  • **kwargs – Additional args are passed to func.

astrodata.nddata module

This module implements a derivative class based on NDData with some Mixins, implementing windowing and on-the-fly data scaling.

class astrodata.nddata.NDAstroData(data, uncertainty=None, mask=None, wcs=None, meta=None, unit=None, copy=False, window=None)[source]

Bases: astropy.nddata.mixins.ndarithmetic.NDArithmeticMixin, astropy.nddata.mixins.ndslicing.NDSlicingMixin, astropy.nddata.nddata.NDData

Implements NDData with all Mixins, plus some AstroData specifics.

This class implements an NDData-like container that supports reading and writing as implemented in the astropy.io.registry and also slicing (indexing) and simple arithmetics (add, subtract, divide and multiply).

A very important difference between NDAstroData and NDData is that the former attempts to load all its data lazily. There are also some important differences in the interface (eg. .data lets you reset its contents after initialization).

Documentation is provided where our class differs.

See also

NDData, NDArithmeticMixin, NDSlicingMixin

Examples

The mixins allow operation that are not possible with NDData or NDDataBase, i.e. simple arithmetics:

>>> from astropy.nddata import StdDevUncertainty
>>> import numpy as np
>>> data = np.ones((3,3), dtype=np.float)
>>> ndd1 = NDAstroData(data, uncertainty=StdDevUncertainty(data))
>>> ndd2 = NDAstroData(data, uncertainty=StdDevUncertainty(data))
>>> ndd3 = ndd1.add(ndd2)
>>> ndd3.data
array([[2., 2., 2.],
    [2., 2., 2.],
    [2., 2., 2.]])
>>> ndd3.uncertainty.array
array([[1.41421356, 1.41421356, 1.41421356],
    [1.41421356, 1.41421356, 1.41421356],
    [1.41421356, 1.41421356, 1.41421356]])

see NDArithmeticMixin for a complete list of all supported arithmetic operations.

But also slicing (indexing) is possible:

>>> ndd4 = ndd3[1,:]
>>> ndd4.data
array([2., 2., 2.])
>>> ndd4.uncertainty.array
array([1.41421356, 1.41421356, 1.41421356])

See NDSlicingMixin for a description how slicing works (which attributes) are sliced.

T
data

An array representing the raw data stored in this instance. It implements a setter.

mask

Mask for the dataset, if any.

Masks should follow the numpy convention that valid data points are marked by False and invalid ones with True.

Type:any type
set_section(section, input)[source]

Sets only a section of the data. This method is meant to prevent fragmentation in the Python heap, by reusing the internal structures instead of replacing them with new ones.

Parameters:
  • section (slice) – The area that will be replaced
  • input (NDData-like instance) – This object needs to implement at least data, uncertainty, and mask. Their entire contents will replace the data in the area defined by section.

Examples

>>> sec = NDData(np.zeros((100,100)))  # doctest: +SKIP
>>> ad[0].nddata.set_section((slice(None,100),slice(None,100)), sec)  # doctest: +SKIP
shape
transpose()[source]
uncertainty

Uncertainty in the dataset, if any.

Should have an attribute uncertainty_type that defines what kind of uncertainty is stored, such as 'std' for standard deviation or 'var' for variance. A metaclass defining such an interface is ~astropy.nddata.NDUncertainty but isn’t mandatory.

Type:any type
variance

A convenience property to access the contents of uncertainty, squared (as the uncertainty data is stored as standard deviation).

wcs

A world coordinate system (WCS) for the dataset, if any.

Type:any type
window

Interface to access a section of the data, using lazy access whenever possible.

Returns:
  • An instance of NDWindowing, which provides __getitem__, to allow the use
  • of square brackets when specifying the window. Ultimately, an
  • NDWindowingAstrodata instance is returned

Examples

>>> ad[0].nddata.window[100:200, 100:200]  # doctest: +SKIP
<NDWindowingAstrodata .....>

astrodata.provenance module

astrodata.provenance.add_provenance(ad, filename, md5, primitive, timestamp=None)[source]

Add the given Provenance entry to the full set of provenance records on this object.

Provenance is not added if the incoming md5 is None or ‘’. These indicate source data for the provenance that are not on disk, so not useful as provenance.

Parameters:value (Provenance to add) –
Returns:
Return type:none
astrodata.provenance.add_provenance_history(ad, timestamp_start, timestamp_stop, primitive, args)[source]

Add the given ProvenanceHistory entry to the full set of history records on this object.

Parameters:
  • ad (AstroData to add history record to) –
  • timestamp_start (datetime of the start of this operation) –
  • timestamp_stop (datetime of the end of this operation) –
  • primitive (str name of the primitive performed) –
  • args (str arguments used for the primitive call) –
Returns:

Return type:

none

astrodata.provenance.clone_provenance(provenance_data, ad)[source]

For a single input’s provenance, copy it into the output AstroData object as appropriate.

This takes a dictionary with a source filename, md5 and both it’s original provenance and provenance_history information. It duplicates the provenance data into the outgoing AstroData ad object.

Parameters:
  • provenance_data (pointer to the AstroData table with the provenance) – information. Note this may be the output AstroData as well, so we need to handle that.
  • ad (outgoing AstroData object to add provenance data to) –
Returns:

Return type:

none

astrodata.provenance.clone_provenance_history(provenance_history_data, ad)[source]

For a single input’s provenance history, copy it into the output AstroData object as appropriate.

This takes a dictionary with a source filename, md5 and both it’s original provenance and provenance_history information. It duplicates the provenance data into the outgoing AstroData ad object.

Parameters:
  • provenance_history_data (pointer to the AstroData table with the history information.) – Note this may be the output AstroData as well, so we need to handle that.
  • ad (outgoing AstroData object to add provenance history data to) –
Returns:

Return type:

none

astrodata.provenance.find_provenance_history_column_indices(ad)[source]
astrodata.provenance.provenance_summary(ad, provenance=True, provenance_history=True)[source]

Generate a pretty text display of the provenance information for an ~astrodata.core.AstroData.

This pulls the provenance and history information from a ~astrodata.core.AstroData object and formats it for readability. The primitive arguments in the history are wrapped across multiple lines to keep the overall width manageable.

Parameters:
  • ad (AstroData) – Input data to read provenance from
  • provenance (bool) – True to show provenance
  • provenance_history (bool) – True to show the provenance history with associated parameters and timestamps
Returns:

Return type:

str representation of the provenance

astrodata.testing module

Fixtures to be used in tests in DRAGONS

astrodata.testing.assert_most_close(actual, desired, max_miss, rtol=1e-07, atol=0, equal_nan=True, verbose=True)[source]

Raises an AssertionError if the number of elements in two objects that are not equal up to desired tolerance is greater than expected.

Parameters:
  • actual (array_like) – Array obtained.
  • desired (array_like) – Array desired.
  • max_miss (iny) – Maximum number of mismatched elements.
  • rtol (float, optional) – Relative tolerance.
  • atol (float, optional) – Absolute tolerance.
  • equal_nan (bool, optional.) – If True, NaNs will compare equal.
  • verbose (bool, optional) – If True, the conflicting values are appended to the error message.
Raises:

AssertionError – If actual and desired are not equal up to specified precision.

astrodata.testing.assert_most_equal(actual, desired, max_miss, verbose=True)[source]

Raises an AssertionError if more than n elements in two objects are not equal. For more information, check numpy.testing.assert_equal().

Parameters:
  • actual (array_like) – The object to check.
  • desired (array_like) – The expected object.
  • max_miss (int) – Maximum number of mismatched elements.
  • verbose (bool, optional) – If True, the conflicting values are appended to the error message.
Raises:

AssertionError – If actual and desired are not equal.

astrodata.testing.assert_same_class(ad, ad_ref)[source]

Compare if two AstroData (or any subclass) have the same class.

Parameters:
astrodata.testing.astrofaker()[source]

Wrapper fixture that prevents undesired behaviour when using astrofaker.

astrodata.testing.change_working_dir(path_to_outputs)[source]

Factory that returns the output path as a context manager object, allowing easy access to the path to where the processed data should be stored.

Parameters:path_to_outputs (pytest.fixture) – Fixture containing the root path to the output files.
Returns:Enable easy change to temporary folder when reducing data.
Return type:contextmanager
astrodata.testing.compare_models(model1, model2, rtol=1e-07, atol=0.0, check_inverse=True)[source]

Check that any two models are the same, within some tolerance on parameters (using the same defaults as numpy.assert_allclose()).

This is constructed like a test, rather than returning True/False, in order to provide more useful information as to how the models differ when a test fails (and with more concise syntax).

If check_inverse is True (the default), only first-level inverses are compared, to avoid unending recursion, since the inverse of an inverse should be the supplied input model, if defined. The types of any inverses (and their inverses in turn) are required to match whether or not their parameters etc. are compared.

This function might not completely guarantee that model1 & model2 are identical for some models whose evaluation depends on class-specific parameters controlling how the array of model parameters is interpreted (eg. the orders in SIP?), but it does cover our common use of compound models involving orthonormal polynomials etc.

astrodata.testing.download_from_archive(filename, sub_path='raw_files', env_var='DRAGONS_TEST')[source]

Download file from the archive and store it in the local cache.

Parameters:
  • filename (str) – The filename, e.g. N20160524S0119.fits
  • sub_path (str) – By default the file is stored at the root of the cache directory, but using path allows to specify a sub-directory.
  • env_var (str) – Environment variable containing the path to the cache directory.
Returns:

Name of the cached file with the path added to it.

Return type:

str

astrodata.testing.get_active_git_branch()[source]

Returns the name of the active GIT branch to be used in Continuous Integration tasks and organize input/reference files.

Note: This works currently only if the remote name is “origin”, though it would be easy to adapt for other cases if needed.

Returns:str or None – the branch name could not be retrieved.
Return type:Name of the input active git branch. It returns None if
astrodata.testing.get_associated_calibrations(filename, nbias=5)[source]

Queries Gemini Observatory Archive for associated calibrations to reduce the data that will be used for testing. :param filename: Input file name :type filename: str

astrodata.testing.path_to_inputs(request, env_var='DRAGONS_TEST')[source]

PyTest fixture that returns the path to where the input files for a given test module live.

Parameters:
  • request (fixture) – PyTest’s built-in fixture with information about the test itself.
  • env_var (str) – Environment variable that contains the root path to the input data.
Returns:

Path to the input files.

Return type:

str

astrodata.testing.path_to_outputs(request, tmp_path_factory)[source]

PyTest fixture that creates a temporary folder to save tests outputs. You can set the base directory by passing the --basetemp=mydir/ argument to the PyTest call (See [Pytest - Temporary Directories and Files][1]).

[1]: https://docs.pytest.org/en/stable/tmpdir.html#temporary-directories-and-files

Returns:Path to the output data.
Return type:str
Raises:IOError – If output path does not exits.
astrodata.testing.path_to_refs(request, env_var='DRAGONS_TEST')[source]

PyTest fixture that returns the path to where the reference files for a given test module live.

Parameters:
  • request (fixture) – PyTest’s built-in fixture with information about the test itself.
  • env_var (str) – Environment variable that contains the root path to the input data.
Returns:

Path to the reference files.

Return type:

str

astrodata.wcs module

class astrodata.wcs.AffineMatrices(matrix, offset)

Bases: tuple

matrix

Alias for field number 0

offset

Alias for field number 1

class astrodata.wcs.FrameMapping(cls, description)

Bases: tuple

cls

Alias for field number 0

description

Alias for field number 1

astrodata.wcs.calculate_affine_matrices(func, shape)[source]

Compute the matrix and offset necessary of an affine transform that represents the supplied function. This is done by computing the linear matrix along all axes extending from the centre of the region, and then calculating the offset such that the transformation is accurate at the centre of the region. The matrix and offset are returned in the standard python order (i.e., y-first for 2D).

Parameters:
  • func (callable) – function that maps input->output coordinates
  • shape (sequence) – shape to use for fiducial points
Returns:

AffineMatrices(array, array)

Return type:

affine matrix and offset

astrodata.wcs.fitswcs_image(header)[source]

Make a complete transform from CRPIX-shifted pixels to sky coordinates from FITS WCS keywords. A Mapping is inserted at the beginning, which may be removed later

Parameters:header (astropy.io.fits.Header or dict) – FITS Header or dict with basic FITS WCS keywords.
astrodata.wcs.fitswcs_linear(header)[source]

Create WCS linear transforms for any axes not associated with celestial coordinates. We require that each world axis aligns precisely with only a single pixel axis.

Parameters:header (astropy.io.fits.Header or dict) – FITS Header or dict with basic FITS WCS keywords.
astrodata.wcs.fitswcs_to_gwcs(hdr)[source]

Create and return a gWCS object from a FITS header. If it can’t construct one, it should quietly return None.

astrodata.wcs.get_axes(header)[source]

Matches input with spectral and sky coordinate axes.

Parameters:header (astropy.io.fits.Header or dict) – FITS Header (or dict) with basic WCS information.
Returns:sky_inmap, spectral_inmap, unknown – indices in the output representing sky and spectral coordinates.
Return type:lists
astrodata.wcs.gwcs_to_fits(ndd, hdr=None)[source]

Convert a gWCS object to a collection of FITS WCS keyword/value pairs, if possible. If the FITS WCS is only approximate, this should be indicated with a dict entry {‘FITS-WCS’: ‘APPROXIMATE’}. If there is no suitable FITS representation, then a ValueError or NotImplementedError can be raised.

Parameters:
  • ndd (NDData) – The NDData whose wcs attribute we want converted
  • hdr (fits.Header) – A Header object that may contain some useful keywords
Returns:

dict

Return type:

values to insert into the FITS header to express this WCS

astrodata.wcs.make_fitswcs_transform(header)[source]

Create a basic FITS WCS transform. It does not include distortions.

Parameters:header (astropy.io.fits.Header or dict) – FITS Header (or dict) with basic WCS information
astrodata.wcs.model_is_affine(model)[source]

” Test a Model for affinity. This is currently done by checking the name of its class (or the class names of all its submodels)

TODO: Is this the right thing to do? We could compute the affine matrices assuming affinity, and then check that a number of random points behave as expected. Is that better?

astrodata.wcs.read_wcs_from_header(header)[source]

Extract basic FITS WCS keywords from a FITS Header.

Parameters:header (astropy.io.fits.Header) – FITS Header with WCS information.
Returns:wcs_info – A dictionary with WCS keywords.
Return type:dict