5. Metadata and Headers¶
Try it yourself
Download the data package (Try it yourself) if you wish to follow along and run the examples. Then
$ cd <path>/ad_usermanual/playground $ python
You need to import Astrodata and the Gemini instrument configuration package.
>>> import astrodata >>> import gemini_instruments
5.1. Astrodata Descriptors¶
We show in this chapter how to use the Astrodata Descriptors. But first let’s explain what they are.
Astrodata Descriptors provide a “header-to-concept” mapping that allows the
user to access header information from a unique interface, regardless of
which instrument the dataset is from. Like for the Astrodata Tags, the
mapping is coded in a configuration package separate from core Astrodata.
For Gemini instruments, that package is named
For example, if the user is interested to know the effective filter used
for an observation, normally one needs to know which specific keyword or
set of keywords to look at for that instrument. However, once the concept
of “filter” is coded as a Descriptor, the user only needs to call the
filter_name() descriptor to retrieve the information.
The Descriptors are closely associated with the Astrodata Tags. In fact,
they are implemented in the same
AstroData class as the tags. Once
AstroData class is selected (upon opening the file), all
the tags and descriptors for that class are defined. For example, all the
descriptor functions of GMOS data, ie. the functions that map a descriptor
concept to the actual header content, are defined in the
This is all completely transparent to the user. One simply opens the data file and all the descriptors are ready to be used.
Of course if the Descriptors have not been implemented for that specific data, they will not work. They should all be defined for Gemini data. For other sources, the headers can be accessed directly, one keyword at a time. This type of access is discussed below. This is also useful when the information needed is not associated with one of the standard descriptors.
To get the list of descriptors available for an
>>> ad = astrodata.open('../playdata/N20170609S0154.fits') >>> ad.descriptors ('airmass', 'amp_read_area', 'ao_seeing', ... ...)
Most Descriptor names are readily understood, but one can get a short description of what the Descriptor refers to by calling the Python help function. For example:
>>> help(ad.airmass) >>> help(ad.filter_name)
The full list of standard descriptors is available in the Appendix List of Gemini Standard Descriptors.
5.2. Accessing Metadata¶
5.2.1. Accessing Metadata with Descriptors¶
Whenever possible the Descriptors should be used to get information from
headers. This allows for maximum re-usability of the code as it will then
work on any datasets with an
Here are a few examples using Descriptors:
>>> ad = astrodata.open('../playdata/N20170609S0154.fits') >>> #--- print a value >>> print('The airmass is : ', ad.airmass()) >>> #--- use a value to control the flow >>> if ad.exposure_time() < 240.: ... print('This is a short exposure.') ... else: ... print('This is a long exposure.') ... >>> #--- multiply all extensions by their respective gain >>> for ext, gain in zip(ad, ad.gain()): ... ext *= gain ... >>> #--- do arithmetics >>> fwhm_pixel = 3.5 >>> fwhm_arcsec = fwhm_pixel * ad.pixel_scale()
The return values for Descriptors depend on the nature of the information
being requested and the number of extensions in the
When the value has words, it will be string, if it is a number
it will be a float or an integer.
The dataset used in this section has 4 extensions. When the descriptor
value can be different for each extension, the descriptor will return a
>>> ad.airmass() 1.089 >>> ad.gain() [2.03, 1.97, 1.96, 2.01] >>> ad.filter_name() 'open1-6&g_G0301'
Some descriptors accept arguments. For example:
>>> ad.filter_name(pretty=True) 'g'
A full list of standard descriptors is available in the Appendix List of Gemini Standard Descriptors.
5.2.2. Accessing Metadata Directly¶
Not all header content is mapped to Descriptors, nor should it. Direct access is available for header content falling outside the scope of the descriptors.
One important thing to keep in mind is that the PHU (Primary Header Unit) and
the extension headers are accessed slightly differently. The attribute
phu needs to be used for the PHU, and
hdr for the extension headers.
Here are some examples of direct header access:
>>> ad = astrodata.open('../playdata/N20170609S0154.fits') >>> #--- Get keyword value from the PHU >>> ad.phu['AOFOLD'] 'park-pos.' >>> #--- Get keyword value from a specific extension >>> ad.hdr['CRPIX1'] 511.862999160781 >>> #--- Get keyword value from all the extensions in one call. >>> ad.hdr['CRPIX1'] [511.862999160781, 287.862999160781, -0.137000839218696, -224.137000839219]
5.2.3. Whole Headers¶
Entire headers can be retrieved as
>>> ad = astrodata.open('../playdata/N20170609S0154.fits') >>> type(ad.phu) <class 'astropy.io.fits.header.Header'> >>> type(ad.hdr) <class 'astropy.io.fits.header.Header'>
In interactive mode, it is possible to print the headers on the screen as follows:
>>> ad.phu SIMPLE = T / file does conform to FITS standard BITPIX = 16 / number of bits per data pixel NAXIS = 0 / number of data axes .... >>> ad.hdr XTENSION= 'IMAGE ' / IMAGE extension BITPIX = 16 / number of bits per data pixel NAXIS = 2 / number of data axes ....
5.3. Updating, Adding and Deleting Metadata¶
Header cards can be updated, added to, or deleted from the headers. The PHU
and the extensions headers are again accessed in a mostly identical way
>>> ad = astrodata.open('../playdata/N20170609S0154.fits')
Add and update a keyword, without and with comment:
>>> ad.phu['NEWKEY'] = 50. >>> ad.phu['NEWKEY'] = (30., 'Updated PHU keyword') >>> ad.hdr['NEWKEY'] = 50. >>> ad.hdr['NEWKEY'] = (30., 'Updated extension keyword')
Delete a keyword:
>>> del ad.phu['NEWKEY'] >>> del ad.hdr['NEWKEY']
5.4. World Co-ordinate System attribute¶
wcs of an extension’s
nddata attribute (eg.
see Pixel Data) may contain an instance of
astropy.wcs.WCS (a standard
FITS WCS object) or
gwcs.wcs.WCS (a “Generalized WCS” or gWCS object). These both define a transformation
between array indices and some other co-ordinate system such as “World”
co-ordinates (see APE 14). GWCS allows
multiple, almost arbitrary co-ordinate mappings from different calibration
steps (eg. CCD mosaicking, distortion correction & wavelength calibration) to
be combined in a single, reversible transformation chain — but this
information cannot all be represented as a FITS standard WCS. If a gWCS object
is defined here, it gets stored as a table extension named ‘WCS’ when the
AstroData instance is saved to a file (with the same EXTVER as the
corresponding ‘SCI’ array). This is independent of any WCS information already
stored in the FITS headers. The representation in the table is produced using
ASDF, with one line of text per row. Likewise,
when the file is re-opened, the gWCS object gets recreated in
wcs from the
In future, it is intended that the
wcs attribute will get populated from
standard FITS headers where there is no overriding ‘WCS’ table extension and
will get saved to standard FITS headers when its type is
Also, where a gWCS object is used, a discrete sampling of the World co-ordinate
values will be stored as part of the FITS WCS, following Greisen et al. (2006), S6 (in addition to the
definitive ‘WCS’ table), allowing standard FITS readers to report accurate
World co-ordinates for each pixel.
5.5. Adding Descriptors [Advanced Topic]¶
For proper and complete instructions on how to create Astrodata Descriptors, the reader is invited to refer to the Astrodata Programmer Manual. Here we provide a simple introduction that might help some readers better understand Astrodata Descriptors, or serve as a quick reference for those who have written Astrodata Descriptors in the past but need a little refresher.
The Astrodata Descriptors are defined in an
AstroData class. The
AstroData class specific to an instrument is located in a separate
package, not in
astrodata. For example, for Gemini instruments, all the
AstroData classes are contained in the
An Astrodata Descriptor is a function within the instrument’s
class. The descriptor function is distinguished from normal functions by
@astro_data_descriptor decorator to it. The descriptor
function returns the value(s) using a Python type,
list; it depends on the value being returned. There is no
special “descriptor” type.
Here is an example of code defining a descriptor:
class AstroDataGmos(AstroDataGemini): ... @astro_data_descriptor def detector_x_bin(self): def _get_xbin(b): try: return int(b.split()) except (AttributeError, ValueError): return None binning = self.hdr.get('CCDSUM') if self.is_single: return _get_xbin(binning) else: xbin_list = [_get_xbin(b) for b in binning] # Check list is single-valued return xbin_list if xbin_list == xbin_list[::-1] else None
This descriptor returns the X-axis binning as a integer when called on a single extension, or an object with only one extension, for example after the GMOS CCDs have been mosaiced. If there are more than one extensions, it will return a Python list or an integer if the binning is the same for all the extensions.
Gemini has defined a standard list of descriptors that should be defined one way or another for each instrument to ensure the re-usability of our algorithms. That list is provided in the Appendix List of Gemini Standard Descriptors.
For more information on adding to Astrodata, see the Astrodata Programmer Manual.