2.5. Metadata and Headers
Try it yourself
Download the data package (Try it yourself) if you wish to follow along and run the examples. Then
$ cd <path>/ad_usermanual/playground
$ python
You need to import Astrodata and the Gemini instrument configuration package.
>>> import astrodata
>>> import gemini_instruments
2.5.1. Astrodata Descriptors
We show in this chapter how to use the Astrodata Descriptors. But first let’s explain what they are.
Astrodata Descriptors provide a “header-to-concept” mapping that allows the
user to access header information from a unique interface, regardless of
which instrument the dataset is from. Like for the Astrodata Tags, the
mapping is coded in a configuration package separate from core Astrodata.
For Gemini instruments, that package is named gemini_instruments
.
For example, if the user is interested to know the effective filter used
for an observation, normally one needs to know which specific keyword or
set of keywords to look at for that instrument. However, once the concept
of “filter” is coded as a Descriptor, the user only needs to call the
filter_name()
descriptor to retrieve the information.
The Descriptors are closely associated with the Astrodata Tags. In fact,
they are implemented in the same AstroData
class as the tags. Once
the specific AstroData
class is selected (upon opening the file), all
the tags and descriptors for that class are defined. For example, all the
descriptor functions of GMOS data, ie. the functions that map a descriptor
concept to the actual header content, are defined in the AstroDataGmos
class.
This is all completely transparent to the user. One simply opens the data file and all the descriptors are ready to be used.
Note
Of course if the Descriptors have not been implemented for that specific data, they will not work. They should all be defined for Gemini data. For other sources, the headers can be accessed directly, one keyword at a time. This type of access is discussed below. This is also useful when the information needed is not associated with one of the standard descriptors.
To get the list of descriptors available for an AstroData
object:
>>> ad = astrodata.open('../playdata/N20170609S0154.fits')
>>> ad.descriptors
('airmass', 'amp_read_area', 'ao_seeing', ...
...)
Most Descriptor names are readily understood, but one can get a short description of what the Descriptor refers to by calling the Python help function. For example:
>>> help(ad.airmass)
>>> help(ad.filter_name)
The full list of standard descriptors is available in the Appendix List of Gemini Standard Descriptors.
2.5.2. Accessing Metadata
2.5.2.1. Accessing Metadata with Descriptors
Whenever possible the Descriptors should be used to get information from
headers. This allows for maximum re-usability of the code as it will then
work on any datasets with an AstroData
class.
Here are a few examples using Descriptors:
>>> ad = astrodata.open('../playdata/N20170609S0154.fits')
>>> #--- print a value
>>> print('The airmass is : ', ad.airmass())
The airmass is : 1.089
>>> #--- use a value to control the flow
>>> if ad.exposure_time() < 240.:
... print('This is a short exposure.')
... else:
... print('This is a long exposure.')
This is a short exposure.
>>> #--- multiply all extensions by their respective gain
>>> for ext, gain in zip(ad, ad.gain()):
... ext *= gain
>>> #--- do arithmetics
>>> fwhm_pixel = 3.5
>>> fwhm_arcsec = fwhm_pixel * ad.pixel_scale()
The return values for Descriptors depend on the nature of the information
being requested and the number of extensions in the AstroData
object.
When the value has words, it will be string, if it is a number
it will be a float or an integer.
The dataset used in this section has 4 extensions. When the descriptor
value can be different for each extension, the descriptor will return a
Python list.
>>> ad.airmass()
1.089
>>> ad.gain()
[2.03, 1.97, 1.96, 2.01]
>>> ad.filter_name()
'open1-6&g_G0301'
Some descriptors accept arguments. For example:
>>> ad.filter_name(pretty=True)
'g'
A full list of standard descriptors is available in the Appendix List of Gemini Standard Descriptors.
2.5.2.2. Accessing Metadata Directly
Not all header content is mapped to Descriptors, nor should it. Direct access is available for header content falling outside the scope of the descriptors.
One important thing to keep in mind is that the PHU (Primary Header Unit) and
the extension headers are accessed slightly differently. The attribute
phu
needs to be used for the PHU, and hdr
for the extension headers.
Here are some examples of direct header access:
>>> ad = astrodata.open('../playdata/N20170609S0154.fits')
>>> #--- Get keyword value from the PHU
>>> ad.phu['AOFOLD']
'park-pos.'
>>> #--- Get keyword value from a specific extension
>>> ad[0].hdr['CRPIX1']
511.862999160781
>>> #--- Get keyword value from all the extensions in one call.
>>> ad.hdr['CRPIX1']
[511.862999160781, 287.862999160781, -0.137000839218696, -224.137000839219]
2.5.2.3. Whole Headers
Entire headers can be retrieved as fits
Header
objects:
>>> ad = astrodata.open('../playdata/N20170609S0154.fits')
>>> type(ad.phu)
<class 'astropy.io.fits.header.Header'>
>>> type(ad[0].hdr)
<class 'astropy.io.fits.header.Header'>
In interactive mode, it is possible to print the headers on the screen as follows:
>>> ad.phu
SIMPLE = T / file does conform to FITS standard
BITPIX = 16 / number of bits per data pixel
NAXIS = 0 / number of data axes
....
>>> ad[0].hdr
XTENSION= 'IMAGE ' / IMAGE extension
BITPIX = 16 / number of bits per data pixel
NAXIS = 2 / number of data axes
....
2.5.3. Updating, Adding and Deleting Metadata
Header cards can be updated, added to, or deleted from the headers. The PHU
and the extensions headers are again accessed in a mostly identical way
with phu
and hdr
, respectively.
>>> ad = astrodata.open('../playdata/N20170609S0154.fits')
Add and update a keyword, without and with comment:
>>> ad.phu['NEWKEY'] = 50.
>>> ad.phu['NEWKEY'] = (30., 'Updated PHU keyword')
>>> ad[0].hdr['NEWKEY'] = 50.
>>> ad[0].hdr['NEWKEY'] = (30., 'Updated extension keyword')
Delete a keyword:
>>> del ad.phu['NEWKEY']
>>> del ad[0].hdr['NEWKEY']
2.5.4. World Co-ordinate System attribute
The wcs
of an extension’s nddata
attribute (eg. ad[0].nddata.wcs
;
see Pixel Data) is stored as an instance of astropy.wcs.WCS
(a
standard FITS WCS object) or gwcs.WCS
(a “Generalized WCS” or gWCS object). This defines a transformation
between array indices and some other co-ordinate system such as “World”
co-ordinates (see APE 14). GWCS allows
multiple, almost arbitrary co-ordinate mappings from different calibration
steps (eg. CCD mosaicking, distortion correction & wavelength calibration) to
be combined in a single, reversible transformation chain — but this
information cannot always be represented as a FITS standard WCS. If a gWCS
object is too complex to be defined by the basic FITS keywords, it gets stored
as a table extension named ‘WCS’ when the AstroData
instance is saved to a
file (with the same EXTVER as the corresponding ‘SCI’ array) and the FITS
header keywords are updated to provide an approximation to the true WCS and an
additional keyword FITS-WCS
is added with the value ‘APPROXIMATE’.
The representation in the table is produced using
ASDF, with one line of text per row. Likewise,
when the file is re-opened, the gWCS object gets recreated in wcs
from the
table. If the transformation defined by the gWCS object can be accurately
described by standard FITS keywords, then no WCS extension is created as the
gWCS object can be created from these keywords when the file is re-opened.
In future, it is intended to improve the quality of the FITS approximation using the Simple Imaging Polynomial convention (SIP) or a discrete sampling of the World co-ordinate values will be stored as part of the FITS WCS, following Greisen et al. (2006), S6 (in addition to the definitive ‘WCS’ table), allowing standard FITS readers to report accurate World co-ordinates for each pixel.
2.5.5. Adding Descriptors [Advanced Topic]
For proper and complete instructions on how to create Astrodata Descriptors, the reader is invited to refer to the Astrodata Programmer Manual. Here we provide a simple introduction that might help some readers better understand Astrodata Descriptors, or serve as a quick reference for those who have written Astrodata Descriptors in the past but need a little refresher.
The Astrodata Descriptors are defined in an AstroData
class. The
AstroData
class specific to an instrument is located in a separate
package, not in astrodata
. For example, for Gemini instruments, all the
various AstroData
classes are contained in the gemini_instruments
package.
An Astrodata Descriptor is a function within the instrument’s AstroData
class. The descriptor function is distinguished from normal functions by
applying the @astro_data_descriptor
decorator to it. The descriptor
function returns the value(s) using a Python type, int
, float
,
string
, list
; it depends on the value being returned. There is no
special “descriptor” type.
Here is an example of code defining a descriptor:
class AstroDataGmos(AstroDataGemini):
...
@astro_data_descriptor
def detector_x_bin(self):
def _get_xbin(b):
try:
return int(b.split()[0])
except (AttributeError, ValueError):
return None
binning = self.hdr.get('CCDSUM')
if self.is_single:
return _get_xbin(binning)
else:
xbin_list = [_get_xbin(b) for b in binning]
# Check list is single-valued
return xbin_list[0] if xbin_list == xbin_list[::-1] else None
This descriptor returns the X-axis binning as a integer when called on a single extension, or an object with only one extension, for example after the GMOS CCDs have been mosaiced. If there are more than one extensions, it will return a Python list or an integer if the binning is the same for all the extensions.
Gemini has defined a standard list of descriptors that should be defined one way or another for each instrument to ensure the re-usability of our algorithms. That list is provided in the Appendix List of Gemini Standard Descriptors.
For more information on adding to Astrodata, see the Astrodata Programmer Manual.