3. Reduction using API

There may be cases where you might be interested in accessing the DRAGONS’ Application Program Interface (API) directly instead of using the command line wrappers to reduce your data. In this case, you will need to access DRAGONS’ tools by importing the appropriate modules and packages.

3.1. The dataset

If you have not already, download and unpack the tutorial’s data package. Refer to Downloading the tutorial datasets for the links and simple instructions.

The dataset specific to this example is described in:

Here is a copy of the table for quick reference.

Science
S20170505S0095-110
Kshort-band, on target, 60 s
Flats
S20170505S0030-044
S20170505S0060-074
Lamp on, Kshort, for science
Lamp off, Kshort, for science
Standard star
S20170504S0114-117
Kshort, standard star, 30 s

Note

A master dark is not needed for GSAOI. The dark current is very low.

3.2. Setting up

3.2.1. Importing Libraries

We first import the necessary modules and classes:

1
2
3
4
5
import glob

from gempy.adlibrary import dataselect
from recipe_system import cal_service
from recipe_system.reduction.coreReduce import Reduce

glob is a Python built-in package. It will be used to return a list with the input file names.

dataselect will be used to create file lists for the darks, the flats and the science observations. The cal_service package is our interface with the local calibration database. Finally, the Reduce class is used to set up and run the data reduction.

3.2.2. Setting up the logger

We recommend using the DRAGONS logger. (See also Double messaging issue.)

8
9
from gempy.utils import logutils
logutils.config(file_name='gsaoi_data_reduction.log')

3.2.3. Setting up the Calibration Service

Before we continue, let’s be sure we have properly setup our calibration database and the calibration association service.

First, check that you have already a rsys.cfg file inside the ~/.geminidr/. It should contain:

[calibs]
standalone = True
database_dir = ${path_to_my_data}/gsaoiimg_tutorial/playground

This tells the system where to put the calibration database. This database will keep track of the processed calibrations as we add them to it.

Note

The tilde (~) in the path above refers to your home directory. Also, mind the dot in .geminidr.

The calibration database is initialized and the calibration service is configured as follow:

10
11
12
13
14
caldb = cal_service.CalibrationService()
caldb.config()
caldb.init()

cal_service.set_calservice()

The calibration service is now ready to use. If you need more details, check the caldb section in the Recipe System Users Manual.

3.3. Create list of files

Next step is to create lists of files that will be used as input to each of the data reduction steps. Let us start by creating a list of all the FITS files in the directory ../playdata/.

15
16
all_files = glob.glob('../playdata/*.fits')
all_files.sort()

Before you carry on, you might want to do print(all_files) to check if they were properly read.

Now we can use the all_files list as an input to select_data(). The dataselect.select_data() function signature is:

select_data(inputs, tags=[], xtags=[], expression='True')

3.3.1. A list for the flats

Now you must create a list of FLAT images for each filter. The expression specifying the filter name is needed only if you have data from multiple filters. It is not really needed in this case.

17
18
19
20
21
22
list_of_flats_Ks = dataselect.select_data(
     all_files,
     ['FLAT'],
     [],
     dataselect.expr_parser('filter_name=="Kshort"')
)

3.3.2. A list for the standard star

For the standard star selection, we use:

23
24
25
26
27
28
list_of_std_stars = dataselect.select_data(
    all_files,
    [],
    [],
    dataselect.expr_parser('observation_class=="partnerCal"')
)

Here, we are passing empty lists to the second and the third argument since we do not need to use the Tags for selection nor for exclusion.

3.3.3. A list for the science data

Finally, the science data can be selected using:

29
30
31
32
33
34
list_of_science_images = dataselect.select_data(
    all_files,
    [],
    [],
    dataselect.expr_parser('(observation_class=="science" and exposure_time==60.)')
)

The exposure time is not really needed in this case since there are only 60-second frames, but it shows how you could have two selection criteria in the expression.

3.4. Create a Master Flat Field

As explained on the calibration webpage for GSAOI, dark subtraction is not necessary since the dark noise level is very low. Therefore, we can go ahead and start with the master flat.

A GSAOI K-short master flat is created from a series of lamp-on and lamp-off exposures. Each flavor is stacked, then the lamp-off stack is subtracted from the lamp-on stack and the result normalized.

We create the master flat field and add it to the calibration manager as follow:

35
36
37
38
39
reduce_flats = Reduce()
reduce_flats.files.extend(list_of_flats_Ks)
reduce_flats.runr()

caldb.add_cal(reduce_flats.output_filenames[0])

Once runr() is finished, we add the master flat to the calibration manager (line 38).

3.5. Reduce Standard Star

The standard star is reduced essentially the same way as the science target (next section). The processed flat field that we added above to the local calibration database will be fetched automatically.

40
41
42
reduce_std = Reduce()
reduce_std.files.extend(list_of_std_stars)
reduce_std.runr()

For stacking the sky-subtracted standard star images, the easiest way is probably to use disco_stu’s command line interface as follow:

$ disco `dataselect *_skyCorrected.fits --expr='observation_class=="partnerCal"'`

If you really want or need to run disco_stu’s API, see the example later in this chapter where we do just that for the science frames.

3.6. Reduce the Science Images

The science observation uses a dither-on-target with offset-to-sky pattern. The sky frames from the offset-to-sky position will be automatically detected and used for the sky subtraction.

The master flat will be retrieved automatically from the local calibration database.

We use similar commands as before to initiate a new reduction to reduce the science data:

43
44
45
46
reduce_target = Reduce()
reduce_target.files.extend(list_of_science_images)
reduce_target.uparms.append(('skyCorrect:offset_sky', False))
reduce_target.runr()

3.7. Stack Sky-subtracted Science Images

The final step is to stack the images. For that, you must be aware that GSAOI images are highly distorted and that this distortion must be corrected before stacking. The tool for distortion correction and image stacking is disco_stu.

Note

disco_stu is installed with conda when the standard Gemini software installation instructions are followed. To install after the fact:

conda install disco_stu

This package was created to be accessed via command line (See the Stack Sky-Subtracted Science Images command line section). Because of that, the API is not the most polished, and using it requires a fair number of steps. If you can use the command line interface, it is recommended that you do so. If not, then let’s get to work.

First, let’s import some libraries:

46
47
48
49
from collections import namedtuple

from disco_stu import disco
from disco_stu.lookups import general_parameters as disco_pars

Then we need to create a special class using namedtuple(). This object will hold information about matching the objects between files:

50
51
52
53
54
55
56
MatchInfo = namedtuple(
    'MatchInfo', [
        'offset_radius',
        'match_radius',
        'min_matches',
        'degree'
        ])

We now create objects of MatchInfo class:

57
58
59
60
61
62
63
64
65
66
67
68
69
object_match_info = MatchInfo(
    disco_pars.OBJCAT_ALIGN_RADIUS[0],
    disco_pars.OBJCAT_ALIGN_RADIUS[1],
    None,
    disco_pars.OBJCAT_POLY_DEGREE
)

reference_match_info = MatchInfo(
    disco_pars.REFCAT_ALIGN_RADIUS[0],
    disco_pars.REFCAT_ALIGN_RADIUS[1],
    disco_pars.REFCAT_MIN_MATCHES,
    disco_pars.REFCAT_POLY_DEGREE
)

Finally, we call the disco() function and pass the arguments.

70
71
72
73
74
75
76
77
disco.disco(
    infiles=reduce_target.output_filenames,
    output_identifier="my_Kshort_stack",
    objmatch_info=object_match_info,
    refmatch_info=reference_match_info,
    pixel_scale=disco_pars.PIXEL_SCALE,
    skysub=False,
)

This function has many other parameters that can be used to customize this step but further details are out of the scope of this tutorial.