3. Reduction using API¶
There may be cases where you would be interested in accessing the DRAGONS’ Application Program Interface (API) directly instead of using the command line wrappers to reduce your data. Here we show you how to do the same reduction we did in the previous chapter but using the API.
3.1. The dataset¶
If you have not already, download and unpack the tutorial’s data package. Refer to Downloading the tutorial datasets for the links and simple instructions.
The dataset specific to this example is described in:
Here is a copy of the table for quick reference.
10 s, i-band
40 to 16 s, i-band
3.2. Setting Up¶
3.2.1. Importing Libraries¶
We first import the necessary modules and classes:
1 2 3 4 5
import glob from gempy.adlibrary import dataselect from recipe_system import cal_service from recipe_system.reduction.coreReduce import Reduce
print_function is for compatibility with the Python 2.7 print
statement. If you are working with Python 3, it is not needed, but importing
it will not break anything.
dataselect will be used to create file lists for the
darks, the flats and the science observations. The
cal_service package is our interface with the local
calibration database. Finally, the
Reduce class is used to set up
and run the data reduction.
3.2.2. Setting up the logger¶
We recommend using the DRAGONS logger. (See also Double messaging issue.)
from gempy.utils import logutils logutils.config(file_name='gmos_data_reduction.log')
3.2.3. Setting up the Calibration Service¶
Before we continue, let’s be sure we have properly setup our calibration database and the calibration association service.
First, check that you have already a
rsys.cfg file inside the
~/.geminidr/. It should contain:
[calibs] standalone = True database_dir = /path_to_my_data/gmosimg_tutorial_api/playground
This tells the system where to put the calibration database. This database will keep track of the processed calibrations as we add them to it.
The tilde (
~) in the path above refers to your home directory.
Also, mind the dot in
The calibration database is initialized and the calibration service is configured as follow:
10 11 12 13 14
caldb = cal_service.CalibrationService() caldb.config() caldb.init() cal_service.set_calservice()
The calibration service is now ready to use. If you need more details, check the Using the caldb API in the Recipe System User’s Manual .
3.3. Create list of files¶
The next step is to create lists of files that will be used as input to each of the
data reduction steps. Let us start by creating a
list of all the
FITS files in the directory
all_files = glob.glob('../playdata/*.fits') all_files.sort()
sort() method simply re-organize the list with the file names
and is an optional step. Before you carry on, you might want to do
print(all_files) to check if they were properly read.
Now we can use the
list as an input to
dataselect.select_data() function signature is:
select_data(inputs, tags=, xtags=, expression='True')
3.3.1. List of Biases¶
Let us, now, select the files that will be used to create a master bias:
17 18 19 20 21
list_of_biases = dataselect.select_data( all_files, ['BIAS'],  )
Note the empty list
 in line 20. This positional argument receives a list
of tags that will be used to exclude any files with the matching tag from our
selection (i.e., equivalent to the
3.3.2. List of Flats¶
Next we create a list of twilight flats for each filter. The expression specifying the filter name is needed only if you have data from multiple filters. It is not really needed in this case.
22 23 24 25 26 27
list_of_flats = dataselect.select_data( all_files, ['FLAT'], , dataselect.expr_parser('filter_name=="i"') )
3.3.3. List of Science Data¶
Finally, the science data can be selected using:
27 28 29 30 31 32
list_of_science = dataselect.select_data( all_files, , ['CAL'], dataselect.expr_parser('(observation_class=="science" and filter_name=="i")') )
Here we left the
tags argument as an empty list and passed the tag
'CAL' as an exclusion tag through the
We also added a fourth argument which is not necessary for our current dataset
but that can be useful for others. It contains an expression that has to be
expr_parser(), and which ensures
that we are getting science frames obtained with the i-band filter.
3.4. Make Master Bias¶
We create the master bias and add it to the calibration manager as follow:
33 34 35 36 37
reduce_bias = Reduce() reduce_bias.files.extend(list_of_biases) reduce_bias.runr() caldb.add_cal(reduce_bias.output_filenames)
Reduce class is our reduction
“controller”. This is where we collect all the information necessary for
the reduction. In this case, the only information necessary is the list of
input files which we add to the
files attribute. The
runr() method is where the
recipe search is triggered and where it is executed.
runr() is finished, we add the master bias to the calibration
manager (line 37).
3.5. Make Master Flat¶
We create the master flat field and add it to the calibration database as follow:
38 39 40 41 42
reduce_flats = Reduce() reduce_flats.files.extend(list_of_flats) reduce_flats.runr() caldb.add_cal(reduce_flats.output_filenames)
3.6. Make Master Fringe Frame¶
3.7. Reduce Science Images¶
We use similar statements as before to initiate a new reduction to reduce the science data:
43 44 45
reduce_science = Reduce() reduce_science.files.extend(list_of_science) reduce_science.runr()
The output stack units are in electrons (header keyword BUNIT=electrons). The output stack is stored in a multi-extension FITS (MEF) file. The science signal is in the “SCI” extension, the variance is in the “VAR” extension, and the data quality plane (mask) is in the “DQ” extension.