The procedure to create master darks from individual exposures is a command-line script with parameters. It allows to reduce all DDDDE files with different exposure times in the same directory. File sequences taken during a UT data change will automatically be found and saved under in the UT date folder where the sequence started.
Typically 5 frames per exposure time are taken in a sequence. The script uses a sigma-clipped mean to combine those to increase the SNR and remove outliers.
Darks taken with a manually set ND filter should be processed individually. These are necessary when the exposure time is not matching the automated ND filter chosen by the instrument control software. The automated ND filter setting is using the red expsore time as the basis of the ND filter setting, as the brightness of the etalon is higher in the red arm than in the blue arm. Different exposure times are sometimes chosen to optimize the duty cycle during time series by choosing a blue exposure time 40s longer than the red exposure time, the difference in read time between the channels.
2-5 DDDDE frames per exposure time. No light any fibre except for the simultaneous calibration fiber.
2D data frame(s) with processed darks saved in specified directory in subfolder ./YYYYMMDD/darks/ with filename
YYYYMMDDTHH_masterdark_type_TTTT.fits or
YYYYMMDDTHH_masterdark_ndxxx_type_TTTT.fits
where date/time (YYYYMMDDHH), and exposure time (TTTT) are from the first input fits file in the input list. If a specific ND filter wheel setting was specified, the xxx is the angle of the filterwheel. Type is either 'mean' for the master dark or (when optional outputs are selected) 'min' and 'sigma'.
From the base directory (maroonx_reduce) call: PYTHONPATH=${PWD} python reduce/recipes/make_master_darks.py
with the following parameters:
-dd DATA_DIRECTORY, --data_directory DATA_DIRECTORY Directory for raw files, e.g., '/data/MaroonX_spectra/'.
-d DATE, --date DATE UTC date of observation, e.g. '20200901'.
-t EXPTIME, --exptime EXPTIME Exposure time in sec, e.g. '300'.
-nd ND_SETTING, --nd_setting ND_SETTING ND Filter wheel setting in deg
-od OUTPUT_DIRECTORY, --output_directory OUTPUT_DIRECTORY Directory stump for output files. Output will go into subfolder ./<date>/darks/
unless a date is already specified in the name.
Default = '/data/MaroonX_spectra_reduced/Maroonx_masterframes/'.
-l FILELIST --filelist FILELIST File with list of input files (incl. full path). Supersedes file collection by date and exposure time.
Example 1: PYTHONPATH=${PWD} python reduce/recipes/make_master_darks.py -dd '/data2/MaroonX_spectra/' -d '20201120'
Example call collects all blue and red frames from /data2/MaroonX_spectra/20201120 and saves the results in '/data2/MaroonX_spectra_reduced/Maroonx_masterframes/20201120/darks/20201120T16_masterdark_mean_DDDDE_b_tttt.fits' and 20201120T16_masterdark_mean_DDDDE_r_tttt.fits' where tttt are the different exposure times found in the directory.
Example 2: PYTHONPATH=${PWD} python reduce/recipes/make_master_darks.py -dd '/data2/MaroonX_spectra/' -d '20201130' -t 1200 -nd 116.7 -od '/data2/MaroonX_spectra_reduced/Maroonx_masterframes/'
Example call collects all blue and red frames with 1200 sec exposure time for a ND filter wheel setting of 116.7 (+/- 0.5deg) from /data2/MaroonX_spectra/20201130 and saving the result in '/data2/MaroonX_spectra_reduced/Maroonx_masterframes/20201130/darks/20201130T16_masterdark_mean_nd117_DDDDE_b_1200.fits' and 20201120T16_masterdark_nd117_mean_DDDDE_r_1200.fits'
Blue and red frames are reduced separately. The input list is checked for consistent ND filter setting and frames not matching the first frame in the list will be disregarded. Sigma-clipped mean, median, and minimum values are computed for each pixel through the stack of accepted input frames. Pixels in the the sigma-clipped frame that did not 'clean' correctly (e.g., have overlapping cosmic ray hits in different input frames) are replaced by the minimum value. The criterion for this is hardcoded: The difference between sigma-clipped mean and minimum must be larger than 2x the global median and larger than 30% of the minimum. Both the mean and the difference of those pixels must also be greater than 10 counts.
Optional outputs (currently hardcoded) are the minimum value image and a sigma map, the ratio between the measured standard deviation for each pixel across the stack and the combination of read noise and photon-noise. The latter could be used to check for excess noise.
Statistics are output on the command line. These could be either written into the fits header and/or into a log file.
The optional outputs (minimum and sigma map) should be made command line options instead of hard coded.
QC: Procedure should raise a flag if it fails (e.g., not enough input frames) or if certain thresholds in the statistics are exceeded (e.g., certain number of replaced pixels)