The 'drift solution' is simply the in-situ dynamical wavelength solution for the science fibers in a science frame based on etalon calibration spectra taken before or after the science frame. The default (and most accurate) method to determine a drift solution is to use the simultaneous calibration fiber to compare the etalon line positions in this fiber between an etalon calibration spectrum and a science spectrum and apply the measured drift to the etalon calibration spectrum to 'predict' the correct wavelength solution for the science fibers in the science spectrum.
The main function fit_and_apply_etalon_wls() in analyze/recipes/etalon_spline_wls_dynamic.py works on individual 1D reduced and fitted etalon and science spectra. A batch version provides functionality to automatically select the nearest calibration spectrum and process more than one file at a time in multi-threaded fashion, see discussion below.
Etalon parameter file (hdf)
1D (hdf) extracted etalon spectra from for the science and simultaneous calibration fiber (DEEEE).
1D (hdf) extracted etalon spectra from the simultaneous calibration fiber of a science frame (SOOOE).
Wavelength solution added to the input hdf file (the science frame).
Log file with ending _spline.log
Plots with ending _spline.pdf
API:
Function fit_and_apply_etalon_wls(param_file,etalon_file,science_file,fibers=[2,3,4],ref_fiber=5, p=None) in analyze/recipes/etalon_spline_wls_dynamic.py
Parameters (mandatory):
param_file Absolute path and filename containing etalon parameters
etalon_file Absolute path and filename containing reduced and fitted etalon spectra for the science fibers and the simultaneous calibration fiber
science_file Absolute path and filename containing reduced and fitted etalon spectra for the simultaneous calibration fiber
Parameters (optional):
fibers List of fibers containing etalon spectra, Default = [2,3,4]
ref_fiber Number of reference (simultaneous calibration) fiber. Default = 5
p Etalon parameters. If param_file is set to None, p must be used to provide the etalon parameters. This input is not used here.
Example: fit_and_apply_etalon_wls(
"/data/MaroonX_spectra_reduced/Maroonx_wls/202005xx/wl_combined_final_etalon_peakmodel_2020.hdf",
"/data2/MaroonX_spectra_reduced/20201120/20201120T063636Z_DEEEE_b_0060.hdf",
"/data2/MaroonX_spectra_reduced/20201120/20201120T075517Z_SOOOE_b_0300.hdf")
API example call to compute a drift per order from the simultaneous calibration fiber and apply this to the etalon positions in the science fibers to fit a spline-based wavelength solution to fibers 2,3,4 of the input etalon spectrum. Results are saved in the input science spectrum. Plots are written to
/data2/MaroonX_spectra_reduced/20201120/20201120T075517Z_SOOOE_b_0300_spline.pdf
From the base directory (maroonx_reduce) call: PYTHONPATH=${PWD} python analyze/recipes/batch_science_spline_wls_dynamic.py
with the following parameters:
-p PARAM_FILE, --param_file PARAM_FILE Full name and path of etalon parameter file.
Default = '/data /MaroonX_spectra_reduced/Maroonx_wls/
202005xx/wl_combined_final_etalon_peakmodel_2020.hdf'.
-dd DATA_DIRECTORY, --data_directory DATA_DIRECTORY Directory for hdf input files. Input files need to have extracted etalon positions.
Default = '/data2/MaroonX_spectra_reduced/'.
-d DATE, --date DATE UTC date of file, e.g. '20200901' or '202009*'.
-o OBS_TYPE, --obs_type OBS_TYPE Obs type, e.g. 'SOOOE' Default: 'SOOOE'
-c CAMERA_ARM, --camera_arm CAMERA_ARM Camera arm, e.g. 'b' or 'r'. Default: '?' for both
-t EXPTIME, --exptime EXPTIME Exposure time in sec to downselect science files, e.g. '300'. Default = '*'
-m MULTITHREAD, --multithread MULTITHREAD Multithread or only loop over files. Default = False (just loop)
-e {closest,before,after}, --etalon_choice {closest,before,after} What Etalon file to choose ('closest', 'before', 'after'). Default = 'closest'
Example: PYTHONPATH=${PWD} python analyze/recipes/batch_science_spline_wls_dynamic.py -d '20201201' -c 'b' -m True
Example call for the multi-file / multi-threaded routine. Routine first collects list of input science frames and finds a suitable etalon frame for each science file. It then
computes a drift per order from the simultaneous calibration fiber and applies this to the etalon positions in the science fibers to fit a spline-based wavelength solution to fibers 2,3,4 of the input etalon spectrum. Results are saved in the input science spectra. All files matching 'SOOOE' type and 20201201 as a date are processed.
Compared to the basic version of fit_and_apply_etalon_wls() in analyze/recipes/etalon_spline_wls.py, the one in analyze/recipes/etalon_spline_wls_dynamic.py used here applies two major steps.
It first calculates offsets in pixel space between etalon peak positions in the simultaneous calibration fiber (aka reference fiber) between the science and etalon frame. Because these offsets are noisy, a cubic spline with three knots at pixel positions 1000, 2000, and 3000 is computed per order that gives a smoothed version of the offsets (effectively the per-order drift between etalon and science frame).
These smoothed shifts are then applied to the measured etalon peak positions in the science fibers in the etalon frame. Based on these 'drift corrected' etalon positions, the algorithm computes a cubic smoothing spline with 30 equidistant knots in each science fiber/order using the global etalon parameters.
The function name fit_and_apply_etalon_wls() is used in different python scripts with different parameters and slight variations of its functionality. This might be confusing. If separate functions are indeed desirable for the different use cases (initial wavelength solution, dynamical wavelength solution, drift solution), then the routine should be renamed.
Due to the lack of spectral coverage, there are no etalon peaks for the first 500 or so pixels of fiber 5 in order 94 of the red arm. This is hard-coded which is fine as long as the spectral format doesn't change but is against the spirit of allowing the code to work even after a slight format change.
A few things are hard coded that can (should?) be made a parameter call:
Number of knots per order (see n_knots)
Sigma clipping value for outlier removal (currently set to 3.5 and 4 - see bad = np.where() and good = np.where() lines)
Number (3), location [1000,2000,3000] and degree (3 - cubic) of knots for the drift correction 'smoothing'.
QC: Procedure should raise a flag if it fails (e.g., no etalon peak data found). Currently only >>ERROR - Exception: 'NoneType' object has no attribute 'loc' is written into the log file.