SUNPIT (SUNdish PIpeline Tool) is the pipeline aimed at the imaging procedure and the data analysis of radio solar data.
SUNPIT is designed for radio data acquired with some radio telescopes of the INAF Network: the Sardinia Radio Telescope (SRT), and the Medicina Radio Telescope; this pipeline will be suitable also for the data of Noto Radio Telescope, in which an upgrading will be provided in the near future. These observations will be enhanced through the upgrading of SRT with the new receivers in Q-band (33 - 50 GHz) and W-band (75 - 116 GHz) in the context of the National Operative Programme (Programma Operativo Nazionale-PON), providing the scientific community with the instrumentation suited to the study of the Universe at high radio frequencies.
This pipeline follows the development of software for solar imaging and data analysis of active regions (ARs), performed in the framework of the SunDish Project, successfully tested: Single-Dish Imager (SDI), and SRT Single-Dish Tools (SDT) for the solar imaging, and SUNDish Active Region Analyser (SUNDARA), for the data analysis of ARs.
SUNPIT produces a complete analysis of a solar map in about one hour, saving a directory which contains images, plots, and several tables with physical information of the solar disk and ARs (brightness temperatures, fluxes, and spectral indices, with respective errors).
This pipeline - successfully tested - represents a crucial tool (1) to analyse solar images observed with the radio telescopes of the INAF Network, and (2) for the Space Weather monitoring network and forecast (soon available) along the solar cycle.
In this part we guide the user to properly reduce and analyse the solar radio data, both in mono-feed and in multi-feed approaches.
Figure 1: Diagram of SUNPIT operation. Input/Output are labelled by yellow boxes, and the software packages are indicated by green boxes.
SUNPIT is composed of three independent packages:
SDI (Single-Dish Imager), an IDL Package designed to perform continuum and spectro-polarimetric imaging, optimized for On-the-fly (OTF) scan mapping, and suitable for most receivers/backends available for INAF radio telescopes (see e.g. [1][8][13]); SDI receives in input the row acquisitions (the original subscans) obtained by INAF radio telescopes, and produces in output a calibrated solar map (Fig. 1). Details about the download and installation of SDI are available in Sect. 2.1, and the description of the imaging procedure with SDI is available in Sect. 3.1.
SDT (SRT Single-Dish Tools), a Python Package designed for the quicklook and analysis of single-dish radio data, starting from the backends present at SRT. Substantially, this package is the Python counterpart of SDI imaging tool; the download/installation procedure of SDT is described in Sect. 2.2, and the imaging procedure is explained in Sect. 3.2.
SUNDARA (SUNDish Active Region Analyser), a Python Package aimed at the automatic data analysis of solar images processed by SDI and/or SDT; SUNDARA receives in input the solar images obtained by SDI and/or SDT, and produces in output a complete analysis in a short time (about 5 minutes for each solar map), saving a directory containing images, plots and several tables with physical information (brightness temperatures, fluxes and spectral indices, with respective errors) of the ARs detected in all the solar maps (Fig. 1). The download and installation procedures of SUNDARA are described in detail in Sect. 2.3, and the description of the data analysis is available in Sect. 4.
These packages are compatible with the following characteristics:
Ubuntu 16.04 (64-bit PC desktop image) - or macOS High Sierra 10.13.6 - or more recent versions as operating system; GNU/Linux environment can be installed also in the Windows Subsystem for Linux (WSL) directly on Windows, without the overhead of a traditional virtual machine or dual-boot setup;
4 GB or more of RAM (16 GB recommended);
10 GB or more of free disk space (necessary for both the installation and the data output);
IDL 8.X.X or more (for SDI);
Python 3.7.X or more (for SDT and SUNDARA).
Finally, the user is also free to download the preferred package of SUNPIT - among SDI, SDT, and SUNDARA - on the basis of the type of analysis which the user wishes to perform. For example, in case of solar imaging procedure, the user can download only SDI and/or SDT; on the other hand, if the user has the solar maps produced by SDI or SDT and just wants to implement an AR analysis, they can download only SUNDARA.
A simplified scheme of the SUNPIT operating is shown in Fig. 1.
It is strongly recommended for the user to install Anaconda (version~4 or higher) or Miniconda (version~3 or higher) in order to manage the Python environment and its libraries.
Anaconda is a free and open-source system installer that allows to easily perform Python data science (and not only); Miniconda is a small version of Anaconda.
The great advantage of this toolkit is the possibility to work with several Python environments allowing - for each of them - to install, update, and manage packages and libraries without interference with other packages and/or libraries installed in the operating system.
Very briefly, once Anaconda (or Miniconda) is installed, following this Installation Procedure (preferably the "regular installation mode"), the user must create and install the preferred Python environment by issuing the command conda create --name pxx python=v.v, where pxx is the name of the Python environment chosen by the user, and v.v is the selected version of Python (recommended: pxx = p39 and v.v = 3.9); for further details, please see the "Managing environments" and "Managing Python" sections at this link. Later, the user must access to the Python environment pxx through the command conda activate pxx (to deactivate the environment, please type the command conda deactivate), in order to use it.
The user can download and manage packages and libraries through the conda management system (see this link).
SUNPIT can be used to process and analyse the solar data obtained with SRT and the Medicina dish both in the "mono-feed mode" and in the "multi-feed mode".
In the first case, this pipeline considers only the reference feed of the receiver, usually the central one, for the analysis; in the second case SUNPIT uses the data recorded with all the feeds available for the receiver (this multi-feed tool in SUNDARA for SDT solar images is in test phase, and therefore we advise to use this procedure with caution).
The multi-feed imaging procedure improves the exposure of the Sun as many times as the number of employed feeds.
To date the solar observations are available in K-band (18 - 26.5 GHz), in particular for the 7-feed receiver at SRT [16], and the dual-feed receiver at Medicina.
The 7-feed receiver at SRT is composed by a central feed (please note that the 19-feed Q-band receiver will have a central feed, as opposed to the case of the 16-feed (4x4 pixels) W-band receiver), surrounded by six lateral feeds arranged in hexagonal configuration (in the Gregorian focal plane); the central feed is labelled as 0, while the lateral feeds are labelled as 1 - 6. This translates into 14 sections/data streams in the FITS files, as each feed observes in two polarisations (left-hand circular polarisation, LCP, and right-hand circular polarisation, RCP).
Using the dual-feed receiver at Medicina, the solar disk is typically observed through the single feed labelled as 1. Feed 0 is saturated for the solar disk signal level, in order to better observe the fainter coronal details. This is achieved by properly setting the attenuators, a procedure depending on the schedule file produced by the users, thus exceptions might exist in the many observing sessions carried out.
Technical issues relative to the dual-feed receiver have caused some temporary changes in the recorded data structure; as of the publication date of this report, SUNPIT automatically considers them in the map-making process (starting from 6th July 2021, the feed 0 acquires only one polarisation due to a partial failure in this feed; starting from 11th January 2022 this polarisation channel is designed to observe the solar disk).
It is worth noting that the optimisation of the specific documentation - and the relative part of the software - about the handling of the feed positional information in the FITS files, used to align the data in the multi-feed imaging procedure, is currently underway.
The download and installation procedures of SUNPIT are strictly connected with the independent solar packages (SDI, SDT, and SUNDARA), which are described in the following subsections.
SDI package (v6.0_2021 is the last stable version) can be downloaded from the section "Internal Documents and Data/SUNDISH Data Analysis SW" of the SunDish Google site, under the authorisation of the SunDish PI. This version is updated with the last available ephemeris, and hence SDI works for solar maps obtained with observations up to 31th December 2021. The user, in case of imaging procedure of solar observations beyond 31th December 2021, must wait for the next versions for coming years, based on the updated ephemeris. This package is based on the programming language IDL (Interactive Data Language), a commercial software with specific INAF licence (see Section 2.1.1 for the installation procedure).
SDI is already functioning when the downloaded SDI package is located in the preferred directory path on the user’s laptop.
SDI is based on the programming language IDL (Interactive Data Language), a commercial software with a specific INAF licence; only for INAF members, further details to obtain the Software licence are available at ICT/INAF link.
In this section, we follow a procedure for the installation of IDL in a Linux environment, based on the software version (IDL v8.8) available at ICT/INAF link.
The user has to follow these steps:
download IDL v8.8 and the "astrolib" IDL library on the user's computer;
from the terminal, in the directory of the downloaded file of IDL, extract the tar.gz file through the command gunzip namefile.tar.gz (in our example, namefile = idl88-linux.tar.gz);
from the terminal, in the directory of the downloaded file of IDL, extract the .tar file obtained through the command tar -xf nomefile.tar (in our example, namefile = idl88-linux.tar), obtaining two directories (install and silent) and the installer file install.sh;
move the extracted file in the preferred directory path of the user;
from the terminal, in the final directory of the installation, type install.sh to initialise the installation procedure;
accept the License Agreement typing y;
select and enter the directory to install in IDL;
accept the License Administrator typing y;
from the terminal, in the final directory of installation, create an empty file labelled as IDL_STARTUP.pro (for example emacs IDL_STARTUP.pro or gedit IDL_STARTUP.pro), where the user must copy these lines to activate the IDL libraries:
!path=!path+':+/home/[path/IDL_library]/astrolib/pro'
[you can add other !path lines to activate other libraries]
!path=expand_path(!path)
print,'Hello [your name]!' (optional)
insert these lines in the .bashrc file to activate the patch IDL_STARTUP.pro in:
. /home/[path/installation/user]/IDL/idl88/bin/idl_setup.bash
PATH=$PATH:/home/[path/installation/user]/IDL/idl88/bin/
PATH=$PATH:/home/[path/installation/user]/IDL/
export PATH
export IDL_STARTUP=/home/[path/IDL_startup_patch]/IDL/IDL_STARTUP.pro
type source .bashrc from the terminal to save the user changes in the .bashrc file;
insert the IDL licence typing harrislicense from the terminal (for further details, see this link); for example, only for INAF members, the user could insert a "License Server", using the server provided by the ICT/INAF (further information about the INAF server are available in the "readme_INAF.txt" downloaded with the IDL file installation).
The download and installation procedures of the open-source version of the Python Package SDT are described in detail in the official SDT link.
The SUNDARA package (v1.2 is the last stable version) can be downloaded from the section "Internal Documents and Data/SUNDISH Data Analysis SW" of the SunDish Google site, under the authorisation of the SunDish PI and the SUNDARA developer.
Before using SUNDARA, the user has to follow a few simple steps:
download the directory SD_sundara_v12 on the user's computer, and place it in your preferred directory path;
from the terminal, in the directory SD_sundara_v12, install the required Python libraries for the correct execution of SUNDARA through the command-line python sundara_easysetup_lib.py;
check the presence in the directory SD_sundara_v12 of the sub-directory fits containing some calibrated solar maps (in FITS file format) - the input files of SUNDARA - already available to the user for practice. These maps, obtained during observing sessions at INAF sites of SRT and Medicina, are the output of the imaging procedure with SDI and the filename have the suffix FEED_W_TI_IMAGE_SELFCAL.fits, where W indicates the feed of the receiver, TI indicates the imaging type called Total Intensity, and IMAGE_SELFCAL indicates that the image is self-calibrated (these aspects will be described in Sect. 3.3); the user can download other solar maps (with the same suffix and taking care to the feed, Sect. 3) from the column IMAGES (DS9 FITS) of the SunDish Archive (the access to this Archive is subject to the authorisation of the SunDish PI), but these maps must be stored in the sub-directory SD_sundara_v12/fits;
check the presence in the directory SD_sundara_v12 of an example of the SunDish Archive SUNDISH_v13.xlsx (in .xlsx Excel spreadsheet file format) already available to the user for practice with SUNDARA; this archive contains the information of the solar maps collected during the observing sessions at INAF sites of SRT and Medicina, and it is crucial for the check phase of the observing parameters of the solar maps with SUNDARA; the user can download an updated version of this archive from the SunDish Archive;
open the configuration file sundara_input.ini in the sub-directory sun_utility and check the correct directory path of (1) the downloaded SunDish Archive in the Excel spreadsheet file format (scaricato_excell, line 7, already set up with the default file SUNDISH_v13.xlsx), (2) the solar maps in FITS file format (directory_fits, line 8), and (3) the SUNDARA output files (directory_output, line 9, already set up with the default directory path SD_sundara_v12/fits); this step is crucial to the proper launch of SUNDARA.
Now, open the terminal in the directory SD_sundara_v12 and choose between two alternative approaches: (1) type the command-line python sundara_v12.py, or (2) type the command-line run sundara_v12.py in the ipython environment. Since SUNDARA requires a lot of libraries to work, this package may not work at the first start because of missing libraries; in this case, the user has to manually download these Python libraries. Once that these libraries are settled, the user can type again python sundara_v12.py in the terminal (or run sundara_v12.py in the ipython environment); in case of successful installation of these libraries, an intuitive widget appears (Fig. 4), and the user is able to start the analysis of the solar data.
For further details about SUNDARA, see the relative INAF Technical Report [10].
Solar observations with the INAF radio telescopes can be carried out following the call of proposal instructions reported on the website of the INAF radio telescopes.
On the other hand, in case the user should only analyse the solar data, already acquired in the frame of the SunDish proposal, it is necessary to contact the SunDish PI for the authorisation of the solar data download. Once the user has downloaded and stored the raw solar data (the original subscans) in a directory, the imaging procedure can begin. Solar images from both Medicina radio telescope and SRT can be obtained through the packages SDI and/or SDT (Fig. 1).
The user, inspecting the raw solar data, has to take into account that:
SRT observations - The central feed is labelled as FEED_0 in the SDI filename of the output (in FITS file format), and the lateral feeds are labelled as FEED_n, where n indicates the number of the relative feed of the receiver. The central feed in SDT is labelled as IMGFEED0 in the output image (in FITS file format), and the lateral feeds are labelled as IMGFEEDn; the data are contained in the extensions IMGFEEDn_LCP and IMGFEEDn_RCP of the FITS file, one for each polarisation.
Medicina observations - In the dual-feed K-band receiver, usually the solar disk is observed through the single feed FEED_1; on the other hand, the FEED_0 is saturated for the solar disk and suited for the corona. Starting from 6th July 2021, due to a failure in the FEED_1, the observations of the solar disk are performed by the FEED_0.
The imaging procedure can be skipped in case the user already has the solar maps obtained through SDI and/or SDT: in this scenario, see Sect. 4 for the solar data analysis with SUNDARA.
Regarding the IDL Package SDI, in the following sub-sections we describe the mono-feed and the multi-feed modes, respectively.
Figure 2: Widget of SDI aimed at both (1) the selection of the working sub-directory, and (2) the selection of the path of the raw solar data. The user must (1) search for - and select by clicking - the desired directory through the "Directories" box on middle left of the widget, (2) select the desired directory through one click in the "Files" box on middle right of the widget, and (3) click the ``OK'' button to confirm and exit. The single-dot (.) and double-dot (..) rows in the "Directories" and "Files" boxes represent the current directory of the user and the parent directory, respectively.
Figure 3: Example of histogram of counts as a function of pixels, in linear scale, produced by the self-calibration process with SDI. Counts are defined as arbitrary electronics measures of the backend, without physics sense. This value is directly proportional to the flux density (and hence the brightness temperature). (top) - Zoom-up part of this distribution, well fitted by a Gaussian (dashed line), whose peak corresponds to the RMS value of the QS brightness temperature, and the width is connected with the solar activity. (bottom) - Complete histogram of the brightness temperature distribution. The tail of the quasi-Gaussian distribution at low brightness temperature in the histogram is due to the brightness gradient of the corona.
Regarding the IDL Package SDI, in mono-feed mode the user must follow these steps for a correct imaging procedure:
Create the working directory in the user's computer, in which all the output files will be saved after the procedure. It is recommended to create sub-directories, one for each project/image/epoch.
Open the terminal, go to the SDI folder, and launch IDL from the same directory (an SDI analysis performed outside the correct folder will give an error message, resulting in the interruption of the analysis).
Launch the command-line sd_install in the IDL terminal to select the working directory (the specific sub-directory); in case of the same working directory for different solar maps, this command can be skipped (but we strongly suggest to launch this command). Later, a widget appears to the user (Fig. 2), who must (1) search for - and select by clicking - the working sub-directory through the "Directories" box on middle left of the widget, and (2) select the working sub-directory through one click in the "Files" box on middle right of the widget. Once clicked the ``OK'' button to confirm and exit from the widget, a summary pipeline instruction is displayed on the IDL terminal.
Modify the inputpars_sun_xxx file (where xxx corresponds to med and srt for Medicina and SRT images, respectively), which contains all the parameters and the information to execute the imaging procedure (to simplify the analysis, each option in the inputpars file is explained with a comment). The user can set some parameters in the inputpars file: outname which changes the prefix of the output filenames; res (line 62 for Medicina, and line 98 for SRT) and tau0 (line 109, only for SRT) to adjust the final image resolution and the opacity value (usually obtained from the skydip), respectively. Usually, the pixel size of solar images with Medicina radio telescope and SRT in K-band ranges between 0.5 and 0.8 arcmin; if after the imaging procedure black pixels appear in the image, please select a higher resolution. Please note that (1) if the variable outname (line 17) is not changed before starting a new analysis, the previous results will be overwritten, and (2) the inputpars file must be saved in the SDI folder, else the imaging procedure with SDI will give an error message, resulting in the interruption of the analysis. Only for SRT, the user must comment the line 24 (;feedmask(*)=1) and decomment the line 23 (feedmask(0)=1) in the inputpars_sun_srt file.
Launch the IDL command-line sd_init, 'inputpars_sun_xxx' (xxx = med for Medicina; xxx = srt for SRT); this command updates the imaging parameters written in the the most recent saved inputpars file; please note that the launch of this IDL command-line is necessary every time the user wants to further modify the parameters in the inputpars file.
Launch the command-line sd_sun_ql in the IDL terminal; a widget appears to the user (Fig. 2), who must (1) search for - and select by clicking - the directory path of the raw solar data (the user should select only one solar map at a time for this analysis, because at the same observing session the RA and DEC solar maps are obtained with a time difference of about two hours, compatible with the solar activity, such as the emergence of a solar flare) through the "Directories" box on middle left of the widget, and (2) select the directory path of the raw solar data through one click in the "Files" box on middle right of the widget; once clicked the ``OK'' button to confirm and exit from the widget, SDI proceeds with the imaging procedure. Solar maps obtained through SDI are calibrated according to self-calibration procedure [12], where the image histogram (counts distribution among pixels) - good modelled with a Gaussian - is compared to the quiet-Sun (QS) level, through a specific brightness reference for radio domain from [6], in order to find a count-to-Kelvin conversion factor; the user sees on screen one histogram (Fig. 3) for each polarisation channel (right-hand circular polarisation, RCP, and left-hand circular polarisation, LCP).
Once this process is successfully completed, the solar maps are stored in the working sub-directory previously selected.
The optimisation of the baseline subtraction and the relative flagging procedure with SDI is beyond the scope of the present technical note (for further details, see [1][8][7]).
Regarding the multi-feed mode with the IDL Package SDI, the user must follow these steps for a correct imaging procedure:
Create the working directory in the user's computer, in which all the output files will be saved after the procedure. It is recommended to create sub-directories, one for each project/image/epoch.
Open the terminal, go to the SDI folder, and launch IDL from the same directory (an SDI analysis performed outside the correct folder will give an error message, resulting in the interruption of the analysis).
Launch the command-line sd_install in the IDL terminal to select the working directory (the specific sub-directory); in case of the same working directory for different solar maps, this command can be skipped (but we strongly suggest to launch this command). Later, a widget appears to the user (Fig. 2), who must (1) search for - and select by clicking - the working sub-directory through the "Directories" box on middle left of the widget, and (2) select the working sub-directory through one click in the "Files" box on middle right of the widget. Once clicked the ``OK'' button to confirm and exit from the widget, a summary pipeline instruction is displayed on the IDL terminal.
Modify the inputpars_sun_xxx file (where xxx corresponds to med and srt for Medicina and SRT images, respectively), which contains all the parameters and the information to execute the imaging procedure (to simplify the analysis, each option in the inputpars file is explained with a comment). The user can set some parameters in the inputpars file: outname which changes the prefix of the output filenames; res (line 62 for Medicina, and line 98 for SRT) and tau0 (line 109, only for SRT) to adjust the final image resolution and the opacity value (usually obtained from the skydip), respectively. Usually, the pixel size of solar images with Medicina radio telescope and SRT in K-band ranges between 0.5 and 0.8 arcmin; if after the imaging procedure black pixels appear in the image, please select a higher resolution. Please note that (1) if the variable outname (line 17) is not changed before starting a new analysis, the previous results will be overwritten, and (2) the inputpars file must be saved in the SDI folder, else the imaging procedure with SDI will give an error message, resulting in the interruption of the analysis. Only for SRT, in case of multi-feed imaging procedure, currently available only for the SRT multi-feed K-band receiver (18 - 26 GHz, [8]), the user must decomment the line 24 (feedmask(*)=1) and comment the line 23 (;feedmask(0)=1) in the inputpars_sun_srt file. In the multi-feed imaging procedure, the side lobes are flattened; the evidence of this flattening is directly proportional to the number of feeds of the receiver. This imaging procedure is particularly useful if the solar map obtained through the mono-feed imaging procedure is characterised by a low quality and/or a bad baseline subtraction.
Launch the IDL command-line sd_init, 'inputpars_sun_xxx' (xxx = med for Medicina; xxx = srt for SRT); this command updates the imaging parameters written in the the most recent saved inputpars file; please note that the launch of this IDL command-line is necessary every time the user wants to further modify the parameters in the inputpars file.
Launch the command-line sd_sun_ql in the IDL terminal; a widget appears to the user (Fig. 2), who must (1) search for - and select by clicking - the directory path of the raw solar data (the user should select only one solar map at a time for this analysis, because at the same observing session the RA and DEC solar maps are obtained with a time difference of about two hours, compatible with the solar activity, such as the emergence of a solar flare) through the "Directories" box on middle left of the widget, and (2) select the directory path of the raw solar data through one click in the "Files" box on middle right of the widget; once clicked the ``OK'' button to confirm and exit from the widget, SDI proceeds with the imaging procedure. Solar maps obtained through SDI are calibrated according to self-calibration procedure [12], where the image histogram (counts distribution among pixels) - good modelled with a Gaussian - is compared to the quiet-Sun (QS) level, through a specific brightness reference for radio domain from [6], in order to find a count-to-Kelvin conversion factor; the user sees on screen one histogram (Fig. 3) for each polarisation channel (right-hand circular polarisation, RCP, and left-hand circular polarisation, LCP).
Once this process is successfully completed, the solar maps are stored in the working sub-directory previously selected.
In case of multi-feed imaging procedure with SRT, SDI produces several maps, one for each feed of the receiver (for example, one for the central feed and six more for each lateral feed in the SRT multi-feed K-band receiver [8] ).
The user must merge these maps through the IDL command-line sd_combine, following these steps:
Modify the inputpars_sun_srt file, where the user must (1) comment the line 24 (;feedmask(*)=1), (2) decomment the line 23 (feedmask(0)=1), (3) modify the MERGING section (lines 73 - 85) as appropriate.
Launch the IDL command-line sd_init, 'inputpars_sun_srt' to update the imaging parameters written in the inputpars_sun_srt file.
Launch the IDL command-line sd_combine to execute the merging of the solar maps corresponding to the feeds of the receiver.
The optimisation of the baseline subtraction and the relative flagging procedure with SDI is beyond the scope of the present technical note (for further details, see [1][8][7]).
For the imaging procedure with SDT, the user must follow these steps:
open the terminal in the directory where the data are located;
type the command-line SDTinspect */ -d to automatically organise the observations in groups, called configuration files (in .ini file format); these files are based on several parameters, such as the observing time, the backend, and the receiver. If this command-line produces many configuration files, the user can manually modify the file associated with the Sun.
open the .ini configuration file of the Sun and modify the "projection" line in projection = TAN, and the "pixel size" line in pixel_size = x, where x is the pixel resolution of the image in units of arcmin, usually ranging between 0.5 and 0.7 arcmin for INAF radio telescopes; if in the image appears black pixels, the user must select a lower resolution.
type the command-line SDTimage -c config_file.ini --quick --noplot --frame sun to produce the image whose the output filename ends with "_sun.fits". --quick and --noplot options allow to save time in the plotting procedure of the data; in addition, --destripe option works only when there are both RA and DEC maps, because it is based on the difference between RA and DEC stripes [11].
The final solar map obtained through SDT is uncalibrated; then, SUNDARA Package is able to implement an accurate image calibration. For further details about imaging procedure with SDT, see the SDT imaging tutorial. Currently the imaging procedure for Medicina radio telescope with SDT is available only for the solar maps obtained through the FEED_0 starting from 6th July 2021; the same procedure for Medicina is under development for the FEED_1 (for further details, see Sect. 3), and therefore in this case we refer to SDI for the imaging procedure (see Sect. 3.1).
At the end of the imaging procedure, the final output files are stored in the working directory (for SDI) and/or in the directory where the data are located (for SDT).
In case of SDI, the user must consider the solar map whose filename ends with FEED_W_TI_IMAGE_SELFCAL.fits, where:
W indicates the feed of the receiver, in particular W = 0 for SRT, W = 1 (or W = 0, see Sect. 3) for Medicina, and W = M for multi-feed mode;
TI indicates the imaging type called Total Intensity (this kind of imaging consists in the average between the two polarisation channels, RCP and LCP);
IMAGE_SELFCAL indicates that the image is self-calibrated, and hence the solar map is represented in units of Kelvin (only for SDI filename); otherwise, the image is uncalibrated, resulting in a solar map represented in units of counts (for further details, see [12]).
In case of SDT, the user must consider only the solar map produced in FITS format, containing all the maps for each polarisation channel and feed.
These output filenames produced by SDI and/or SDT must be renamed according to the following configuration:
SUN_TEL_YYMMDD_HHMM_XX.XGHz_Z.Zr_FEED_W_TI_IMAGE_SELFCAL.fits for SDI;
SUN_TEL_YYMMDD_HHMM_XX.XGHz_Z.Zr_SDT.fits for SDT;
where:
SUN indicates the source (the Sun);
TEL indicates the radio telescope (SRT for Sardinia Radio Telescope, MED for Medicina, and NOT for Noto), visible from the keyword ANTENNA in the primary header of the FITS file of the raw data;
YYMMDD indicates the epoch of observation (year/month/day), visible from the filename of the directory of the raw data;
HHMM indicates the starting time of observation (hours/minutes), visible from the filename of the directory of the raw data;
XX.X indicates the central observing frequency (in units of GHz). In SDI, the central observing frequency is visible through the launch of the command-line sd_importfits in the IDL terminal; in SDT, this frequency is calculated as freq_0 + 0.5 d_freq, where freq_0 is the starting frequency (in units of GHz) and d_freq is the bandwidth (in units of GHz); freq_0 and d_freq are visible in the header of the FITS file of the SDT final solar image through the keys frequency and bandwidth, respectively.
Z.Z indicates the pixel size (in units of arcmin);
SDT indicates that SDT Python Package has been used in the imaging procedure; otherwise, this part is undeclared.
These FITS files, organised with a correct filename, are crucial to execute the data analysis with SUNDARA.
Figure 4: Widget of SUNDARA Python Package. The user must compile all the boxes before clicking the “Click and Go!” button, and hence executing SUNDARA.
The imaging procedure - performed by SDI and/or SDT - creates a group of solar maps in FITS file format and organised with a correct filename (Sect. 3.3 and Fig. 1). These maps are collected in a specific directory, whose path is reported in the variable directory_fits (line 8) of the file sundara_input.ini (see Sect. 2.3); this file contains all the FITS filenames (according to the procedure described in Sect. 3.3) in the array name_fits_array (line 38), in order to be processed by SUNDARA Package. Currently, the name_fits_array is updated to December 2020; further solar maps can added from the user.
SUNDARA has been successfully tested on about 200 solar maps implemented with SRT and Medicina radio telescopes. This Python Package receives in input these solar maps, and produces in output a complete analysis in a short time (about 5 minutes for each solar map), saving a directory containing images, plots and several tables with physical information (brightness temperatures, fluxes and spectral indices, with respective errors) of the ARs detected in all the solar maps (Fig. 1). SUNDARA unearths candidate ARs through several algorithms, that search patterns consistent with an elliptical 2D-Gaussian kernel [10]. The detected ARs are further modelled through an elliptical 2D-Gaussian function with noise [9].
The intuitive widget of SUNDARA appears (Fig. 4) when the user types python sundara_v12.py in the terminal (or run sundara_v12.py in the ipython environment). After filling the form with the required details for the analysis, the "Check configuration" yellow button allows to verify the configuration on the terminal, and eventually to modify this configuration. Once satisfied with the selected configuration, the "Click and Go!" green button allows to execute SUNDARA.
This widget includes the following tools:
"Insert the path of the FITS files", "Insert the path (with the filename) of the Excel table", "Insert the path of the output directory" -- In these boxes there are already the default directory paths - inserted by the user during the installation of SUNDARA (Sect. 2.3) - of (1) the work directory of the FITS files, (2) the Excel table of the SunDish archive (Sect. 2.3), and (3) the output directory. These directory paths can be changed by the user, modifying these widget boxes.
"Operating system" -- The user can select the operating system between Linux and Windows.
"Do you want to consider the multi-feed receivers?" -- For SRT solar images obtained with SDT, the user can select if consider only the central feed (mono-feed mode) or all the feeds of the receiver (multi-feed mode), with the automatic production of the merged solar image. The multi-feed tool in SUNDARA for SDT solar images is in test phase, and therefore we advise to use this procedure with caution.
"Do you want to execute the re-calibration tool for solar maps?" -- Only for calibrated solar maps obtained with SDI, the user can execute a new self-calibration procedure, if necessary; in case of uncalibrated solar maps obtained with SDT, these maps are automatically self-calibrated with the re-calibration tool of SUNDARA.
"Do you want to execute the online server for solar catalogues?" -- The user can select a specific tool that automatically associates the AR candidates in position with the detected ARs at other observing frequencies (Figs. 5a and 5b), reported in the Heliophysics Event Knowledgebase (HEK, see [5]).
"Do you want to execute the histogram tool for the analysis?" -- The user can select the analysis of the histogram of the brightness temperature distribution (Fig. 6), to calculate the width "sigma" of the Gaussian distribution (with its uncertainty), indicative of the solar activity.
"What kind of calibration do you prefer?" -- The user can select the kind of calibration. "Landi" indicates the self-calibration procedure, as implemented by SDI Package [6][12]; "CasA" indicates a specific absolute brightness calibration procedure with respect to the young and bright Cas A (3C461) Supernova Remnant, an ideal flux calibrator circumpolar at the INAF radio telescopes latitudes, and characterised by a high flux density (about 2,400 Jy at 1 GHz)[12].
"Likelihood maximisation through curve_fit" -- The user can select the curve_fit tool (yes button) of the SciPy Python library [15], that uses non-linear least squares procedure to model the ARs detected in the solar disk through SUNDARA. Otherwise (no button), ARs are modelled through the maximum log-likelihood method.
"Do you want to execute the MCMC approach for this analysis?" -- The user can select a deep analysis (also a few hours) through emcee Python Package [3], based on the Markov Chain Monte Carlo (MCMC) analysis in Bayesian approach. emcee is able to flush out degeneracies in the model parameters, with the aid of corner plots [2] (a corner plot is an illustrative representation of different projections of samples in high-dimensional spaces to reveal covariances). These parameters are constrained through the definition of prior distributions that encode preliminary and general information (SUNDARA considers uniform priors, but the exact ranges are still under development). In the MCMC analysis, the beginning of the ensemble sampler is characterised by an initial period - called ``burn-in'', discarded by the analysis - where the convergence of the average likelihood across the chains is unstable (default chains: 200; recommended: 500). The number of subsequent Markov chains (steps) is set up between 1,000 (default) and 10,000 (recommended), depending on the computational characteristics, with a recommended number of 40 walkers. All the uncertainties are reported at 68% (1-sigma, recommended).
For further technical details about SUNDARA, see [10].
Figure 5a: Detected ARs at other observing frequencies, reported in the Heliophysics Event Knowledgebase (HEK), associated with the image of the Sun (at the same observing epoch) at (left) 18.3 GHz obtained with Medicina Radio Telescope on June 23th 2018.
Figure 5b: Detected ARs at other observing frequencies, reported in the Heliophysics Event Knowledgebase (HEK), associated with the image of the Sun (at the same observing epoch) at 24.7 GHz obtained with SRT on January 28th 2020.
Figure 6: Histogram of counts as a function of pixels, in linear scale, produced by the self-calibration process. The histogram is referred to the 18.3 GHz observation performed on June 23th 2018 at Medicina radio telescope. The upper part of the distribution is well fitted by a Gaussian (orange line), whose peak corresponds to the RMS value of the QS brightness temperature, and the width is connected with the solar activity. The low-counts tail of the quasi-Gaussian distribution in this histogram is due to the brightness gradient of the corona.
As shown in Fig. 7, the path SD_sundara_v12/output contains all the output files produced by SUNDARA.
In particular, the output of each execution of SUNDARA is stored in the sub-directory Catalogue_YYYYMMDD_HHMMSS, containing images, plots and several tables with physical information of the ARs detected in the solar maps; this sub-directory contains the following folders:
temp -- tables for each analysed solar map;
FITS_SOLARNET -- all the analysed solar maps in FITS file format, with the header compatible with the SOLARNET metadata recommendations for solar observations [4];
plot_cycle -- all the plots about the time evolution of the solar flux density for each observing frequency and radio telescope;
mcmc -- tables, plots, and corner plots about the MCMC analysis in Bayesian approach (this directory appears only when the user activates the mcmc option);
SUNDISH_yymmdd_hhmm_XX.XGHz_YYYYMMDD_HHMMSS_fW -- images, plots and tables with physical information of the ARs detected in the solar maps; SUNDARA saves several directories of this kind, one for each solar map.
Figure 7: Structure of SUNDARA output. YYYYMMDD and HHMMSS indicate the epoch (year/month/day) and the time (hours/minutes) of saving file in the user's computer, respectively; yymmdd and hhmm indicate the epoch (year/month/day) and the starting time of observation (hours/minutes) of the solar map, respectively; XX.X indicates the central observing frequency (in units of GHz) of the solar map; W indicates the feed of the receiver, in particular W = 0 for SRT, W = 1 (or W = 0, see Sect. 3) for Medicina, and W = M for multi-feed mode. Green boxes indicate directories, and the orange box indicates the presence of tables in .tex format (useful for Latex tables).
Moreover, the directory filename SUNDISH_yymmdd_hhmm_XX.XGHz_YYYYMMDD_HHMMSS_fW could include the following suffixes (zero or more), indicating warning messages for the user:
FREQ -- the central frequency reported in the FITS filename of the solar map does not correspond to the central frequency reported in the Excel table of the SunDish archive (Sect. 2.3);
TM -- the minute value of the starting time reported in the FITS filename of the solar map does not correspond to the minute value of the starting time reported in the Excel table of the SunDish archive;
TH -- the hour value of the starting time reported in the FITS filename of the solar map does not correspond to the hour value of the starting time reported in the Excel table of the SunDish archive;
TA -- all the starting time reported in the FITS filename of the solar map does not correspond to all the starting time reported in the Excel table of the SunDish archive;
OUT -- the observing epoch of the analysed solar map is not included in the Excel table of the SunDish archive;
D -- the calibration tool identifies a double Gaussian in the histogram of the brightness temperature (see [10] and [12] for further details).
The folder Catalogue_YYYYMMDD_HHMMSS, in addition to including the aforementioned directories, contains 11 tables (in .tex file format); the user, for their analysis, can consider these 4 tables (the tables with a prefix between 00 and 06 essentially contain the same information of the tables with a prefix between 07 and 10, with more significant digits), that include - in a compact manner - the physical information of the ARs detected in the solar maps:
07_table_flusso_tutto_finale.tex} -- the flux densities for each AR detected in the solar maps;
08_table_spindex_tutto_finale.tex -- the spectral indices flux calculated for each AR detected in a specific observing epoch (and for each radio telescope);
09_table_idcard_finale.tex -- the general information about the solar maps analysed with SUNDARA (for example, the total flux density of the solar disk, and the number of detected AR for each solar map);
10_table_idcard_corr.tex -- similar to the previous table, but the total flux density of the solar disk is normalised with respect to the perihelion.
In these tables are reported the following information:
ID -- the identification number for every single map in the format XB, where X indicates the radio telescope (M for Medicina, S for SRT, and N for Noto), and B indicates the ID number of the map;
Epoch -- the observation date (expressed as yy-mm-dd);
T -- the acquisition time interval of the map (in units of Universal Time);
nu_{obs} -- the central observing frequency (in units of GHz);
sigma_{disk} -- the standard deviation of the solar disk brightness distribution with respect to the QS-level (in units of K);
AR_n -- the number of identified ARs in each solar map;
F -- the total flux density (in units of sfu) of the solar disk (with uncertainty), normalised with respect to the perihelion. The solar flux unit (sfu) is a convenient measure of flux density often used in solar radio observations; 1 sfu corresponds to 10,000 Jy.
ar_id -- the AR name (if present), according to the HEK archive;
Size -- the AR size, at twice the fitted semi-axes level (in units of arcmin^2);
T_{p,tot} and T_{p,ex} -- the maximum brightness temperature and the peak of the excess brightness temperature, respectively, for each AR (with uncertainties). The excess brightness temperature of ARs above quiet Sun levels, T_{ex}, is defined as T_{p,tot} - T_{b(QS)}, where T_{b(QS)} is the quiet Sun temperature.
S_{sub} and S_{tot} -- the AR flux density of the QS-subtracted image and the original image, respectively (in units of sfu); these values are given by Rayleigh-Jeans approximation, with uncertainties [10];
alpha_{T_p}, alpha_{T_{ex}}, and alpha_{tot} -- the spectral indices (with uncertainties) referred to T_{p,tot}, T_{ex}, and S_{tot}, respectively [14][10];
Notes -- further AR flags ("b" indicates if the AR position is located outside of the 95%-level of the solar radius; "k" indicates the distance between 2 different ARs < 2 beams of the receiver; "C" indicates an AR located inside a confused region; sequential numbers are related to multiple AR detection for the same observing session).
Finally, the user analyses the output directory and evaluates, through visual inspection, the possibility to reject a part of the final automatic analysis, in order to avoid fake AR detection. In particular, if the user analyses a wrong solar map obtained with Medicina radio telescope - for example, the analysis of a solar map with respect to the FEED_0 obtained with an observation before 6th July 2021 (Sect. 3) - SUNDARA displays a warning message in the terminal and is able to skip the analysis of that solar map.
Upon a specific computational improvement and more robust test phase with other solar maps, SUNPIT could be an even more complete tool for solar physics thanks to future implementation of other physical aspects in SUNDARA (for example, polar brightening and coronal holes), and further input FITS files coming from other international facilities in a broad range of the electromagnetic spectrum (from radio to X-ray frequencies). Last but not least, this pipeline will also be suitable for the solar images obtained with both Noto radio telescope (in the near future), and the new receivers in Q-band (33 - 50 GHz) and W-band (75 - 116 GHz), soon installed at SRT in the context of the National Operative Programme (Programma Operativo Nazionale-PON), to the study of the Universe at high radio frequencies.
These improvements are crucial for a future complete sharing of SUNPIT with the international community; for further information and collaboration, the reader is encouraged to contact the authors of this technical note.
REFERENCES
[12] A. Pellizzoni and et al. SunDish Project: Single-Dish Solar Radio Imaging with INAF Radio Telescopes. 2021 in prep.