SUMMARY

Edited Magnetic Resonance Spectroscopy (MRS) is used to quantify metabolites overlapped, in typical scans, by those of higher concentration. For example, it is used to quantify Gamma Aminobutyric Acid (GABA), which is overlapped by creatine and glutamate. However, high quality data requires long scan times. Therefore, we propose an Edited-MRS reconstruction challenge. The objective is to investigate machine learning models to reconstruct spectra using three quarters less data than current Edited-MRS scans, which will translate to four times faster edited-MRS scans. Teams participating in the challenge will be provided with simulated and in vivo data training sets representing GABA-edited MEGA-PRESS scans composed of two subspectra (ON and OFF). The participants will also be provided with scripts for adding varying noise, frequency and phase shifts, which can be used for data augmentation purposes. The models submitted by the teams will be tested on three different sets: simulated data, homogeneous in vivo data (i.e., single-vendor data), and heterogeneous in vivo data (i.e., multi-vendor data). Evaluation and ranking of different solutions will be based on a weighted combination of quantitative metrics, such as mean squared error, signal-to-noise ratio, linewidth, and peak shape. We expect at least five teams will participate in the challenge. The challenge will receive submissions from teams from different backgrounds ranging from Magnetic Resonance (MR) spectroscopy and MR imaging to machine learning experience. The results of the challenge will be summarized and submitted for a joint publication.

For access to the data corruption script, metrics calculation script and baseline model, please visit the following GitHub repository 

This challenge will have the following three tracks: 

Data

A simulated development set of 5,000 ground truth spectra will be provided in addition to a data corruption script to add noise and frequency and phase offsets. Subsequently, a simulated test set of 500 corrupted spectra will be provided.

Simulated Data is provided through the challenge LINK

In Vivo Data is available through https://www.nitrc.org/projects/biggaba/

Evaluation

Evaluation of the models will be done using the evaluation datasets which have similar features to the training datasets. The in vivo subset may differ in traditional features, such as by vendor and echo time (for task 3). A subset of the simulated data will be of lower quality. Examples of lower quality data may include a lower mean SNR and larger frequency and phase shifts

Ranking of the models will be made per task (simulated, in vivo homogeneous, in vivo heterogeneous). Each metric will be ranked individually per task and then combined using the given weights to create the final task ranking. The metrics used for evaluation and ranking are as follows:

1. Mean Squared Error (MSE) - Weight: 40%

   1. For the simulated data, the ground-truths will be used as a reference.

   2. For the in vivo data, the resulting spectrum preprocessed with the full amount of transients using Gannet will be used as a reference.

   3. Only the spectral region of most relevance will be considered

2. Signal to noise ratio (SNR) - Weight: 20%

   1. Will be used to assess the model’s improvement of the data quality and the reliability of the final quantification.

   2. Will be measured as the amplitude of the GABA peak divided by the standard deviation of the signal between 9 and 10 ppm.

3. Linewidth (FWHM) - Weight: 20%

   1. Will be used to assess the model’s alignment of the subspectra and the reliability of the final quantification.

   2. Will be measured as the full width at half max of the GABA peak centered at 3 ppm.

4. Peak Shapes - GABA, GLX - Weight: 20%

   1. Will be used to assess subtraction artifacts resulting from improper alignment of subspectra.

   2. Will be measured using a similarity index applied to GABA and GLX peaks in the final spectrum.

A script provided by the organizers to participants will be used to calculate the above metrics. While GABA quantification will not be used to determine ranking, it will be provided by the organizers during presentation of results using GANNET to provide additional context and support to the results.

Submission Format

All participants will be required to release their source code via GitHub under the MIT license. 

For track 01 (simulated data), participants will submit an H5 file containing the reconstructed spectra of the data test set by their model. An example submission file (utils.py) for track 01 can be found on the challenge GitHub repository.