The evaluation process according to  the 4 shared-tasks is given below:

Task 1: Classification

Given a set of figure images, identify the class/type of the figure that is one of the 47 classes (visualization). example: bar chart, scatter plot, spectra chart, etc.

The performance was assessed using the following metrics:

• Accuracy

• Precision

• Recall

• F1-score

Task 2: Data Table Extraction

Given a set of quantitative-plot figures, extract the underlying data table in the standard markdown format. example:

| First Header | Second Header |

| --- | --- |

| Content Cell | Content Cell |

| Content Cell | Content Cell |

The Performance will be assessed using the following metrics:

• RMS (Relative Mapping Similarity) (Liu et al. 2023)

• TEDS (Tree Edit Distance Similarity) (Smock et al, 2021)

Task 3: Summarization

Given a set of quantitative-plot figures (and optionally caption and text content), extract an insightful summary. example:

"The chart illustrates a timeline of a single ALE cycle showing the values of the following process parameters: DC power, ICP power, CF4+H2 flow rate, and Ar flow rate."

The Performance will be assessed using the following metrics:

• ROUGE (Lin et al, 2004)

• BERTScore (Zhang et al, 2019)

Task 4: Visual Question Answering (VQA)

Given a set of quantitative-plot figures (and optionally caption and text content), perform visual question-answering. example:

Q: What are the three main steps of this ALE cycle and what function fulfills the last sub-cycle?

A: The three main parts in this ALE cycle are the dosing of a plasma containing Ar, CH4, and H2 related species, a plasma exposure, solely consisting of Ar, and lastly a Ar plasma exposure with a biased substrate. This last step is introduced to enable energetic ions to bombard the substrate, leading to selective etching.

The Performance will be assessed according to each answer-type using the following metrics:

• Paragraph - ROUGE and BERTScore

• Factoid - Exact Match (EM) and ROUGE

• Yes/No - Accuracy, Precision, Recall, F1-score

• List - Set-based Precision, Recall, F1 (unordered lists)