To further clarify the contribution of the Image2Code component, here we provide a case study to analyze how the Image2Code component works.

As shown in Figure 1-a, we demonstrate a task instance eslint-15243, which was successfully resolved by GUIRepair$_{I2C}$, but the GUIRepair$_{base}$ fix failed. In Figure 1-a, GUIRepair$_{I2C}$ accurately located the bug file \textit{cli.js} and generated a fix behavior consistent with the developer patch. Yet GUIRepair$_{base}$ did not locate the correct bug file, which caused the fix to fail. 

Further, we try to analyze the reasoning trace behind these two variants to reveal why GUIRepair$_{I2C}$ was able to locate the bug file accurately. As shown in Figure 1-b, GUIRepair$_{base}$ reads the issue report directly to locate suspicious files from the codebase, however, since the model does not have project-specific knowledge, it struggles to accurately locate the relevant code files behind the multimodal issue. 

In contrast, in Figure 1-c, GUIRepair$_{I2C}$ first learns the project details in the knowledge mining phase, where it reads the key documents and learns the role of the cli.js file. Then, benefiting from the fact that the model has learned that project knowledge, when implementing fault localization, the model considers cli.js as a suspicious file by viewing the multimodal issue report. In this way, GUIRepair$_{base}$ misses the opportunity to localize to the bug file due to its lack of project knowledge, while GUIRepair$_{I2C}$ compensates for this by using the Image2Code, which improves the overall effectiveness.

Figure 1-a: The developer/GUIRepair_base/GUIRepair_I2C patch of eslint-15243.

Figure 1-b: The reasoning trace of GUIRepair_base to solve eslint-15243.

Here we show a task instance next-4182 that can only be resolved by GUIRepair$_{full}$. As shown in Figure 2-a, the issue report for this instance does not provide the complete reproduced code, which results in GUIRepair$_{C2I}$ unable to implement patch validation for selecting patches. Furthermore, even though GUIRepair$_{I2C}$ is able to generate the repro code during the fault comprehension phase, it is not equipped with the Code2Image module resulting in the inability to fully utilize the repro generation. Therefore, these variants cannot solve this case. 

At the same time, we show the reasoning trace of GUIRepair$_{full}$ in Figure 2-b. Benefiting from the Image2Code component, the model successfully generates repro code (2️⃣Repro Generation) for the bug scenario after learning project-specific knowledge (1️⃣Knowledge Mining) and helps the model locate the correct bug file cascader-select.jsx (3️⃣File Localization). 

Then, the Code2Image component uses the previously generated repro code to capture the visual effects presented by the fixing behavior. When the iteration validates to Patch_8 (7️⃣Patch Selection), the model observes that the bug component renders properly on the web page and considers the patch resolved the issue scenario. 

In this case, the reproduced code generation is a key factor in helping to solve the problem by tightly linking the Image2Code and Code2Image components to better understand the bug and validate the patch. Considering that most multimodal instances lack reproduction links, the full repair potential can only be unlocked by using both key components together. Overall, GUIRepair requires not only the Image2Code to understand the multimodal problem and generate the reproduced code, but also the Code2Image to use the reproduced code to capture the actual effect of the fixing behavior, which is an inextricably intertwined process.

Figure 2-a: The issue report of next-4182. Note: left is the textual description, right is the image in the issue report.

Figure 2-b: The reasoning trace of GUIRepair_full to solve next-4182.