We use GPT-4o-mini, Deepseek-chat, Llama-3.1-70b, Qwen2-72b-instruct, and Vicuna-7b as our mutation-model candidates. Qwen2-72b-instruct achieves the highest ASR in both Top1 and Top5 metrics, demonstrating superior performance. Deepseek-chat and GPT-4o-mini also perform well in terms of ASR. However, GPT-4o-mini stands out for its significantly shorter mutation time compared to the other models, which is crucial considering that our method involves running thousands of iterations. Althoug GPT-4o-mini's accuracy is slightly lower than that of Deepseek-chat by 7.13% and Qwen2-72b-instruct by 22.72%, its significant time-saving advantage of 36.13% over Deepseek-chat and 67.06% over Qwen2-72b-instruct makes it highly effective for tasks requiring numerous iterations. Additionally, despite being a commercial model, GPT-4o-mini is more cost-effective than Qwen2-72b-instruct.
This balance between efficiency and cost makes GPT-4o-mini a highly suitable choice for iterative mutation tasks where both time and budget constraints are critical.