Embedded Files
To answer RQ2, we aim to explore the impact of different query schemes on the data quality and the performance of imitation attacks.
Additional material for query schemes
Additional material for query schemes
In the paper, we only simply describe the query schemes (ZSQ | ICQ | ZS-COT). Actually, we provide the following link if readers would like to explore it.
A trend starts from "Chain of Thought Prompting Elicits Reasoning in Large Language Models."
From the result listed, we can find that the ICQ achieves better performance than the other schemes on both CSum and CT tasks.
Moreover, ZS-COT achieves a CodeBLEU score of 24.84 on the CSyn task for CodeT5, which outperforms ICQ by 4.94%.
Conclusion
Conclusion
Query schemes have a major impact on the performance of imitation attacks. It is vital to construct query schemes with a suitable template design. Queries with sufficient context generally enhance the attack toward all tasks, while the CoT only shows a noticeable boost in code synthesis.
Page updated
Google Sites
Report abuse