AdaCom: On-the-Fly Reinforcement on Code Comment Generation
More Examples for Adacom
Case Study with ChatGPT
Table1 shows the comparison examples between AdaCom and ChatGPT where ChatGPT gives wrong predicitons. In Test Sample1, ChatGPT says that "stop a currently running process" but based on the ground truth, it stops the "timer". In Test Sample2, ChatGPT says that "finding a field in a thrift object by its ID" while ground truth is "find the fields constant that matches fieldID". In Test Sample3, the ChatGPT is totally different from ground truth and the method is not always "return the first element".
Table2 shows the comparison examples between AdaCom and ChatGPT where ChatGPT gives verbose comments.
ChatGPT usually translate the code line by line and follow a structure like "defines a method called X", "takes X and X as parameters", "return the X object".
More example compared with ChatGPT can be found in this link.
More Details