Pie chart depicting the answer to the question “Does the refactored code preserve the behavior of the original?” Out of the 320 refactored code segments, 311 segments (97.2%) preserved the behavior and 9 segments (2.8%) did not preserve the behavior of the original.
Code behavior was preserved in refactored segments:
Original BREADTH_FIRST_SEARCH.java code segment
Refactored BREADTH_FIRST_SEARCH.java code segment for performance
There were many reasons why the refactored code did preserve the behavior of the original code, but these 6 reasons were seen repeatedly in all 312 out of 321 refactored files.
In Figure 2.2, it can be seen that ChatGPT imported all of the necessary classes from the “java.util.*” package used in the original code segment shown in Figure 2.1, which includes the Hashset, Queue, and Set. This behavior does not change the functionality of the code. ChatGPT used the necessary packages for the code rather than utilizing the whole package when it was not necessary.
Figure 2.2 depicts how 2 public classes were declared compared to the 3 public classes in Figure 2.1.
For example, in Figure 2.1, the public class BREADTH_FIRST_SEARCH does not follow Java naming conventions, which is why ChatGPT renamed it to BreadthFirstSearch. Even though this change does not change the functionality of the code, it makes it easier to read.
If a method was created, then ChatGPT followed the java naming conventions. As depicted in Figure 2.2, the node “node” used the poll() method instead of the removeFirst() method in Figure 2.1. The reason this was done is because poll() is a method of Queue while removeFirst() is a method used for linked lists. It is better to use poll() since the code works with Queues.
ChatGPT will declare the variable first, and then in a new line initialize it. For instance in Figure 2.2, ChatGPT declared the Queue node “queue” as an ArrayDeque(), and in the next line initialized it using the add() method. This allows for the object to be initialized later in the code if it is not known what value this object will hold.
ChatGPT uses while loops and for loops to perform recursive operations. In the case of the code used in Figures 2.1 and 2.2, the while loop has remained the same which portrays that the behavior of the while loop as preserved. However, there were some cases where if there was an if-statement used to perform recursive operations, ChatGPT would use a while or for loop to perform the same operation.
Overall, most of the changes made by ChatGPT did not change the functionality of the code, which is why the behavior was preserved.
Code behavior was not preserved in the refactored segment:
Original PossibleChange.java code segment
PossibleChange.Java segment that was refactored for performance
There were three main reasons as to why the refactored code segments did not preserve the behavior and logic of the original files.
The incorrect call can be from a name change or a mistake made by the AI. This can be seen in Figures 3.1 and 3.2 where recursive calls were passed through the wrong variable creating an infinite loop causing the code to break from the behavior of the original segment. The first reason for making a call to the wrong variable was why 6 of the 9 code segments did not preserve the behavior of the original with all 6 segments being refactored versions of PossibleChange.java for different quality attributes.
In the original code, the conditions used in for loops were simply "less than," whereas in the refactored code, the conditions were modified to be "less than or equal to." This change affected the number of iterations performed by the for loop and consequently altered the behavior of the code. The modification of loop conditions accounted for the lack of behavior preservation in 2 out of the 9 code segments.
The last reason that behavior was not preserved from the original file to the refactored version is the fact that in the refactored code one of the variables was changed from an ArrayList to a List and in another section of the code there was a call to the wrong variable when updating the List.