RegMiner - Regression Examples

Quality of Mined Regressions (Sampled)

Examples of Reported Regressions

We sample 30 regressions with high coverage score (>0.9), 30 with medium score (0.4-0.7), and 30 with low score (<0.1). Samples as follow：

qualitative-regression

Case1: Fixed by Semantic Reversion

BugID: fastjson_15dead6fcbb32402b9a3a4802c24f1780d6e1901 (There is more than two years between the regression bug-inducing commit and bug-fixing commit ）

Feature description:

As shown in Figure 1, a refactoring was submitted in the bug-inducing commit(bic). The semantics of the code, (In >= 0—>false )& (In < 0—>true) is modified to (In = -1 —> true) & (In >= 0 —> false )&( In < -1 —> false), obviously ( In < -1 —> false) introduces wrong semantics .

Therefore, in the bug-fixing commit(bfc), the semantics are reverted.

Note that! even though working code " if (Arrays.binarySearch(excludes, label) >= 0) {return false};" and bug-fixing code " return Arrays.binarySearch(excludes, label) < 0" have different textual representations, they have the same semantics, i.e. (In >= 0—>false )& (In < 0—>true)

Fig.1 In this example, " if (Arrays.binarySearch(excludes, label) >= 0) {return false};" in working commit was changed to "return Arrays.binarySearch(excludes, label) == -1;" in bug-inducing commit to introduce regression, and then changed to " return Arrays.binarySearch(excludes, label) < 0" in bug-fixing commit to be fixed. For convenience, we use a formal way to express the semantics of the code, for example, "if (Arrays.binarySearch(excludes, label) >= 0) {return false};" is expressed as (In >= 0—>false )& (In < 0—>true).

Case2: Reimplementation Inducing a Performance regression

BugID: jsoup_51cb86699beae26b74c594e2daaa098116f9f572

Bug-inducing commit reimplemented the parser module to adapt to the new specifications. However, regression with an endless loop was introduced, caused by the missing changes.

Feature Description:

Fig.2 If the test time of the test case in bic is much longer than working commit(wc) and bug-fixing commit(bfc), RegMiner regards it as performance regression.

Case3: Refactoring feature causes a regression

BugID: jcifs_c22a612ca28a47305a044c71ec13984da3897f27

Bug-inducing commit refactored the encoding/decoding module, but a pair of common combinations miss its members. There is about one year between bug-fixing commit and bug-inducing commit.

Feature Description:

Moritz Bechler, the developer of this project, responded to our accuracy of marking the bic root cause.

Case4：Regression from feature enhance

BugID: druid_103ee3e1a3c446590d0bd61e8e29f1d0d643b7e4

Bug-inducing commit add new method to SQL parser module to enhance the feature, and regression also comes with it.

Feature Description:

Case5: Bug-fixing in bug-inducing commit cause a NullPointException regression

BugID: univocity-parsers_52e62f8d4d690627a56b8ab084ecdfbc5ae610bd

Bug-inducing commit aims to fix issue #253 that annotated fields won't be processed when reading rows with fewer elements. However, a regression had caused by missing null pointer check.

Feature Description: