CatWalk

This weblog describes the history of the evolution of CatWalk, including updates and fixes to the tool. The weblog is in reverse historical order.  The bug reports and fixes are in order of detection.  

CatWalk Development History

06 July 2026:            Add mock types for Consumer, Predicate UnaryOperator, Comparator

10 January 2026:      Submit major revision of CatWalk paper to the journal STVR.

10 September 2025:  Receive first reviews from the journal STVR.

27 September 2024:  Public launch of the CatWalk website, with a free trial of the tool.

11 September 2024:  Start constructing the CatWalk website, with a user guide.

10 September 2024:  Submit CatWalk journal paper to Software Testing, Verification and Reliability.

09 September 2024:  Change all citations from Harvard to IEEE style, on request.

04 September 2024:  Final review of second draft of CatWalk journal paper.

01 May 2024:            Start work on second draft of CatWalk journal paper.

14 September 2023:  Fixes to issues revealed by evaluation on repositories.

13 September 2023:  Change input sampling method from odometer to Cantor zig-zag.

05 September 2023:  Results collated from evaluation of CatWalk on repositories.

16 August 2023:        Major evaluation of CatWalk on Java libraries, Defects4J repository

21 June 2023:           First draft of CatWalk journal paper, without major evaluation.

11 April 2023:           Start work on first draft of CatWalk journal paper.

06 April 2023:           Post-review revision to CatWalk.

07 March 2023:         Demonstration of CatWalk to British Telecom.

10 December 2021:   CatWalk development, third iteration.

09 December 2021:   First seminar about CatWalk, to the Testing Research Group.

06 December 2021:   Review of CatWalk, after a long break in development.

06 March 2020:         Second evaluation of CatWalk.

21 February 2020:     CatWalk development, second iteration.

04 November 2019:   First evaluation of CatWalk.

12 September 2019:  CatWalk development, first iteration.

CatWalk Bug Tracking History

Bug 1: Failure to load an external test class outside the CatWalk project.

Fix: Recreate a TestDirectoryLoader every time the test project root directory changes.

Bug 2: Failure to find a concrete subclass for an abstract class in the Ferromone Trails project.

Fix: Add a URLDecoder to prevent %20 escaped spaces from not matching searched folder names.

Bug 3: Failure to find a concrete subclass of Iterable class Consumer in the JAST project.

Fix: Filter out default spliterator methods, unless the COMPLETE convention is chosen; then mock Consumer.

Bug 4: Java runtime stack overflow when generating divergent object construction trees.

Fix: Set a recursion limit in ObjectGenerator, so that this bottoms out with nulls.

Bug 5: ArrayIndexOutOfBounds when displaying a partition frequency histogram for the JAST project.

Fix: Increase the default array length to allow more RESULT and FAILURE cases, up to 2n+1.

Bug 6: Printed oracle representation of objects includes both normal and static fields.

Fix: Add modifier checks for static, to ensure either normal, or static fields are printed.

Bug 7: ResponseEncoder did not emit TRIVIAL response for testing constructors with null inputs.

Fix: Refactor the TestStep tree, migrating isTrivial() to ParamStep, inherited also by CreateStep.

Bug 8: Some interfaces like Iterator, ListIterator, Consumer do not have public implementing classes.

Fix: Add a special case in InterfaceGenerator to mock instances of these. Also fixes Bug 3.

Bug 9: Display of a TestSummary was missing a space between the printed STRATEGY and MODALITY.

Fix: Add a space to the buffered output.

Bug 10: Failure to reset the default JUnit testPackage when the testClass is reset.

Fix: Set the default testPackage whenever the testClass is reset.

Bug 11: TestSettings are reset to defaults too often, by saveSettings() of CatWalkFrame.

Fix: Add an equality-check on the testClassDirectory so that saveSettings() does not always reset.

Bug 12: Swing generates an internal ArrayIndexOutOfBounds when the JTabbedPane thread is out of synch.

Fix: Internal Java bug: finesse by dispatching an anticipated Warning dialog, so CatWalk does not hang.

Bug 13: Repeated pop up dialogs from recovered instantiation exceptions in probe and test cycles.

Fix: Change reporting method to direct throw of WalkerException, rather than sending Failure report.

Bug 14: Failure to recognise and prune exceptions in the prefix when an Error was raised.

Fix: Change how exceptional prefixes are detected, as instances of Throwable, not of Exception.

Bug 15: ArrayIndexOutOfBounds raised when construction fails at the maximum inputs threshold.

Fix: Create the index offset arrays with one more cell, to allow recovery mechanism to work.

Bug 16: Unable to distinguish maximum recursion cases from failure to instantiate cases.

Fix: Split exception handling blocks to distinguish StackOverflowError from other Exceptions.

Bug 17: Recursive constructors cause ObjectGenerator to proceed directly to max recursions.

Fix: Increment the input index before generating recursive construction arguments, to try other approaches.

Bug 18: Internal binary Java VerifyError when testing a subclass adding equals(), hashCode().

Fix: Catch any VerifyError thrown by ClassInspector.discoverProtocol() and report as a Java fault.

Bug 19: Incomplete coding of sample test class Gensym, which replicated StaticGensym.

Fix: Recode Gensym to use nonstatic methods, as originally intended. Update the Javadoc.

Bug 20: When testing classes in dependent projects, cannot load base classes from core project.

No fix yet: CatWalk cannot introspect about CLASSPATH, so cannot access outside current project.

Bug 21: When generating JUnit code for CLI BasicParser, the names of tests contained [] brackets.

Fix: Have JUnitPrinter convert array types to variable names, replacing any [] with the string "Ar".

Bug 22: Saved oracle files for CLI HelpFormatter could not be re-read, due to newline in text.

Fix: Add new clauses to Formatter.formatSpecial() to deal with more escaped characters like newline.

Bug 23: JUnitPrinter.printSequence() failed when casting an Error to an Exception (disjoint).

Fix: Have JUnitPrinter cast to Throwable; but let JUnit @Test methods still declare an Exception.

Bug 24: When generating JUnit tests for CLI OptionGroup, a static field resetter was included.

Fix: Ensure ClassInspector.inspectFields() clears the initialValues map from earlier static class test.

Bug 25: When validating test classes with both normal and static APIs, oracle files are overwritten.

Fix: Ensure Oracle's open() and close() methods append "Static" to the oracle filename for static API testing.

Bug 26: When generating JUnit code for classes with both normal and static APIs, JUnit files are overwritten.

Fix: Ensure JUnitPrinter appends "Static" to the JUnit filename for static API testing.

Bug 27: When testing DigestUtils in the Codec project, could not mock InputStream, OutputStream arguments.

Fix: Have AbstractGenerator create concrete mocks for these, and also for Reader and Writer interfaces.

Bug 28: When testing type-overloaded methods with trivial null inputs, flaky tests resulted.

Fix: Fault in the tested software - behaviour under null inputs is inconsistent for different argument types.

Bug 29: When testing classes in the JAST project, VerifyError claimed typecasting was happening.

No fix: Internal Java error, since the JAST source code was not doing the claimed typecasting.