Overview

Testing is the most direct and effective technique to ensure software quality. Test scripts always play a more important role in mobile app testing than test cases for source code, due to the GUI-intensive and event-driven characteristics of mobile applications (app). Test scripts focus on user interactions and the corresponding response events, which is significant for testing the target app functionalities. Therefore, it is critical to understand the test scripts for better script maintenance and modification. There exist some mature code understanding (i.e., code comment generation, code summarization) technologies that can be directly applied to functionality source code with business logic. However, such technologies will have difficulties when being applied to test scripts, because test scripts are loosely linked to apps under test (AUT) by widget selectors, and do not contain business logic themselves.

In order to solve the test script understanding gap, this paper presents a novel approach, namely TestIntention, to infer the intention of GUI test scripts. Test intention refers to the user expectations of app behaviors for specific operations. TestIntention formalizes test scripts with an operation sequence model. For each operation within the sequence, TestIntention extracts the target widget selector and links the selector to the GUI layout information or the corresponding response events. For widgets identified by XPath, TestIntention utilizes the image understanding technologies to explore the detailed information of the widget images, the intention of which is understood with a deep learning model. For widgets identified by ID, TestIntention first maps the selectors to the response methods with business logic, and then adopts code understanding technologies to describe code in natural language form. Results of all operations are combined to generate test intention for test scripts. An empirical experiment including different metrics proves the outstanding performance of TestIntention, outperforming baseline by 10% -- 21%. Also, it is shown that TestIntention can save about 70% developers' time to understand test scripts.

Contribution

• We propose a novel approach, TestIntention, utilizing GUI image and code understanding to fully explore the code, textual, and image information, and then TestIntention generates test script intentions in natural language.

• We present a novel algorithm to map test operations in test scripts to the corresponding response methods in source code and GUI image information in the GUI layout files.

• We propose a test operation sequence model to formally represent the test script, and include the necessary information to map the operations, like operation type, selector, operation, and specific contents.

• An empirical experiment and a user study show the effectiveness of TestIntention and the capability to alleviate developers' burden to understand test scripts.

Framework

Test Script Intention Example (Intentions in comments)

TestIntention Reproduction Package

source code: https://github.com/iGUITest/TestIntention-v2

replace the <root>/CodeIntent/code2seq/models/java-large-model/model_iter52.release.data-00000-of-00001 file with this file.

Experiment

Test App of the Experiment

Test Script Demo

Widget Intention Understanding Dataset

RQ1: TestIntention Effectiveness

RQ2: Mapping Effectiveness

RQ3: User Experience

Detailed explanation is presented in the paper.