Embedded Files
Core features (functionalities) of an app can often be accessed and invoked in several ways, i.e., through alternative sequences of user interface (UI) interactions. Given the manual effort of writing tests, developers often only consider the typical way of invoking features when creating the tests (i.e., the “sunny day scenario”). However, the alternative ways of invoking a feature are as likely to be faulty. These faults would go undetected without proper tests. To reduce the manual effort of creating UI tests and help developers more thoroughly examine the features of apps, we present Route, an automated tool for feature-based UI test augmentation for Android apps. Route first takes a UI test and the AUT as input. It then applies novel heuristics to find additional high-quality UI tests, consisting of both inputs and assertions, that verify the same feature as the original test in alternative ways. Application of Route on several dozen tests for popular apps on Google Play shows that for 96% of the existing tests, Route was able to generate at least one alternative test. Moreover, the fault detection effectiveness of augmented test suites in our experiments showed substantial improvements of up to 39% over the original test suites.
Artifacts
Artifacts
Prerequites (Tested on Window 10 and Oracle VirtualBox v6.0)
Prerequites (Tested on Window 10 and Oracle VirtualBox v6.0)
Python 3.7 with the following packages installed: Appium-Python-Client, astor, beautifulsoup4, editdistance, Flask, Flask-RESTful, html5lib, lxml, numpy, requests, networkx, matplotlib, xmldiff
Appium-Desktop 1.17.1, with ANDROID_HOME and JAVA_HOME environment variable set.
Oracle VirtualBox VM, with Android-7.1-r4 ISO installed. You also need to set port forwarding on the VM, from 127.0.0.1:6000 to :5555
Install Python interface of VirtualBox's COM API by following this instruction
Download the zipped file and apk file (65MB) containing the source code, subject tests and apks in our experiment
You may need to set environment variables ANDROID_HOME and JAVA_HOME and add the path of JDK and VirtualBox SDK into PATH
Change the value of CLONE_FOLDER and ADB in const.py accordingly
Getting Started
Getting Started
We suggest starting with TestCheckAll for Shopping List to get a quick overview on how Route works.
Install apk/shoppinglist2.apk in the VM
Start Appium-Desktop
The original test is in test/shoppinglist2/original/TestCheckAll.py, which can be directly executed with the above pip packages and VM installed
If encounter a problem, check the config in config/shoppinglist2/shoppinglist2.json
Instrument the original tests with instrument.py, by changing the app and test name in the code:
line 219: app = "shoppinglist2"
line 224: if t != "TestCheckAll.py":
The augmented test will be generated as test/shoppinglist2/instrumented/ins_TestCheckAll.py
To run the instrumented test, first launch the app, and take a snapshot named as "base". After executing the instrumented test, the base path and related config will be generated under graph/shoppinglist2/ins_TestCheckAll and config/shoppinglist2/ins_TestCheckAll
Run spawn.py by changing the code. This creates multiple VMs for each base state for the exploration later.
line 41: app, test = "shoppinglist2", "ins_TestCheckAll"
Run Explorer.py for App Exploration, by changing the code:
line 156: app, test, appium_port = "shoppinglist2", "ins_TestCheckAll", "4723"
line 158: for snapshot in ['82099819', '40762332', '99078092']:
(# check the snapshot names (base states) in graph/shoppinglist2/ins_TestCheckAll/ins_TestCheckAll.json)
Run log_parser.py by changing the code:
line 153: app, test = "shoppinglist2", "ins_TestCheckAll"
The graphs after exploration for each base states will be stored in graph/shoppinglist2/ins_TestCheckAll/3-step
Run merge_graph.py by changing the code:
line 4: app = "shoppinglist2"
line 5: test = "ins_TestCheckAll"
A merged graph will be generated in graph/shoppinglist2/ins_TestCheckAll/3-step/ins_TestCheckAll-merged.json
Run TestGenerator.py by changing the code:
line 290: app = "shoppinglist2"
line 291: test = for test in ["ins_TestCheckAll"]:
New tests will be generated under test/shoppinglist2/augmented/ins_TestCheckAll/3-step/ins_TestCheckAll-IDX.json
The generated tests can be executed by runngin TestExecutor.py
Notable Folders and Files
Notable Folders and Files
The original tests are in test/[APP]/original, written with Appium and Python. (They will be later instrumented by instrument.py)
The generated tests are in test/[APP]/augmented/[TEST_NAME]/3-step, stored in JSON format.
graph/[APP]/[TEST_NAME]: base path and explored graphs for each base state
config/[APP]/[APP].json: the config file for running the app
Subject Apps
Subject Apps
Page updated
Google Sites
Report abuse