Rua, Rui, & Saraiva, João. (2023). A large-scale empirical study on mobile performance: Energy, Run-time and Memory. https://doi.org/10.5281/zenodo.8138554

Data Characterization

Apps

Here we present a list of  the apps with certain characteristics that were executed using our execution pipeline. The following resources contain the name and version of the projects, usually in ther format "<app_package>/<app_version>".  In order to obtain the respective Android Projects' source code, please use AndroZooOpen Crawler or consult the AndroZoo dataset which contains the package name and project's url of each of the respective packages.

• Apps using Kotlin

• Apps with Red APIs: 

  • Apps with red APIs in its sources

  • Apps that invoked red APIs

  • Apps with each Red API

• List of apps with Lint Issues

• Apps using HTTP Libraries

• Apps using JSON Libraries

• Apps using Threading Libraries

• Apps using Logging Libraries

• Apps using Image Libraries

• Apps using IO Libraries

Metrics

* not present in the paper

Results

Dataset: https://doi.org/10.5281/zenodo.8138554

Zip with all collected results: dataset.zip

File description:

• cloc.out: contains static metrics obtained with scc (e.g. programming languages found in the project, #lines, #files, etc);

• TracedMethods<int>.txt: method traces for each test executed, identified by an integer;

• all_data.csv: contains metrics that resume the set of tests executed (average energy consumed per test,  time, memory, etc);

• test<int>resume.json: json file with values for dynamic metrics obtained during test execution;

• begin_state<int>.json, end_state<int>.json: Metrics describing the state of the system before and after each test executed*;

• all_methods.json: all Java or Kotlin methods that our instrumentation approach can trace;

• <package>.json: all app methods that can be invoked, including methods not presented in the source code (available via heritance from framework classes)*.

• ...

With our empirical experiment we gathered many other metrics and that contain hundreds of GBs of data, such as the device logs during the tests execution. Given the difficulty of sharing  these large-size resources, we will only share these resources upon request. If you plan to use these resources for your research, please contact the authors.

Future Work Directions

The data contained in this dataset presents results from more than one month of continuous test execution. Given the extent of the work presented, from the process of extracting and filtering applications to the agglomeration and obtaining of its results, it is pertinent to share it with the community so it can be used in further studies aimed at analyzing the performance of applications or devices, comparisons of Android testing frameworks, evaluation of software evaluation aspects, between others. In this way, it is intended to contribute to the increase of knowledge regarding the factors that influence the performance of Android applications and their respective weight. Some examples of other research questions that might be answered by analyzing the data contained in this dataset or possible expansions of it are:

• How do different testing frameworks explore application performance?

• What is the most suitable testing framework to detect functional errors in applications?

• How does applications' performance evolve throughout their versions?

• What is the relation between semantic version changes and performance?