HTTP

In order to answer RQ3, "Are there significant performance differences between apps that use different competing libraries implementing typical programming tasks?", we selected a set of 7 programming tasks typically performed by Android apps. These libraries are listed in Table 12, organized in networking over HTTP, Collections, Image rendering/loading, JSON manipulation, I/O, Logging and Threading.

Taking into account the extent of the analysis carried out, we needed to select a limited set of tasks to be considered. Furthermore, we considered tasks with at least 2 alternatives/libraries, which have methods to be invoked in at least 5 different apps, across at least 10 different versions, and with at least 25 different test scenarios. The libraries under analysis are some of the most used libraries for Android for each category, according to AppBrain.

In the following figures, we present the results obtained from the evaluation of 3 performance indicators of tests executed over the apps containing some of the identified HTTP Libraries.

In the case of libraries for HTTP, apps that use Volley tend to have more energy-efficient tests, suggesting that this API might be the most energy-efficient API, despite being the worst in terms of run-time when considering the median and top whisker of the box plots of the Figures. In our experiment, Picasso seems to be the most memory and run-time efficient, being the worst in terms of energy. These results are different from the ones from another research study [37] that analyzed the run-time performance of Volley and Retrofit, having concluded that Volley is faster and offers more features than Retrofit. This divergence can be again justified by the fact that our setup consists of a different execution context, considering different real-world applications and many execution contexts, while the experimental setup used in [ 37] considered only one application connected to a custom server, aiming at analyzing and measuring only specific methods of these APIs. However, in these Figures, we can see that despite Volley obtained a higher median in the memory results, tests using Retrofit obtained less consistent results, where many tests had high-consuming tests above the upper percentile when comparing to the values of the tests that used the Volley library.