For WeChat, we communicate with the developers of WeChat, who give us the common scenarios for evaluation. For the remaining apps, we identify common usage scenarios from reading the apps’ description on the Google Play store. Some of the scenarios identified by us require only one device while some scenarios require two. In either case, we identify one device to be the primary device while the other device to be the secondary device. The primary device is a Nexus 5X running Android 6.0 while the secondary device is a Nexus 5 running Android 6.0.
Recorded traces of the scenarios
Recorded videos of the scenarios
- Create post. This scenario requires two Facebook accounts that are already friends with each other. The main objective of this scenario is for the primary device to create a wall post and for the secondary device to like and comment on the post.
- Watch video. This scenario requires just the primary device. The main objective of this scenario is for the primary device to watch some cat videos through the Facebook app. More specifically, the user searches for ``scaredy cat'' in Facebook's video tab and then watches the first returned result.
- Share photo. This scenario requires just the primary device. The main objective of this scenario is for the primary device to share a personal photo from his gallery on to Facebook.
- Play games. This scenario requires just the primary device. The main objective of this scenario is for the primary device to browse the two popular games in the ``Instant Games'' embedded app of Facebook.
- Tutorial. This scenario only appears in the first run. The main objective of this scenario is to accept the license agreement and see some usage instructions.
- Explore menu. The main objective of this scenario is to learn about functions that can be invoked from different menus. Besides, the user closes an open window by clicking ``X'' button on it in the side menu.
- Browse directories. The main objective of this scenario is to utilize the file browsing feature. It starts from a list of virtual folders (like ``Download'', ``Videos'' and ``Main Storage''). Then the user goes into the first folder and gets back by clicking the home icon. The user also repeats such behavior for the next several folders.
- New windows. The main objective of this scenario is to utilize the multiple window feature. The user opens up the side menu and clicks the add button to create a new window. After repeating this for several times, the user clicks ``X'' buttons in the side menu one by one to close recently created windows.
All the scenarios require the user to log in to WeChat on devices in advance. Each device has its own WeChat account. All WeChat accounts are mutual contacts.
- Group chat/Link. Two devices are required in this scenario. The user creates a new group chat, opens a link in the chat and then navigates to Moments on the primary device. On the secondary device, the user posts a link in the group chat after it is created.
- Group chat/Photo. Two devices are required in this scenario. It is also required that there is at least one photo in galley on the secondary device. The user first creates a new group chat on the primary device. Then, a photo is posted to both the group chat and Moments on the secondary device. The photo is viewed in the group chat and Moments on the primary device. After that, the group conversation is deleted on both devices. The user deletes the photo in Moments on the secondary device as well.
- Gallery/Photo. Only one device is required in this scenario. It is also required that there is at least one photo in galley on the device. The user first launches the gallery app and opens a photo. Then, the user clicks the share icon and chooses to share the photo via WeChat. One WeChat contact is chosen to receive the photo. After that, the user views the photo sent and deletes the conversation.
- Moments/Photo\&Link. Two devices are required in this scenario. It is also required that there is at least one photo in galley on the secondary device. The user first posts a link and a picture to Moments separately on the secondary device. Then, the link is visited and the picture is viewed on the primary device. After that, the user deletes the posted link and picture on the secondary device.
- Moments/Post. Two devices are required in this scenario. The user first posts some texts to Moments on the primary device. Then, on the secondary device, the posted texts are commented. After that, the user deletes the posted texts on the primary device.
Besides the record-and-replay tools mentioned in the paper, we also found two additional tools that supports recording and replaying, namely Bot Maker and Troyd. These two tools differ from the ones mentioned in the paper because these two tools do not record the user's actions on the screen. Instead, to record, these tools require the user to manually specify what, where and how to click while the other tools mentioned in the paper can record simply by having the user click on the screen. For the purpose of our study we decided to omit such tools since these tools are generally more difficult for developers to record test cases.