INCLINE

Online Appendix

Description

The research prototype, INCLINE, allows to integrate cloned variant code with intentions, visualizes the integration progress to the user in five views, and ultimately executes the declared intentions producing a configurable integrated platform.

The tool specifically addresses product lines implemented using #ifdefs which is commonly used in C and C++ applications, e.g., Linux uses #ifdef for thousands of optional features. As such we focus on the use of the preprocessor for variability rather than to enable reuse (using macro expansion).

We use JetBrains’ language workbench, The Meta-Programming System (MPS)[ 1 ], as a basis for implementation of INCLINE[ 2 ]. In projectional editing, also known as syntax-directed or structural editing[ 41 ], user’s program-editing gestures directly change the underlying AST, unlike in traditional character-based editing. The AST is still rendered into concrete syntax that the user sees. Projectional editing is well-suited for creating editable views needed in INCLINE’s method (e.g., the integrated side-by-side view).

Process. The tool design followed an iterative improvement process, with a prototyping and evaluation feedback loop. In our first prototype of INCLINE, we implemented the basic functionality: source file import, user interface to declare intentions, and back-end to resolve intentions. After that, we have executed a pilot user study, to detect and resolve inefficiencies in the tool.

Implementation. We have implemented a simplified version of the C preprocessor language ( #ifdef , #else , #if , #elif , #endif directives) using MPS’s meta-modeling facilities.

Evaluation

Subject Systems

Marlin ( > 40 KLOC of C++ code) has over 4000 forks, many of which evolve separately, and independently add new functionality. Given this richness and the existing re-integration efforts of the community, Marlin is an ideal subject for our evaluation. We mined from 1,715 approved pull requests and three forks.

BusyBox ( > 160 KLOC of C code) is a collection of common shell commands (e.g., grep, cut) for embedded Linux applications. We use a fork tailored for Android. ( https://github.com/jcadduono/android_external_busybox)

Vim ( > 300 KLOC of C code) is a popular terminal-based text editor extending the Unix vi standard. We derived an integration task using a fork maintaining support for OS/2. (Interestingly, this fork remains active at the time of writing.) (https://github.com/h-east/vim/tree/clpum)

Libuv (> 50 KLOC of C code) is a popular IO library.

PHP (> 1,000 KLOC of C code) is the interpreter for the PHP scripting language.

Pre-Study 1 (Sec. IV.)

To validate the functionality and usability of the rst prototype, we have recruited 16 MSc students to execute two integration tasks using INCLINE. We compared performance of INCLINE against Eclipse merge tools as a baseline. We extracted two tasks from histories of two real open-source C projects (BusyBox and Vim). Each task concerned integration of code from a fork to the mainline. The participants were given a correct target solution (the result of integration) and a brief description of the integration goal.

Tasks

pre-study-tasks.pdf

Experiments

RQ1: What is the coverage of the proposed set of intentions with respect to real-world integration needs? (Sec VI. A)

To show that the defined set of intentions is complete enough to handle real-world integration tasks, we replay non-trivial merges from history of Marlin. To identify complex merge tasks, we extract those that had conflicts, yielding 49 merges. Through further filtering, we select 35 commits that we use to replay using intentions.

A few real-world examples from the set of 35 commits, where intentions are applied can be found in this page: Intentions Examples

For Libuv: we retrieved 101 merge commits from the project's history, 54 of those are with conflicts, and 15 of those have changes to #ifdefs.

For PHP: we retrieved 14,836 merge commits from the project's history, in a sample of 5,000 of those merge commits we found 63 commits with conflicts, 12 of those with changes to #ifdefs/#else

We replayed 10 of the identified commits from libuv and PHP each, in addition to the commit from Marlin.

Conclusion: The proposed set of intentions suffices for real-world variant integration.

RQ2: Is the implementation of intention resolution correct with respect to their semantics, even for large files? (Sec VI. B)

We validate that INCLINE produces correct results when intentions are assigned correctly, and that we can use it on large files without scalability problems. We use a diverse collection of integration tasks, to ensure a good coverage of this test, including applications of intentions in diverse contexts. We simulate ten integrations in total. The frst group of seven commits is selected randomly from the 35 merge commits of the previous experiment. The second group are three integration tasks selected from Marlin forks—we use the second group to avoid focusing on merge conflict commits only

The tasks used in this experiment can be downloaded from here: https://drive.google.com/open?id=1OX4U--wdVE851tcofdksoYIjidVeI92H

Conclusion: INCLINE produces semantically correct output and it scales to files up to 4K lines of code without difficulties.

RQ3: What is the benefit of INCLINE over manual variant integration from users’ perspective?

(Sec VI. C)

We conduct a controlled experiment with 12 developers (experienced PhD students) familiar with the C preprocessor. We use a 2x2 within-subjects counterbalanced Latin square design and reuse the Vim and BusyBox examples from the pre-study. Participants are trained through a video tutorial on how to use both tools, as well as being instructed on preprocessor usage (they only needed to use #ifdef , #else , and #endif ). Each participant first solves a training task extracted from Marlin, to get familiar with the tools. The training task performance is not included in the experiment results. During the actual experiment, we record the screen and log keystrokes (in Eclipse) and intentions (in INCLINE). We measure the performance of subjects using proxies: number of mistakes per task, the time to complete each task, and the number of edit operations (and number of intentions) applied per task.

Tasks and descriptions

group1-tasks.pdf

group2-tasks.pdf

Conclusion: INCLINE supports integration with much lower error rate than Eclipse merge tools, with significantly less edit operations, at no significant increase in cost (time).

RQ3: User Survey (Sec VI. C)

Questionnaire: (L - likert scale, 1 Strongly Disagree, 5 Strongly Agree; O - open question)

1. L Integration with INCLINE is faster than in Eclipse.

2. L I make fewer mistakes in INCLINE than in Eclipse.

3. L Mistakes are easier to notice in INCLINE than in Eclipse.

4. L Mistakes are easier to fix in INCLINE than in Eclipse.

5. L INCLINE is a mature tool.

6. L INCLINE is not mature enough.

7. L Intention-based integration is not complex.

8. L The Keep intention is intuitive.

9. L The Remove intention is intuitive.

10. L The Implicit Keep/Remove intentions are intuitive.

11. L The Keep as Feature intention is intuitive.

12. L The Exclusive intention is intuitive.

13. O Which intentions did you find the most useful?

14. O What are the advantages of using intentions for variant integration compared

to manual integration with Eclipse?

15. O What are your perceived disadvantages of using intentions for variant integration

compared to manual integration with Eclipse?

16. O Are there any possible improvements to intention-based integration or in

particular INCLINE?

Survey responses as PDF file.

survey-answers-exp2.pdf

Heatmap Survey - 1 Strongly Disagree, 2 Disagree, 3 Neutral, 4 Agree, 5 Strongly Agree

heat-survey-exp2.pdf

INCLINE IDE Tool

Download

The INCLINE tool can be downloaded as a Virtual Box image from here:

Installation

If you wish to install and use the INCLINE VM, please use the following guidelines.

Download and install the latest Virtual Box https://www.virtualbox.org/wiki/Downloads
If you have installed a previous version, make sure it is at least version 5 (otherwise guest addition will not work!) If it's not, please install the latest version.
Download the VM image; it is failry large ~ 4GB, so make sure you have space and download it beforehand as it takes a while.
Import the image into your Virtual Box via the Import Appliance button from File menu.
Make sure you have 15GB free of HDD wherever you select to have the virtual machine folder.
When importing, you might select the size of the RAM for the virtual machine. Make sure you do not select the same size as the physical RAM installed in your machine. Give it 4-8 GB depending on your memory installed.
Start the VM and log in into group 1 with group1 as password.
Start the INCLINE tool by double clicking on MPS icon. Once it has started, navigate to Examples, Imported2, and open a file from there. Once the file has opened, press ALT+V to start the 4 view panel. Alternatively, use the Tools menu on the top bar. If you want to switch from side-by-side-view (bottom left), to the integrated view, right click in that editor panel, select push editor hints, and then deselect side-by-side view checkbox.
Keyboard shortcuts: Keep - ALT+K; Remove: ALT+R; Keep as Feature: ALT+F; Save Intentions: ALT+S; Export a file: ALT+X. Update the preview view: ALT+U.

Google Sites

Report abuse