We conducted a controlled experiment to evaluate the comprehensibility of contextual variability modeling by comparing eCFM and CFM techniques. We asked the subjects to perform tasks of mapping, reading, and modifying elements of two DSPL projects: a smart home system and a mobile application. These tasks are available on the following links: Tasks for Group 1 and Tasks for Group 2. Table 1 describes the main aspects of the study design.

To achieve our work goal, we identified two main research questions:

• RQ1. Which is the most expressive variability modeling technique to deal with DAS projects?
• RQ2. Which is the easiest variability modeling technique to model DAS projects?

In this sense, we asked for the subjects answer a form to background characterization available on: Background Characterization. Moreover, they answered two forms to provide their impressions about both approaches. These forms are available on: Feedback Questions 1 and Feedback Questions 2.

We asked whether they agree about eCFM expressiveness. From the results, 93.3% agree or totally agree that eCFM is more expressive. Only a subject disagreed about this question. When we asked about the most suitable technique to model DSPL projects, all subjects answered eCFM. Among their justifications, some of them stated that eCFM technique has a greater expressive power to represent adaptation rules among contexts and system features.

Next, we asked also about four specific topics regarding to the easiness of use both techniques. Figure 1 shows the results of these questions, which compare eCFM and CFM regarding to the easiest technique to:

• Understand the modeling;
• Model the DAS elements;
• Include new elements; and
• Model complex scenarios of DAS projects.

The four pie charts indicates positive results for our approach.

Figure 1 Answers from the Feedback Form

References

[1] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in Software Engineering. Springer, 2012.

[2] I. Hadar, I. Reinhartz-Berger, T. Kuflik, A. Perini, F. Ricca, and A. Susi, “Comparing the comprehensibility of requirements models expressed in Use Case and Tropos: Results from a family of experiments,” Information and Software Technology, vol. 55, no. 10, pp. 1823–1843, 2013.

[3] K. Saller, M. Lochau, and I. Reimund, “Context-aware dspls: Model- based runtime adaptation for resource-constrained systems,” in 17th International Software Product Line Conference, ser. SPLC ’13 Work- shops. ACM, 2013, pp. 106–113.