Guidelines for Conducting and Reporting Case Study Research in Software Engineering

Abstract:

Case study is a suitable research methodology for software engineering research since it studies contemporary phenomena in its natural context. However, the understanding of what constitutes a case study varies, and hence the quality of the resulting studies. This paper aims at providing an introduction to case study methodology and guidelines for researchers conducting case studies and readers studying reports of such studies. The content is based on the authors’ own experience from conducting and reading case studies. The terminology and guidelines are compiled from different methodology handbooks in other research domains, in particular social science and information systems, and adapted to the needs in software engineering. We present recommended practices for software engineering case studies as well as empirically derived and evaluated checklists for researchers and readers of case study research.

Research Methodology:

• Survey: which is the “collection of standardized information from a specific population, or some sample from one, usually, but not necessarily by means of a questionnaire or interview”.
• Experiment/Controlled Experiment: which is characterized by “measuring the effects of manipulating one variable on another variable” and that “subjects are assigned to treatments by random.”.
• Quasi-Experiments are similar to controlled experiments, except that subjects are not randomly assigned to treatments. Quasi-experiments conducted in an industry setting may have many characteristics in common with case studies.
• Action Research: with its purpose to “influence or change some aspect of whatever is the focus of the research”, is closely related to case study. More strictly, a case study is purely observational while action research is focused on and involved in the change process.
• Ethnographic Studies: as a specialized type of case studies with focus on cultural practices or long duration studies with large amounts of participant-observer data. Case studies do not generate the same results on e.g. causal relationships as controlled experiments do, but they provide deeper understanding of the phenomena under study.

Case Study Research Process:

1. Case study design: objectives are defined and the case study is planned.
2. Preparation for data collection: procedures and protocols for data collection are defined.
3. Collecting evidence: execution with data collection on the studied case.
4. Analysis of collected data
5. Reporting

Case Study Design and Planning:

1. Defining a Case:
   • Objective—what to achieve?
   • The case—what is studied?
   • Theory—frame of reference & Research questions—what to know?
   • Methods—how to collect data?
   • Selection strategy—where to seek data?
2. Case Study Protocol: is a container for the design decisions on the case study as well as field procedures for its carrying through. The protocol is a continuously changed document that is updated when the plans for the case study are changed.

1. Ethical Considerations
2. Checklist

Summary:

Case study research is conducted in order to investigate contemporary phenomena in their natural context. That is, no laboratory environment is set up by the researcher, where factors can be controlled. Instead the phenomena are studied in their normal context, allowing the researcher to understand how the phenomena interact with the context. Selection of subjects and objects is not based on statistically representative samples. Instead, research findings are obtained through the analysis in depth of typical or special cases.

Cases study research is conducted by iteration over a set of phases. In the design phase objectives are decided and the case is defined. Data collection is first planned with respect to data collection techniques and data sources, and then conducted in practice. Methods for data collection include, for example, interviews, observation, and usage of archival data. During the analysis phase, insights are both generated and analyzed, e.g. through coding of data and looking for patterns. During the analysis it is important to maintain a chain of evidence from the findings to the original data. The report should include sufficient data and examples to allow the reader to understand the chain of evidence.