A class of research methods focusing on multi-method synthesis of the literature is emerging especially in healthcare. These methods would fall under the umbrella of methodological pluralism. Healthcare, especially nursing [Paterson et al., 2001] and evidence-based medicine [Sandelowski and Barroso, 2007], have embraced multi-method research approaches that synthesize diverse sets of qualitative and quantitative data. These methodologically diverse approaches are ideal tools for healthcare IT researchers who must understand both the complexity of the healthcare domain and increasingly sophisticated inter-organizational multi-actor usage of healthcare information systems.
Multi-method research embraces both type of method (qualitative to quantitative) and the breadth of analysis techniques that exist (empirical to synthesis). While quantitative meta-analysis has existed for years, multi-method research typically have a qualitative foundation. The umbrella for multi-method research in education is methodologically inclusive research synthesis (MIRS) [Suri and Clarke, 2009]. MIRS includes methods such as research synthesis, meta-analysis, meta-ethnography, meta-synthesis, meta-study, qualitative meta-analysis, qualitative synthesis, aggregated analysis, and synthesis of qualitative research. Meta-study (interpretive qualitative), popular in nursing [Paterson et al., 2001], consists of meta-data analysis, meta-theory, meta-method and meta-synthesis. Research synthesis in management and organization studies consists of a combination of narrative synthesis, meta-ethnographic, meta-narrative, and realist approach [Denyer et al., 2008]. The IS field has its meta-triangulation approach which is a form of qualitative meta-analysis [Saunders, 2003, Webster and Watson, 2002].
The most recent example of using a synthesis approach is a January 2010 article in the Milbank Quarterly by Greenhalgh, Potts, Wong, Bark and Swinglehurst at University College London titled "Tensions and Paradoxes in Electronic Patient Record Research: A Systematic Literature Review Using the Meta-narrative Method". At the moment (of this posting) the article is free to download at this link. This is one of the first (if not the first) use of a synthesis approach in the information systems literature. The meta-narrative uses interpretive synthesis by reading and re-reading primary sources and summarize their key methods and findings via a narrative.
Multi-method research is not new. Two articles published in Information Systems Journal (ISJ) pointed to the scarcity of multi-method research [Chen and Hirschheim, 2004, Mingers, 2003]. ISJ editors reminded us again that mixed-method research, an entree to multi-methods research, constituted only 5.8% of their papers [Avison et al., 2008]. For the AIS Top Basket journals, mixed methods account for only 6% of published articles between 1998 and 2007 [Myers and Liu, 2009].
Mixed methods have been referred to as the third research paradigm by various authors although a matter of debate. My view is more pragmatic and in line with the definitions of Morse that follow [2002]. Mixed-methods use both quantitative and qualitative methods within a single project. Multi-methods have a greater reliance on synthesis as inter-related projects comprised of quantitative and qualitative methods are brought together to address a broader topic. One of the few examples of a mixed-method approach from an information systems perspective is a meta-narrative review of electronic patient records [Greenhalgh et al., 2009]. It was published in a health policy journal and only seven of the health-related references from over 100 studies reviewed are from AIS Top Basket journals.
References
Avison, D. E., Y. K. Dwivedi, G. Fitzgerald, and P. Powell (2008) "The beginnings of a new era: time to reflect on 17 years of the ISJ," Information Systems Journal (18) 1, pp. 5-21.
Chen, W. and R. Hirschheim (2004) "A paradigmatic and methodological examination of information systems research from 1991 to 2001," Information Systems Journal (14) 3, pp. 197-235.
Denyer, D., D. Tranfield, and J. E. Van Aken (2008) "Developing Design Propositions through Research Synthesis," Organization Studies (29) 3, pp. 393-413.
Mingers, J. (2003) "The paucity of multimethod research: a review of the information systems literature," Information Systems Journal (13) 3, pp. 233-249.
Morse, J. (2002) Principles of Mixed-and Multi-Method Research Design in A. Tashakkori and C. Teddlie (Eds.) Handbook of mixed methods in social & behavioral research: Sage Publications, pp. 189-208.
Myers, M. and F. Liu. (2009) What Does the Best IS Research Look Like? An Analysis of the AIS Basket of Top Journals. PACIS 2009 Proceedings, 2009.
Paterson, B. L., C. Canam, S. E. Thorne, and C. Jillings (2001) Meta-Study of Qualitative Health Research: A Practical Guide to Meta-Analysis and Meta-Synthesis. Thousand Oaks, California: Sage.
Sandelowski, M. and J. Barroso (2007) Handbook for synthesizing qualitative research. New York: Springer.
Saunders, C. C., Traci A.; Jasperson, 'Jon (Sean); and Butler, Brian S. (2003) "Lessons from the Trenches of Metatriangulation Research," Communications of the Association for Information Systems (11) 1, pp. Available at: http://aisel.aisnet.org/cais/vol11/iss1/14
Suri, H. and D. Clarke (2009) "Advancements in Research Synthesis Methods: From a Methodologically Inclusive Perspective," Review of Educational Research (79) 1, pp. 395-430.
Webster, J. and R. T. Watson (2002) "Analyzing the Past to Prepare for the Future: Writing a Literature Review " MIS Quarterly (26) 2, pp. xiii-xxiii.
Methodological Pluralism
There are many arguments against methodological pluralism including the level of expertise required for studying software engineering (link) and that qualitative methods aren't holistic enough for healthcare (link). See Daniel Little's essay arguing for methodological pluralism in social sciences. Inevitably, healthcare IT research must include a social element. So being pragmatic, how does one conduct research where the social and material are given equal weight or ideally seen as an integrated whole (i.e., sociomateriality).
Lilford, Foster and Pringle (200) wrote a PLOS Medicine essay on need for methodological pluralism in evaluating the implementation of Health IT systems. The use of multiple levels of analysis (patient and system) along with identifying the causal pathway makes the essay an interesting read.
Interpretive Synthesis
TBD
References
Dixon-Woods, M, Cavers, D, Agarwal, S, Annandale, E, Arthur, A, Harvey, J, Hsu, R, Katbamna, S, Olsen, R, Smith, L, Riley, R & Sutton, A 2006, 'Conducting a Critical Interpretive Synthesis of the Literature on Access to Healthcare by Vulnerable Groups', BMC Medical Research Methodology, Vol. 6, no. 1, p. 35.
Flemming, K 2010, 'Synthesis of Quantitative and Qualitative Research: An Example Using Critical Interpretive Synthesis', Journal of Advanced Nursing, Vol. 66, no. 1, pp. 201-17.
Lomas, J 2005, 'Using Research to Inform Healthcare Managers and Policy Makers Questions: From Summative to Interpretive Synthesis', Healthcare Policy, Vol. 1, no. 1.
Weed, M 2008, 'A Potential Method for the Interpretive Synthesis of Qualitative Research: Issues in the Development of Meta-Interpretation', International Journal of Social Research Methodology, Vol. 11, no. 1, pp. 13 - 28.