Testing

How can we test the interdisciplinary insight(s) generated by the interdisciplinary research process? That is, what sort(s) of empirical evaluation might allow us to increase our confidence in these insights? Often, interdisciplinary insights are complex, and this can make testing more challenging.

[Repko, Interdisciplinary Research, 2011, 418-25 lists and integrates four broad kinds of test of the comprehensive understanding achieved through integration:

· Ask whether the results are useful. Do they solve the problem; answer the question? Do they support effective action?

· Do others find the research useful and interesting?

· Do we gain insight that is superior to what existed before? Is the interdisciplinary understanding better in some way than disciplinary understandings?

· Is the research program clear and were all steps performed well? In particular, are disciplinary insights appropriately represented in the interdisciplinary understanding? [See Assessing Interdisciplinary Research]

Note that the first three are holistic tests. The last instead suggests a number of more precise questions regarding how the research was performed. Both types of test are important. The proposed tests are complements.

Bammer devotes particular attention to the testing of policy proposals. If interdisciplinarians will investigate complicated systems of relations among diverse sets of phenomena, then there is likely to be a great deal of uncertainty surrounding any policy recommendations that might result from interdisciplinary research (indeed, Bammer identifies uncertainty as one of the hallmarks of interdisciplinary research). These recommendations must thus be explained carefully to policy-makers and the public, and carefully tested in practice. The idea that academics can simply produce ideas to be taken “off the shelf” by policy-makers will be especially inappropriate for interdisciplinary research. Rather, academics, policy-makers, and the public should work together in testing policy recommendations (Bammer, Gabriele (2005) “Integration and Implementation Sciences: Building a New Specialization,” Ecology and Society 10(2): 6.

URL: http://www.ecologyandsociety.org/vol10/iss2/art6/

Bammer, Gabriele Disciplining Interdisciplinarity (2013) provides a set of questions that a researcher could ask at each stage of the research process.

Scholz, Roland W., Embedded Case Study Methods: Integrating Quantitative and Qualitative Knowledge suggests six practical criteria for assessing the validity of a collaborative case study:

· Do study members agree?,

· Do case members [those being studied] agree?,

· Does the public agree?,

· Do disciplinary experts agree? (there are dangers of disciplinary bias here),

· Do transdisciplinarians think that the case followed appropriate methods (and was a transdisciplinary approach appropriate)?

· Does the case study lead to positive impacts? [The latter two criteria replicate criteria above]