Do you trust opinion surveys about bioethics topics? How do we know a survey is valid? Validity has two aspects: Does it measure what it purports to measure (internal validity)? Does the survey sample faithfully represent the group of interest (external validity)? External validity can be described in numbers and so it is easier to evaluate.
Internal validity is harder to measure. How do we know the respondents truly understood the issues and the questions? Do their answers reflect 'considered' judgments, the kind of judgments that we hope would inform our policies?
My obsession with these questions about informing bioethics policy with considered views of the relevant stakeholders has led to a series of studies using a method borrowed from political science, called democratic deliberation. I call it our 'wedding banquet' research because we typically use a large banquet hall with round tables of 6-8 people each with trained facilitators. We usually take an entire day with experts representing various sides of an issue with previously vetted educational materials, small groups deliberations, and pre- and post-deliberation surveys. Also, we conduct these as randomized experiments (comparing attendees of our deliberation days with different control groups).
Some of our deliberation study materials (facilitator guides, plenary session powerpoints, surveys used, etc) can be found here.
Kim SYH, Wall IF, Stanczyk A, De Vries R. Assessing the Public’s Views in Research Ethics Controversies: Deliberative Democracy and Bioethics as Natural Allies. Journal of Empirical Research on Human Research Ethics 2009; 4(4): 3-16.
Kim SYH, Uhlmann RA, Appelbaum PS, Knopman DS, Kim HM, Damschroder L, Beattie E, Struble L, De Vries R. Deliberative Assessment of Surrogate Consent for Dementia Research: Views of Caregivers. Alzheimer’s and Dementias 2010; 6(4): 342-350.
De Vries R, Stanczyk A, Uhlmann RA, Damschroder L, Kim SYH. Assessing the quality of democratic deliberation: A case study of public deliberation on the ethics of surrogate consent for research. Social Science and Medicine 2010; 70 (12):1896-1903.
Kim SYH, Kim HM, Knopman DS, De Vries R, Damschroder L, Appelbaum PS. Effect of Public Deliberation on Attitudes Toward Surrogate Consent for Dementia Research. Neurology 2011;77(24):2097-2104.
De Vries R, Stanczyk A, Ryan K, Kim SYH. A Framework for Assessing the Quality of Democratic Deliberation: Enhancing Deliberation as a Tool for Bioethics. Journal of Empirical Research on Human Research Ethics 2011; 6(3): 3-17.
Goold SD, Neblo M, Kim SY, DeVries R, Rowe G, Muhlberger P. What is good quality public deliberation? Hastings Center Report 2012;42(2):24-6.
Gornick MC, Scherer AM, Sutton EJ, Ryan KA, Exe NL, Li M, Uhlmann WR, Kim SYH, Roberts JS, De Vries RG. Effect of Public Deliberation on Attitudes toward Return of Secondary Results in Genomic Sequencing. Journal of Genetic Counseling. 2016. DOI: 10.1007/s10897-016-9987-0.
Tom Tomlinson T, De Vries RG, Kim HM, Gordon L, Krenz CD, Ryan KA, Jewell S, Kim SYH. Effect of Deliberation on the Public’s Attitudes Toward Consent Policies for Biobank Research. European Journal of Human Genetics. 2018;26(2):176-185.
De Vries RG, Ryan KA, Gordon L, Krenz CD, Tomlinson T, Jewell T, Kim SYH. Biobanks and the Moral Concerns of Donors: A Democratic Deliberation. Qualitative Health Research. 2018:1049732318791826.
Kim SYH. Theory and Practice of Democratic Deliberation in Bioethics Research. Empirical Bioethics: Theoretical and Practical Perspectives. J Ives, M Dunn, and A Cribb, editors. Cambridge University Press, 2017.
Kuylen MN, Kim SYH, Ruck Keene A, Owen GS. Should age matter in COVID-19 triage? A deliberative study. Journal of Medical Ethics 2021. DOI: 10.1136/medethics-2020-107071