Workshop on Optimizing Experimental Designs: Theory, Practice, and Applications

July 22 (Wednesday), 2015

Pasadena Convention Center, Pasadena, CA

Organizers: Jay Myung (Ohio State University), Mark Pitt (Ohio State University), Maarten Speekenbrink (University College London)

Introduction: The accurate and efficient measurement of observations is at the core of empirical scientific research. To ensure measurement is optimal, and thereby maximize inference, there has been a recent surge of interest among researchers in the design of experiments that lead to rapid accumulation of information about the phenomenon under study with the fewest possible measurements. Statisticians have contributed to this area by introducing methods of optimizing experimental design (OED). The methodology involves adapting the experimental design in real time as the experiment progresses. The purpose of this full-day workshop, held as part of the 2015 Annual Meeting of the Cognitive Science Society, is to introduce the principles underlying OED, illustrate how to apply OED in practice using widely and freely available software tools (e.g., R) to showcase applications of OED in areas such as cognitive psychology, education and assessment, and machine learning, and provide a platform to share work on OED.

Workshop format: The workshop will be organized around two specific goals: (1) to educate the cognitive science community about optimal experimental design (OED) and (2) to bring practitioners together who use it to share and showcase their latest work with the community. The first goal will be met in the morning session, which will include a 60-minute tutorial on the theoretical and computational foundations of OED given by Jay Myung and then another 60-minute hands-on session on the practical and implementation aspects of OED given by Maarten Speekenbrink. The second goal will be met by six 20-minute invited presentations featuring example applications demonstrating the use of OED in various disciplines.

Target audience: Graduate students, postdoctoral researchers, and scientists, who are new to OED and have workable knowledge of statistics on a graduate level.

Invited speakers: The following invited speakers have confirmed their participation:

Daniel Cavagnaro (California State University Fullerton, USA)

Christopher DiMattina (Florida Gulf Coast University, USA)

Woojae Kim (Ohio State University, USA)

Jay Myung (Ohio State University, USA)

Jonathan Nelson (Max Planck Institute for Human Development, GERMANY)

Anna Rafferty (Carleton College, USA)

Eric Schulz (University College London, UK)

Maarten Speekenbrink (University College London, UK)

Workshop Program

Selected readings on OED

Cavagnaro, D. R., Gonzalez, R., Myung, J. I., & Pitt, M. A. (2013). Optimal decision stimuli for risky choice experiments: An adaptive approach. Management Science, 59 (2), 358-375. [PDF]

DiMattina, C. & Zhang. K. (2013). Adaptive stimulus optimization for sensory systems neuroscience. Frontiers in Neural Circuits, 7 (101), 1-16. [PDF]

Kim, W., Pitt, M. A., Lu, Z.-L., & Myung, J. I. (under review). Planning beyond the next trial in adaptive experiments. [PDF]

Myung, J. I., Cavagnaro, D. R., & Pitt, M. A. (2013). A tutorial on adaptive design optimization. Journal of Mathematical Psychology, 57, 53-67. [PDF]

Rafferty, A. N., Zaharia, M., and Griffiths, T. L. (2014). Optimally designing games for behavioural research. Proceedings of the Royal Society Series A, 470. [PDF]

(Note: Website created and maintained by Jay Myung, myung.1@osu.edu; Last updated Jul 10 2015.)