Design

An introduction to study design issues. 

Topics

• kinds of studies -- experiments, field studies, simulations, participant observation, interviews, etc

• validity, reliability, generalizability, robustness, utility

• measurement, descriptive, and inferential statistics

• probability and odds

• significance and power; p-value; null hypothesis

• P-hacking, harking, data sharing, publication pressures, junk science

• sampling

• Concepts of validity, reliability, generalizability, replicability

• Exploratory vs hypothesis testing 

• Qualitative versus quantitative

Handouts

• Types of studies

  • Design elements

  • Notation for describing experiments

  • Participation observation

  • Simulations

• Types of validity

• Validity and reliability

• Sampling. [html]

• Harking

• Reference sheet

Readings

• Murphy and Aguinis. HARKing: How Badly Can Cherry-Picking and Question Trolling Produce Bias in Published Results?

• Shadish, W, Cook, T. D., & Campbell, D. T., 2002. Experimental and quasi-experimental designs for generalized causal inference. [read preface and ch 1]

Materials

• Permutations. Illustrates permutation tests, as well as dangers of p-hacking

• class notes