CAMA workshop

Community-Augmented Meta-Analyses: How to get good CAMA?

A satellite event of EMLAR XI (Experimental Methods in Language Acquisition Research)

Timeline

22.01.2015 First announcement of the upcoming CAMA (Community-Augmented Meta-Analysis) workshop

31.03.2015 Registration deadline

14.04.2015 Workshop date (Drift 25, room 0.03, Utrecht, NL)

15.04.-17.04. 2015 EMLAR XI (Experimental Methods in Language Acquisition Research, Utrecht, NL)

About

When carefully looking into a research topic, it is often the case that many papers report on relevant findings. But all these reported experiments might be difficult to compare due to differences in stimuli, methods, and statistics on the outcomes. Still, is it necessary to combine all data on a well-defined topic, not only to get a better look at the underlying "true" effect and its variance, but also to estimate how many participants are necessary in a future study and to integrate seemingly conflicting findings into a coherent framework.

Meta-analytic methods are, despite allowing for aggregating and comparing data from different studies, still rare. One of the goals of this workshop is to lower the hurdles to using meta-analyses.

A second goal is to promote the use and sharing of already conducted meta-analyses. A CAMA (Community-Augmented Meta-Analysis) combines meta-analyses with emerging trends towards continuously accumulating evidence and an increased need for transparency and data accessibility. CAMAs allow researchers to build on each others' work: they collect evidence for an effect (and its potential moderators) and can be re-used in their most recent form for specific purposes (eg. gaining an unbiased overview of the literature, calculating power previous to data collection, answering new questions).

Schedule

Morning session (10-12): Introduction to CAMAs

• Introduction to meta-analyses
• How to (and why) calculate prospective power before starting to collect data
• Introduction to existing CAMAs (open, shared meta-analyses that can be re-used and expanded)
• Overview of building your own CAMA
• (If time permits) Publication biases: an overview over a hotly debated topic and a brief introduction to p-curves

Afternoon session (14-17): Hands-on experience with CAMAs

(Note: bring your own laptop to join)

• Using existing CAMAs to calculate power
• Asking your own questions
• Visualizing data in CAMAs
• Expanding existing CAMAs (extracting and entering new data)

And here are the slides

Further Information

1. CAMAs:
   1. The paper: S. Tsuji, C. Bergmann, & A. Cristia. 2014. "Community-augmented meta-analyses: Toward cumulative data assessment." In Perspectives on Psychological Science (Special Section on Moving Toward a Cumulative Science: Maximizing What Our Research Can Tell Us). Volume 9, Number 6, pp. 661-665. DOI: 10.1177/1745691614552498 (open access)
   2. CAMA on Infant Word Segmentation
   3. CAMA on Vowel Acquisition
   4. A bar camp on CAMAs in language development research
   5. How to build your own CAMA
2. Meta-analyses etc
   1. The PRISMA statement on structured reviews and meta-analyses, which comes with a checklist and a flowchart
   2. D. Lakens. (2013). Calculating and Reporting Effect Sizes to Facilitate Cumulative Science: A Practical Primer for t-tests and ANOVAs. Frontiers in Psychology 4:863. Material: osf.io/ixgcd/files/
   3. P-curve online tool and detailed explanation
   4. Lakens: What p-hacking really looks like (with supplementary material)