Advanced Research Methods

Advanced Research Methods

• P-hacking

  • Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological science, 0956797611417632.

  • P-hacked hypotheses are deceivingly robust

  • P-curves. Full website including links to articles and an app that lets YOU make P-curves. The app is very easy to use.

  • Pritschet, L., Powell, D., & Horne, Z. (2016). Marginally Significant Effects as Evidence for Hypotheses Changing Attitudes Over Four Decades.Psychological Science, 0956797616645672.

  • Geoff Cumming thinks P-Hacking Could be to blame for the Reproducibility Crisis (from Oct 2016)

  • The Replication Crisis

  • Psychology's Replication Crisis Has a Silver Lining. Article by Paul Bloom 2/19/2016

  • The replication crisis is a good thing

  • Are bad theories (or badly specified theories) the cause of the replication crisis? link to the paper

  • Are most research findings actually false? (pre-print)

    • Is there really a crisis? 3/18/2016

    • Interview about replication with the Editor of Psychological Science from 1/16/2016

    • The APS's official registered replications page

    • A personal perspective on the replication crisis

  • Hüffmeier, J., Mazei, J., & Schultze, T. (2016). Reconceptualizing replication as a sequence of different studies: A replication typology. Journal of Experimental Social Psychology.

  • Can "context sensitivity" partially explain the negative results of the 2015 Open Science Collab replication project?

  • 70% of scientists say they have failed to reproduce a result. What??

• The Open Science Collaboration

  • Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7(6), 657-660.

  • Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.

  • What is the OSC? Article in Science Magazine 6/26/2015

  • 40% full is not 60% empty (opinion)

  • Dan Gilbert is skeptical of the skeptics in the OSC

• Retractions

  • Data manipulation (Feb 2017)

• The New Statistics

  • Geoff Cumming's "The New Statistics"

  • Evaluating Psychological Science's backing away from NHST

  • You are USING p-VALUES WRONG click here for help

• Problems for Publishers

  • Improving the peer review process: a proposed market system

  • Why publishing negative findings is hard. Retraction Watch 2.17.2016

  • Retractions Aren't Enough. Retraction Watch article by Andrew Gelman 5/19/2016

  • 10 Suggestions for the editors of Psychological Science

    • Possible solution to the "file drawer problem"

  • The Pipeline Project

    • One CS journal's solution

• Problems for real people (you're a real people)

  • Scientific fraud hurts people

  • John Oliver's segment on science reporting in the media is hilarious and timely. Warning: foul language. It's hilarious though.

  • Research indicates that when studies are contradicted, the new evidence isn't taken as seriously as the original finding. Isn't this just the anchoring bias?

  • Toward the Future of Science

  • Defining reproduction

  • Acknowledging and Overcoming Nonreproducibility in Basic and Preclinical Research by Ionnadis (Feb 2017)

    • On Perverse Incentives and Replication in Science by Douglas Campbell PhD (econ; March 2017)