From_R2D2

Post date: Jan 28, 2014 8:21:07 AM

When we want to run well-powered experiments, we need to estimate the effect size we expect to observe. One source of information can be related articles published in the literature. Sometimes you need to convert between a reported effect size (e.g., r) and the effect size you need for your power analysis (e.g., Cohen’s d). Since not everyone reports effect sizes, sometimes you’ll need to know how to calculate effect sizes based on the test statistics (t and F values) or perhaps even only based on the p-value and N. I created an effect size conversion spreadsheet, From_R2D2 that can help you with this. You can download it from the Open Science Framework: https://osf.io/ixgcd/osffiles/From_R2D2.xlsx/. There are several other resources to convert effect sizes online, but I still thought I could add a little to existing options for the following reasons.

1) It’s called From_R2D2. Nuff said.

2) I tried to make a spreadsheet that clearly tells you what you are getting out, depending on what you are putting in, and to prevent people from making conversions that are not correct. Some spreadsheets do not clearly differentiate between Cohen’s d_s and Cohen’s d_pop which annoys me, even though the difference is often small. I like adjustments for bias (such as r_adjusted instead of r), and if you are using a spreadsheet anyway, it’s not any more work, so it makes sense to use adjusted effect sizes.

3) You don’t have to know what you can calculate based on what you have. You just put in the information you have available, and the spreadsheet will turn green if it could calculate anything. So, it requires less knowledge of what you can calculate – you just give in as much info as possible, and the program will let you know what you can get out of it.

4) From_R2D2 helps you to calculate the most accurate effect sizes. For example, if you have the number of participants in each condition, it will give back a more accurate conversion than when you only have the total N, and provides clear pointers what to use in the form of tooltips.

5) From_R2D2 is also useful if you have a within design. Although you need the correlation or SD’s to calculate Cohen’s d_av or d_rm (see Lakens, 2013), it’s relatively straightforward to calculate r (based on the fact that in both within as between designs, the relation between a t-value and F-value is F = t*t). At the same time, it will help to prevent conversions that do not make sense (e.g., converting r to d_pop in a within design).

6) From_R2D2 also provides the common language effect size. It’s a pretty interesting way to interpret Cohen’s d, and I’ll continue to try to make it more accessible for people (see also Lakens, 2013).

7) You read point 1, right?

Perhaps you already have a conversion tool that works for you, but perhaps you don’t and you find my spreadsheet useful. It complements my other spreadsheet and article on calculating effect sizes if you have the raw data (and which allows you to calculate cool things such as generalized eta squared). Together with Paul Turchan and Andy Woods we are working on a free iPhone and Android app that will allow you to perform these and other calculations, but this is a weekend project so it will take some weeks before that is ready.

I recently wrote a blog post about the replicability of published research, and how it is a characteristic of the data, and not a characteristic of the researcher. In that post, I hinted there are ways you can get an idea of the likelihood a study will replicate. To perform the calculations necessary to get an idea of this likelihood, we’ll need r_adjusted, which you can calculate with this spreadsheet. Yes, there is an order to this madness.