Update: Since a lot of people are finding this blog post, please note you can download the practical primer I've written about calculating and reporting effect sizes here: http://openscienceframework.org/project/ixGcd/ On this page, you can also download a spreadsheet to calculate effect sizes when you have the data, and my new effect size spreadsheet (From_R2D2) that you can use to calculate effect sizes from the published literature, or that can be used to convert between effect sizes.
Until approximately one
month ago, I had the following understanding of effect sizes. If you do a I am tempted to write υ =
.21 after an Just as in Fight Club,
where nobody is supposed to talk about Fight Club but they end up all knowing
about where to go for the next Fight Club, I’ll talk about what I didn’t know
about effect sizes. When I tried to calculate an a-priori sample size from the
results of a paired-samples I didn’t know that for a One-Way ANOVA, partial eta squared is the
same as eta-squared. The fact that Something else I didn’t
know, was that you can always calculate partial eta squared from the F-value,
and the two degrees of freedom associated with an F-test. For example, if an
articles gives Another thing I didn’t know is that when you are performing an a-priori power analysis for a within-subject design, you should not directly insert the partial eta squared value that SPSS provides into G*Power. G*Power by default uses a different way to calculate partial eta squared, and using the SPSS version will give you a wrong sample size estimate. It is an easy mistake to make. I only figured it out when I tried to compare sample size estimates from an a-priori power analysis for a paired t-test and a repeated measures ANOVA, and had to e-mail the G*Power team to ask for an explanation (who replied within an hour with the answer – they are great). There are published articles that make this mistake, and studies with a sample size that is assumed to lead to 95% power, while the actual power of the study is much lower. However, the most important thing I didn’t know is how easy it is to understand effect sizes. Sure, there are many situations where there are different, all equally defensible, ways in which you can calculate an effect size. And calculating generalized omega squared for a 2X3 mixed model design where you’ve thrown in a covariate for good measure will probably take you the better part of an afternoon (but don’t worry, there will be only about 12 people in the world that are able to judge whether the value you calculated is correct or not). But for most practical purposes, and most of the studies you have done so far, it’s really pretty easy. I wrote a short article that explains how you can calculate effect sizes for t-tests and ANOVA’s. You might wonder why you would want to read it, given that I just explained how incredibly limited my knowledge of effect sizes was a month ago. I don’t blame you. I surely didn’t think I would be explaining others how to calculate effect sizes. So why did I write the article? It started just as some note taking when I tried to figure out what to do. But then I started to get annoyed. There are some good books about effect sizes (.g., Aberson, 2010; Cohen, 1988; Cumming, 2012; Ellis, 2010; Grissom & Kim, 2005; Maxwell & Delaney, 2004; Murphy, Myors, & Wolach, 2012) but I don’t expect you to spend the next 5 weeks reading them. Besides some minor annoyances (e.g., information being spread out over 2 dozen articles, a focus on between-subject designs, despite the prevalence of within-designs in experimental psychology, describing a lot of different effect sizes and their unbiased estimates, but not providing guidance in which effect sizes to report for what) my major annoyance was that the articles provided formula’s, and left it up to the reader to figure out what to do with them. I don’t know many statisticians, so I’m just going to assume they are empathic, friendly people, who understand most individuals are not very interested in statistics, and therefore try to make it as easy as possible for people to use formula’s (for excellent examples, see ESCI (Cumming & Finch, 2001) and G*Power (Faul, et al., 2009). I would expect these pro-social individuals to realize that the larger part of the scientific community just wants to report the correct effect size with as little effort as possible (which is a very rational goal), and that authors would make it easy for researchers to calculate effect size by providing, oh I don’t know, a spreadsheet? So in addition to the article, I made a spreadsheet (download here: http://openscienceframework.org/project/ixGcd/). It has a decision tree (which was a great suggestion by Job van Wolferen) to guide you to the correct calculation. You fill in the required numbers in the green cells, and the grey cells provide the output you need to report the effect size. A short sentence is provided as an example of how to report the outcome of the statistical test, and the effect sizes. It should be pretty easy to use. The article and spreadsheet are not yet peer-reviewed, so I can give no guarantees, but I’m pretty sure all calculations are correct. However, if you know more about this than I do, and you find some mistakes, let me know, and I’ll update the spreadsheet. I've submitted the manuscript, because peer-review can only improve it, but I have no idea whether this would be interesting for a journal. However, what's more important for me is that I sincerely hope this is useful for some of you, and this will make it easier for you to report the information that will allow other people to perform a-priori power analyses when building on your work, or to include your studies in a meta-analysis. |

Blog >