How to Write Critically

• Highlight strengths and weaknesses. Add some words reflecting study quality, for example: “a large sample of 8,132,” “a small sample of American university students (N = 154),” “an adequately-powered randomized controlled trial of 100 MDD patients in South Korea.” Words like ‘small,’ ‘large,’ ‘adequately-powered’ help the readers understand the quality of these works. Other points succinctly give sample details (e.g., ‘American university students’).

You may also bring in outside sources. For example, you might comment on the statistical analyses “authors used PCA, however, nearly all experts recommend using EFA in this situation (citation to another source).

Why? Why is the point you mention a strength or weakness? The example above shows the authors did not follow recommended protocol, that is very different from stating “authors found all items showed high factor loadings.” This statement provides no comment on the appropriateness of the analyses but assumes the results are valid and strong. You don’t have to expand on such points every time. For example, if you state “authors used a small sample of university students” you may not have to comment on ‘small,’ but you might want to mention why university students might not be representative or appropriate for this work.

• Be concise but informative. Here are a couple of examples:

“In their study on the Stroop effect and first language background, Etal and colleagues (2018) employed a robust randomized controlled trial approach with 150-210 participants in each of five language groups. A double-masked design helped prevent researcher and social desirability biases. However, the study focused on Indo-European languages and did not include Sino Tibetan, Dravidian or Austric languages, and there were no participants over 50 years old.”

Here, in three sentences, we see the type of study, rough sample size, and specific strengths and weaknesses. We might include more details depending on the nature of our report.

“The Happy Idiot Scale (HIS) was developed with four samples (Ns ranged 487 to 5,022) with a 30-item item pool and determined a three-factor solution (citation). The authors reported high internal consistency and validity for the HIS but did not state what type of factor analysis was used, nor did they report coefficient alpha for other measures used in the study. This brings into question..(citation to expert study on analyses).”

In these three sentences we see details of the study purpose, sample sizes, and what the authors reported. We also see critical analysis of what is missing – details of how analyses were done.

Long or brief critical review? Sometimes you might just mention a couple of study details, strengths and weaknesses, sometimes you might spend a paragraph or more (rare) on this. Try not to put in too much, the emphasis should still be on their findings, with help to readers on how to interpret the soundness of those findings.

What NOT to report. Rarely do you need to report central tendency stats (e.g., M, SD). Report that only for comparison purposes if needed. For percentages and risk ratios etc., remember that those stats are usually sample-specific – other samples and the general population will likely have different ratios/proportions etc. Therefore, there is no need to be very specific. For example: “their study found about half of participants reported ABC symptoms” is preferable to “51.22% had ABC.” Another example is: “males had about twice the likelihood of developing XYZ.” Rather than “Males are 2.24 times (OR = 2.24) more likely to develop XYZ.” Really, there should be error bars/confidence intervals around nearly all of our statistics. The field of psychology is moving in that direction, thanks to lots of work by statisticians, software engineers, psychometricians, etc. The studies you review may not provide CIs or error terms for many stats, and you may not have studied how to interpret some that do. A good approach is to imagine rough CIs and to not take these findings so literally.