Fundamentally, appraisal of medical literature simply has to do with evaluating how to respond to information presented and determining how (or even, whether) to incorporate it into practice. Most simply, reading an article and deciding whether it does or does not “make sense” is a very basic level of appraisal, while far more in-depth critique and assessment is necessary for the publication of systematic reviews, guidelines, and health policies. Many and varied sets of recommendations have been published to guide critical appraisal in specific settings; this guide does not seek to replicate or replace these, rather it is intended as a basic guide to aid critical appraisal of medical literature for application at the point of patient care. All of the questions raised in the guide are addressed in more depth in the references for each question, to which links are provided below.
Printable two-page version of this guide to critical appraisal available here
Reviewing medical literature for Patient-Oriented Evidence that Matters (1)
Is this patient-oriented?
1) If the information is true, will it require me to change my practice? (2) Is this feasible in my practice? Is this an outcome about which my patients will care [eg, Mortality, Morbidity, Cost, Simplicity]? (3)
2) Is a clearly focused question investigated in an appropriate group of patients? (4) Is the question clear? Were appropriate inclusion and exclusion criteria applied to select patients? Are the patients that were included sufficiently similar to your patients to allow for application of the findings to your patients? For systematic reviews and meta-analyses, were appropriate criteria used to select individual studies of appropriate patients? Was follow-up sufficiently long for answering the question?
What is the evidence?
1) Is there adequate justification of the methods chosen to answer the question, and do the methods seem appropriate? (4) For intervention or treatment studies, were patients randomly allocated to appropriate comparisons? For diagnostic test studies, were all patients assessed with both the test being evaluated and an appropriate reference standard? For cohort studies, was there appropriate and consistent classification of the presence or absence of the exposure (or risk factor) in question? For studies of clinical prediction rules, was there appropriate clinical and statistical determination of variables included, and was there independent validation of the rule in a group of patients different from the group used to derive the rule? For qualitative studies of patients’ subjective experiences, is there appropriate explanation and justification of the methods used?
2) Is there appropriate justification of, explanation of, and measurement of outcomes? (4) Are the reported outcomes relevant to patients’ morbidity, mortality, or other quality-of-life considerations, or are they disease-oriented surrogate markers or lab tests? Is the primary outcome appropriately specified and reported? Are the secondary outcomes and any adverse outcomes reported appropriately? Was there appropriate blinding of participants, health workers, and outcome assessors to ensure objective outcome assessment?
Does it matter to my patient(s)?
1) What are the results? Are they of sufficient magnitude and precision to be clinically useful? (5) (6)
2) For intervention or treatment studies, what are the results for the primary, secondary, and adverse outcomes? (6) (7) How can the relative and absolute benefits and harms of the intervention(s) under consideration be described?[eg, how many patients out of 10 or 100 are likely to experience each of the primary, secondary or adverse outcomes?]
3) For studies of diagnostic tests or prediction rules, is sufficient information presented to allow calculation of positive and negative predictive values, and likelihood ratios? (6) (8) How can the meaning of a normal or abnormal result on the test be presented to your patient(s)?
4) How would you rate this using the Strength of Recommendation Taxonomy (SORT)? (3) See details on page 2
Strength of Recommendation Taxonomy (SORT) (3)
Strength Definition
Based on consistent and good quality patient-oriented evidence
Based on inconsistent or limited quality patient-oriented evidence
Based on consensus, usual practice, opinion, disease-oriented evidence, and case series for studies of diagnosis, treatment, prevention, or screening.
References
(1) Shaughnessy AF, Slawson DC, Bennett JH. Becoming an information master: a guidebook to the medical information jungle. J Fam Pract. 1994 Nov;39(5):489-99. PMID: 7964548. (https://www.ncbi.nlm.nih.gov/pubmed/7964548)
(2) Slawson DC, Shaughnessy AF. Obtaining useful information from expert based sources. BMJ. 1997 Mar 29;314(7085):947-9. PMID: 9099121. (https://www.bmj.com/content/314/7085/947.long)
(3) Ebell MH, Siwek J, Weiss BD, Woolf SH, Susman J, Ewigman B, Bowman M. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician. 2004 Feb 1;69(3):548-56. PMID: 14971837 (http://www.aafp.org/afp/20040201/548.html)
(4) Critical Appraisal Skills Programme (2018). CASP Checklists. [online] Available at: https://casp-uk.net/casp-tools-checklists/ Accessed: 11 July 2019
(5) Montori VM, Kleinbart J, Newman TB, Keitz S, Wyer PC, Moyer V, Guyatt G; Evidence-Based Medicine Teaching Tips Working Group. Tips for learners of evidence-based medicine: 2. Measures of precision (confidence intervals). CMAJ. 2004 Sep 14;171(6):611-5. PMID: 15367466. (http://www.cmaj.ca/content/171/6/611)
(6) Schrager SB. Five Ways to Communicate Risks So That Patients Understand. Fam Pract Manag. 2018 Nov/Dec;25(6):28-31. PubMed PMID: 30422613. (https://www.aafp.org/fpm/2018/1100/p28.html)
(7) Barratt A, Wyer PC, Hatala R, McGinn T, Dans AL, Keitz S, Moyer V, For GG; Evidence-Based Medicine Teaching Tips Working Group. Tips for learners of evidence-based medicine: 1. Relative risk reduction, absolute risk reduction and number needed to treat. CMAJ. 2004 Aug 17;171(4):353-8. Review. PMID: 15313996. (http://www.cmaj.ca/content/171/4/353)
(8) Schranz DA, Dunn MA. Evidence-based medicine, part 3. An introduction to critical appraisal of articles on diagnosis. J Am Osteopath Assoc. 2007 Aug;107(8):304-9. PMID: 17785687. (https://jaoa.org/article.aspx?articleid=2093492)
Open access article series on evidence-based medicine and critical appraisal
Tips for Learners of Evidence-Based Medicine, CMAJ, 2005: https://www.ncbi.nlm.nih.gov/sites/myncbi/bill.cayley.1/collections/58716654/public/
Evidence-Based Medicine, JAOA, 2007: https://www.ncbi.nlm.nih.gov/sites/myncbi/bill.cayley.1/collections/48271577/public/