ISCB2023

Papers as background to ISCB 2023 course on Prediction model performance

Performance assessment for binary outcome models
Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW.
Assessing the performance of prediction models: a framework for traditional and novel measures.
Epidemiology. 2010 Jan;21(1):128-38. doi: 10.1097/EDE.0b013e3181c30fb2. (pdf)

Performance assessment for survival models
DJ McLernon, D Giardiello, B Van Calster…
Assessing performance and clinical usefulness in prediction models with survival outcomes: practical guidance for Cox proportional hazards models
Annals of Internal Medicine 2023 

Performance for competing risk models
N Van Geloven, D Giardiello, EF Bonneville, L Teece…
Validation of prediction models in the presence of competing risks: a guide through modern methods
BMJ 2022

Van Calster B, Nieboer D, Vergouwe Y, De Cock B, Pencina MJ, Steyerberg EW.

A calibration hierarchy for risk models was defined: from utopia to empirical data.
J Clin Epidemiol. 2016 Jun;74:167-76. doi: 10.1016/j.jclinepi.2015.12.005

Assessing discriminative ability for categorical outcomes: polytomous or ordinal
Van Calster B, Vergouwe Y, Looman CW, Van Belle V, Timmerman D, Steyerberg EW.
Assessing the discriminative ability of risk models for more than two outcome categories.
Eur J Epidemiol. 2012 Oct;27(10):761-70. doi: 10.1007/s10654-012-9733-3

Van Calster B, Van Belle V, Vergouwe Y, Timmerman D, Van Huffel S, Steyerberg EW.
Extending the c-statistic to nominal polytomous outcomes: the Polytomous Discrimination Index.
Stat Med. 2012 Oct 15;31(23):2610-26. doi: 10.1002/sim.5321

Van Calster B, Van Belle V, Vergouwe Y, Steyerberg EW.
Discrimination ability of prediction models for ordinal outcomes: relationships between existing measures and a new measure.
Biom J. 2012 Sep;54(5):674-85. doi: 10.1002/bimj.201200026

Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, Roobol MJ, Steyerberg EW.
Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators.
Eur Urol. 2018 Dec;74(6):796-804. doi: 10.1016/j.eururo.2018.08.038

Original proposal for DCA
Vickers AJ, Elkin EB.
Decision curve analysis: a novel method for evaluating prediction models.
Med Decis Making. 2006 Nov-Dec;26(6):565-74. doi: 10.1177/0272989X06295361

Perspective on internal and external validation

Steyerberg EW, Harrell FE Jr.
Prediction models need appropriate internal, internal-external, and external validation.
J Clin Epidemiol. 2016 Jan;69:245-7. doi: 10.1016/j.jclinepi.2015.04.005

Riley RD, Debray TPA, Collins GS, Archer L, Ensor J, van Smeden M, Snell KIE.
Minimum sample size for external validation of a clinical prediction model with a binary outcome.
Stat Med. 2021 Aug 30;40(19):4230-4251. doi: 10.1002/sim.9025

Machine learning is data hungry
van der Ploeg T, Austin PC, Steyerberg EW.
Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints.
BMC Med Res Methodol. 2014 Dec 22;14:137. doi: 10.1186/1471-2288-14-137

Guidelines for quality improvement of prediction with machine learning and AI
de Hond AAH, Leeuwenberg AM, Hooft L, Kant IMJ, Nijman SWJ, van Os HJA, Aardoom JJ, Debray TPA, Schuit E, van Smeden M, Reitsma JB, Steyerberg EW, Chavannes NH, Moons KGM.
Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review.
NPJ Digit Med. 2022 Jan 10;5(1):2. doi: 10.1038/s41746-021-00549-7

Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS.
Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.
Ann Intern Med. 2015 Jan 6;162(1):W1-73. doi: 10.7326/M14-0698