Introductory Papers and Guides

Original Paper

2006

Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006 Nov-Dec;26(6):565-74. PMID: 17099194.

Best Basic Introduction

2016

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016 Jan 25;352:i6. PMID: 26810254.

How to Understand a Decision Curve

2019

Vickers, AJ, van Calster, B & Steyerberg, EW. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res 3, 18 (2019).

Guide for Reporting Decision Curves

2018

Ben Van Calster, Laure Wynants, Jan FM Verbeek, Jan Y Verbakel, Evangelia Christodoulou, Andrew J Vickers, Monique J Roobol, Ewout W Steyerberg. Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators. European Urology, Volume 74, Issue 6, 2018, Pages 796-804, ISSN 0302-2838

Extensions to DCA

Applications to Survival Time Data, Competing Risks, and Cross Validation

2008

Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak. 2008;8:53. Published 2008 Nov 26.

DCA & Treatment Response: Model Evaluation

2007

Vickers AJ, Kattan MW, Daniel S. Method for evaluating prediction models that apply the results of randomized trials to individual patients. Trials. 2007;8:14. Published 2007 Jun 5.

DCA & Treatment Response: Trial Design

2006

Vickers AJ, Kramer BS, Baker SG. Selecting patients for randomized trials: a systematic approach based on risk group. Trials. 2006 Oct 5;7:30. PMID: 17022818; PMCID: PMC1609186.

Net Benefit Under Resource Constraints

2023

Karandeep Singh, Nigam H Shah, Andrew J Vickers, Assessing the net benefit of machine learning models in the presence of resource constraints, Journal of the American Medical Informatics Association, Volume 30, Issue 4, April 2023, Pages 668–673

Discussion Papers and Theoretical Background

Theoretical Background (1)

2008

Vickers AJ. Decision analysis for the evaluation of diagnostic tests, prediction models and molecular markers. Am Stat. 2008;62(4):314-320

Theoretical Background (2)

2010

Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol. 2010;37(1):31-38.

Mathematical Underpinnings

2009

Baker SG, Cook NR, Vickers A, Kramer BS. Using relative utility curves to evaluate risk prediction. J R Stat Soc Ser A Stat Soc. 2009 Oct 1;172(4):729-748. PMID: 20069131; PMCID: PMC2804257.

Introduction of 3-Step Approach of Discrimination, Calibration, Clinical Utility

2010

Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010 Jan;21(1):128-38. PMID: 20010215; PMCID: PMC3575184.

Systematic review of how decision curves used in the literature

2019

Capogrosso P, Vickers AJ. Med Decis Making. 2019 Jul;39(5):493-498. PMID: 30819037.

Decision curve analysis: confidence intervals and hypothesis testing for net benefit

2023

Vickers AJ, Van Claster B, Wynants L, Steyerberg EW. Decision curve analysis: confidence intervals and hypothesis testing for net benefit. Diagn Progn Res. 2023 Jun 6;7(1):11. PMID: 37277840; PMCID: PMC10243069.

Editorials & Commentaries in Major Journals

Journal of Clinical Oncology

2016

Kerr KF, Brown MD, Zhu K, Janes H. Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol. 2016 Jul 20;34(21):2534-40. PMID: 27247223; PMCID: PMC4962736.

British Medical Journal

2016

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016 Jan 25;352:i6. PMID: 26810254; PMCID: PMC4724785.

The Journal of the American Medical Association

2015

Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. 2015 Jan 27;313(4):409-10. PMID: 25626037.

Annals of Internal Medicine

2012

Localio AR, Goodman S. Beyond the usual prediction accuracy metrics: reporting results for clinical decision making. Ann Intern Med. 2012 Aug 21;157(4):294-5. PMID: 22910942.

Decision Curves for Time-to-Event Data

2023

McLernon DJ, et al. Ann Intern Med. 2023 Jan;176(1):105-114. PMID: 36571841.

Other Editorials

European Radiology

2022

Vickers AJ, Woo S. Decision curve analysis in the evaluation of radiology research. Eur Radiol. 2022 Sep;32(9):5787-5789. Epub 2022 Mar 29. PMID: 35348862.

Spine

2021

Vickers AJ, Holland F. Decision curve analysis to evaluate the clinical benefit of prediction models. Spine J. 2021 Oct;21(10):1643-1648. Epub 2021 Mar 3. PMID: 33676020; PMCID: PMC8413398.

Acta Neurochirurgica

2022

Mijderwijk HJ, Nieboer D. Is My Clinical Prediction Model Clinically Useful? A Primer on Decision Curve Analysis. Acta Neurochir Suppl. 2022;134:115-118. PMID: 34862535.

The European Respiratory Journal

2021

Sadatsafavi M, Adibi A, Puhan M, Gershon A, Aaron SD, Sin DD. Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models. Eur Respir J. 2021 Nov 4;58(5):2101186. PMID: 34735641.

DCA & Machine Learning/AI

DCA & Machine Learning/AI (1)

2019

Christodoulou E, et al. J Clin Epidemiol 2019;110:12–22.

DCA & Machine Learning/AI (2)

2021

Jung K, et al. J Am Med Inform Assoc 2021; 28 (6): 1149–58.

DCA & Machine Learning/AI (3)

2023

Dhiman P, Ma J, Andaur Navarro CL, Speich B, Bullock G, Damen JAA, Hooft L, Kirtley S, Riley RD, Van Calster B, Moons KGM, Collins GS. Journal of Clinical Epidemiology. 2023; ISSN 0895-4356.

DCA & Machine Learning/AI (4)

2022

de Hond AAH, Leeuwenberg AM, Hooft L, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. npj Digit Med. 2022 Jan 20;5(1):2. PMID: 35348862; PMCID: PMC8393232.

DCA & Machine Learning/AI (5)

2024

Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024 Apr 18;385:q902. PMID: 38636956; PMCID: PMC11025451.