PubHawk
BMC Med December 2019 PMID 31842878

Calibration: the Achilles heel of predictive analytics.

Ben Van Calster, David J McLernon, Maarten van Smeden, Laure Wynants, Ewout W Steyerberg

Full text PDF Download social card

Mentions

167
Peak: 23 in Feb 2023
across 4 sources since Dec 2019

Extended reach

980,232
potential audience

Total likes

514

Video views

50

Mentions per month

Dec 2019 – Sep 2025

Where it's discussed

  • X / Twitter160 posts
  • Blogs4 articles
  • YouTube2 videos
  • Bluesky1 posts

AI synopsis

What people are saying

Many discussions emphasize calibration's role in predictive analytics, with users noting statements like 'Calibration is the Achilles heel of predictive analytics' and sharing critiques of common tests like the Hosmer-Lemeshow test. The tone of these discussions ranges from critical to supportive, with phrases indicating urgency such as 'You want in the small generally speaking' and 'Calibration is your friend,' which underscores the necessity of addressing calibration for better predictive outcomes.

AgreementModerate agreement

The majority of discussions indicate a consensus on the importance of calibration in predictive models, suggesting moderate to strong agreement with the article's viewpoints.

InterestHigh interest

There is a high level of interest, evidenced by multiple users sharing and discussing key points from the article and expressing curiosity about calibration methods and their implications.

EngagementModerate engagement

Engagement seems moderate; while many users reference calibration, only a subset delve into specific methodologies or detailed discussions, indicating some depth but not exhaustive exploration.

ImpactHigh impact

The perceived impact is considered high, as users frequently describe calibration as critical and refer to its potential consequences for clinical decision-making.

Abstract in brief

  • Calibration performance in risk prediction models often gets overlooked, but it's crucial because poorly calibrated algorithms can mislead clinical decisions.
  • To prevent poor calibration, it's important to balance the complexity of the model with the size of the data sample during development and validation.
  • Updating algorithms and continuously evaluating their calibration can enhance their effectiveness in supporting clinical practice and improving patient counseling.
Read the full abstract

Top posts

Most liked on X and Bluesky
Richard Riley (R²)
@Richard_D_Riley · X

February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 79 likes · 22 shares · 12,871 followers View post →
Gary Collins
@GSCollins · X

"it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the algorithm performs in a particular setting, where predictions may go wrong" A must read to understand the importance of calibration https://t.co/C15LPsnxE5 https://t.co/N6jA1gkukm

Apr 9, 2022 · 78 likes · 15 shares · 10,360 followers View post →
Gary Collins
@GSCollins · X

A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to try & show how well it predicts -> Calibration is your friend https://t.co/JArFHziq69 Focussing only on rank based measures is not enough #statstwitter https://t.co/6sEkAhCYdJ

Jan 25, 2022 · 58 likes · 15 shares · 10,360 followers View post →
BMC Medicine
@BMCMedicine · X

Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t.co/bIWwSNFmoU #BMCMedicine focus on #validation continues. Also read by @MaartenvSmeden et al https://t.co/koEmi9mSyj https://t.co/hCLwOO1gFh https://t.co/RVw5pLuNDI

Sep 6, 2023 · 30 likes · 14 shares · 9,194 followers View post →
Valeriy M., PhD, MBA, CQF
@predict_addict · X

“Calibration: the Achilles heel of predictive analytics” https://t.co/EZFPY669yu #calibration #conformalprediction https://t.co/9uHsbtdJej

Nov 3, 2023 · 29 likes · 8 shares · 36,090 followers View post →
Ewout Steyerberg
@ESteyerberg · X

And here we emphasize calibration for Predictive Analytics as a field: https://t.co/C4WtqetISU @BenVanCalster While clinical usefulness is an overarching metric https://t.co/4ewcfoPU2t @VickersBiostats https://t.co/kgtYNzQcqH

Jan 21, 2022 · 27 likes · 4 shares · 4,294 followers View post →

All mentions

Yonemoto N
@nyonenyone · X

https://t.co/d9QpJCJgep

Sep 18, 2025 · 1 like · 1,272 followers View post →
Maarten van Smeden
@maartenvsmeden.bsky.social · Bluesky

7) Goodness-of-fit Hosmer-Lemeshow test The HW test is often used as a test for the calibration of prediction models. Unfortunately, it is not a good test for calibration doi.org/10.1186/s12916-019-1466-7

Dec 23, 2024 · 14 likes · 2 shares View post →
Joshua Fein
@joshuaafein · X

RT @DrAEvens: @drjgauthier Yep. #Calibration is the Achilles heel of most prognostic/predictive models.

Oct 8, 2024 · 2 shares · 891 followers View post →
David Russler-Germain, MD/PhD
@dgermain21 · X

RT @DrAEvens: @drjgauthier Yep. #Calibration is the Achilles heel of most prognostic/predictive models.

Oct 8, 2024 · 2 shares · 3,669 followers View post →
Andrew M. Evens, DO, MBA, MSc
@DrAEvens · X

@drjgauthier Yep. #Calibration is the Achilles heel of most prognostic/predictive models.

Oct 8, 2024 · 3 likes · 2 shares · 3,484 followers View post →
Hazel Gibson
@HazelGibsonAIMS · X

Calibration: the Achilles heel of predictive analytics | https://t.co/9XqBUX8teH #CustomerAI #AI #AICustomerSegmentation #AIforCustomerSegmentation #ArtificialIntelligence [Video] https://t.co/WbCExbTddc

Aug 29, 2024 · 22 followers View post →
Nick Mark MD
@nickmmark · X

RT @doc_BLocke: @GuyattGH roughly equal wins/losses on discrimination, better calibration, and prediction+judgement > judgement alone... ye…

Aug 21, 2024 · 1 share · 65,693 followers View post →
Brian Locke, MD MSCI
@doc_BLocke · X

@GuyattGH roughly equal wins/losses on discrimination, better calibration, and prediction+judgement > judgement alone... yet the summ is "you can't presume model discrimination is better?" No one presumed that! Odd interpretation. https://t.co/RU5rCiftkK https://t.co/hHBfPCdRot

Aug 21, 2024 · 2 likes · 1 share · 1,294 followers View post →
David Russler-Germain, MD/PhD
@dgermain21 · X

RT @DrAEvens:

Aug 20, 2024 · 1 share · 3,669 followers View post →
Andrew M. Evens, DO, MBA, MSc
@DrAEvens · X

Aug 19, 2024 · 7 likes · 1 share · 3,484 followers View post →
Maarten van Smeden
@MaartenvSmeden · X

@topepos @BenVanCalster I would further guess that the results will be somewhat disappointing in terms of what we can say something about the predictions after (re-)calibration, this is because recalibration can only give some guarantees on "weak calibration" https://t.co/fCa4J6LETH

May 24, 2024 · 3 likes · 39,855 followers View post →
Yuta Aoki, D.Sc.
@YutaMAoki · X

RT @predict_addict: https://t.co/a5MLPEWZuH

Apr 30, 2024 · 1 share · 617 followers View post →
Valeriy M., PhD, MBA, CQF
@predict_addict · X

https://t.co/a5MLPEWZuH

Apr 30, 2024 · 6 likes · 1 share · 36,090 followers View post →
David Russler-Germain, MD/PhD
@dgermain21 · X

RT @DrAEvens: …a start. We need more rigor, including CALIBRATION: how the predicted outcome is to the actual/observed outcome in external…

Apr 26, 2024 · 2 shares · 3,669 followers View post →
Andrew M. Evens, DO, MBA, MSc
@DrAEvens · X

RT @DrAEvens: …a start. We need more rigor, including CALIBRATION: how the predicted outcome is to the actual/observed outcome in external…

Apr 26, 2024 · 2 shares · 3,484 followers View post →
Spyros Iatropoulos
@spyros_iatrop · X

How can we calculate the slope of a calibration curve of a prognostic model? (https://t.co/nykLyqC9gx) I am trying to fit a logistic function to the curve, but results seem wrong. Any help please? @BenVanCalster @MaartenvSmeden @Richard_D_Riley @GSCollins

Mar 7, 2024 · 1 like · 113 followers View post →
Asurari
@Asurariii · X

RT @predict_addict: Calibration the Achilles heel of predictive analytics a must read paper by @MaartenvSmeden https://t.co/I5Gv8yyT0u #…

Jan 13, 2024 · 4 shares · 247 followers View post →
Lawrence
@lawrence0321bk · X

RT @predict_addict: Calibration the Achilles heel of predictive analytics a must read paper by @MaartenvSmeden https://t.co/I5Gv8yyT0u #…

Jan 13, 2024 · 4 shares · 305 followers View post →
thebluenote
@thebluenote6 · X

RT @predict_addict: Calibration the Achilles heel of predictive analytics a must read paper by @MaartenvSmeden https://t.co/I5Gv8yyT0u #…

Jan 13, 2024 · 4 shares · 3 followers View post →
Matej
@MatejKorvin · X

RT @predict_addict: Calibration the Achilles heel of predictive analytics a must read paper by @MaartenvSmeden https://t.co/I5Gv8yyT0u #…

Jan 13, 2024 · 4 shares · 86 followers View post →
Valeriy M., PhD, MBA, CQF
@predict_addict · X

Calibration the Achilles heel of predictive analytics a must read paper by @MaartenvSmeden https://t.co/I5Gv8yyT0u #calibration https://t.co/B3PRt9gFHt

Jan 13, 2024 · 13 likes · 4 shares · 36,090 followers View post →
SANDRAJOHNSTON
@sandra04850184 · X

https://t.co/A49Cy3TAEe

Dec 11, 2023 · 950 followers View post →
Junior_prompt_engineer
@bert_on_spec · X

RT @predict_addict: “Calibration: the Achilles heel of predictive analytics” https://t.co/EZFPY669yu #calibration #conformalprediction h…

Nov 3, 2023 · 8 shares · 1,548 followers View post →
Cumi
@idelacruzm · X

RT @predict_addict: “Calibration: the Achilles heel of predictive analytics” https://t.co/EZFPY669yu #calibration #conformalprediction h…

Nov 3, 2023 · 8 shares · 23 followers View post →
priya joseph
@ayirpelle · X

RT @predict_addict: “Calibration: the Achilles heel of predictive analytics” https://t.co/EZFPY669yu #calibration #conformalprediction h…

Nov 3, 2023 · 8 shares · 4,923 followers View post →
Yurii
@exdddee · X

RT @predict_addict: “Calibration: the Achilles heel of predictive analytics” https://t.co/EZFPY669yu #calibration #conformalprediction h…

Nov 3, 2023 · 8 shares · 40 followers View post →
Valeriy M., PhD, MBA, CQF
@predict_addict · X

“Calibration: the Achilles heel of predictive analytics” https://t.co/EZFPY669yu #calibration #conformalprediction https://t.co/9uHsbtdJej

Nov 3, 2023 · 29 likes · 8 shares · 36,090 followers View post →
Seena Fazel
@seenafazel · X

Calibration is arguably the most important performance measure - it tests how close predicted are to observed probabilities – a key metric in the field: https://t.co/XBhaB4Dti3. For OxSATS, reported here: https://t.co/OojLaVaJjQ

Sep 28, 2023 · 3 likes · 3,358 followers View post →
Seena Fazel
@seenafazel · X

Main results - in external validation, calibration good. Here are observed/expected plots. Some overestimation at higher end - relevant to v few cases. Calibration is rarely reported but key - see good paper: https://t.co/XBhaB4Dti3 https://t.co/ADPOhHpMLT

Sep 20, 2023 · 1 like · 3,358 followers View post →
Pedro Lopez-Ayala
@LopezAyalaP · X

@manuel_carnero @cirujano_cardio Ahora bien, esto es la teoría y como apunta @manuel_carnero el modelo se ha validado externamente y muestra una buena discriminación y calibración, dos requisitos obligatorios y de hecho para mi la calibración es incluso más importante, por lo que... https://t.co/fJtbWxhuNW

Sep 6, 2023 · 1 like · 274 followers View post →
M. Bolton
@5_utr · X

RT @BMCMedicine: Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t…

Sep 6, 2023 · 14 shares · 2,386 followers View post →
Altay de Souza
@altayals · X

RT @BMCMedicine: Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t…

Sep 6, 2023 · 14 shares · 17,262 followers View post →
José A. Ávila Cabreja
@a_cabreja · X

RT @BMCMedicine: Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t…

Sep 6, 2023 · 14 shares · 80 followers View post →
David A. Oliver
@DavidAOliverJr · X

RT @BMCMedicine: Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t…

Sep 6, 2023 · 14 shares · 927 followers View post →
Giuseppe Biondi-Zoccai
@gbiondizoccai · X

RT @BMCMedicine: Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t…

Sep 6, 2023 · 14 shares · 12,164 followers View post →
Nicole White
@nicolem_white · X

RT @BMCMedicine: Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t…

Sep 6, 2023 · 14 shares · 878 followers View post →
Maarten van Smeden
@MaartenvSmeden · X

RT @BMCMedicine: Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t…

Sep 6, 2023 · 14 shares · 39,855 followers View post →
BMC Medicine
@BMCMedicine · X

Evidence of questionable research practices in clinical #predictionmodel exposed by @nicolem_white @aidybarnett https://t.co/bIWwSNFmoU #BMCMedicine focus on #validation continues. Also read by @MaartenvSmeden et al https://t.co/koEmi9mSyj https://t.co/hCLwOO1gFh https://t.co/RVw5pLuNDI

Sep 6, 2023 · 30 likes · 14 shares · 9,194 followers View post →
Gary Collins
@GSCollins · X

@DrMheissen @JClinEpi

Aug 23, 2023 · 1 like · 10,360 followers View post →
Maarten van Smeden
@MaartenvSmeden · X

Hosmer–Lemeshow test: often used to test for calibration in a logistic regression model. But it is not a test for calibration https://t.co/fCa4J6LETH

Aug 17, 2023 · 8 likes · 39,855 followers View post →
Danny Whiting
@Dr_dannywhiting · X

N.B. Calibration esp. important if use absolute % to inform decisions. Tool w/good AUC can still be systematically off-target. Examining also allows simple updating to baseline risk in new settings (as we did here). Calibration often overlooked, see

Jun 21, 2023 · 406 followers View post →
MrHonkster
@MrHonkster · X

RT @MaartenvSmeden: @TomBoylesID If you are really interested here is quite some info: https://t.co/fCa4J6LETH

May 11, 2023 · 2 shares · 96 followers View post →
Michael Henry
@brainplast · X

RT @MaartenvSmeden: @TomBoylesID If you are really interested here is quite some info: https://t.co/fCa4J6LETH

May 11, 2023 · 2 shares · 385 followers View post →
Maarten van Smeden
@MaartenvSmeden · X

@TomBoylesID If you are really interested here is quite some info: https://t.co/fCa4J6LETH

May 11, 2023 · 11 likes · 2 shares · 39,855 followers View post →
Sebastian Marciano
@sebas_marciano · X

Finally, most models are judged by their discrimination (AUC), but even models with modest discrimination might be very useful, so it is essential also to check their calibration too! https://t.co/tooQoHIwLs

Apr 30, 2023 · 2 likes · 573 followers View post →
Andrew M. Evens, DO, MBA, MSc
@DrAEvens · X

…a start. We need more rigor, including CALIBRATION: how the predicted outcome is to the actual/observed outcome in external groups- aka the accuracy of absolute risk estimates. https://t.co/RXRE7w1J9w And https://t.co/c0pBhGaGwa 2/3

Apr 27, 2023 · 2 likes · 2 shares · 3,484 followers View post →
Karen Bonilla-Aguilar
@karelibonilla · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 2, 2023 · 22 shares · 23 followers View post →
epi_zzz
@epi_Zzzz · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 2, 2023 · 22 shares View post →
Anselmo Alegría M.
@chemoignacio · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 2, 2023 · 22 shares · 230 followers View post →
Sofia Cividini
@SofiaCividini · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 2, 2023 · 22 shares · 49 followers View post →
Miguel Á. Armengol de la Hoz
@miguearmengol · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 1,162 followers View post →
Domenico Bellomo
@BellomoDomenico · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 416 followers View post →
David Russler-Germain, MD/PhD
@dgermain21 · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 3,669 followers View post →
Banafsheh Arshi
@_BArshi_ · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 62 followers View post →
Karen Céspedes
@KarenCespedes22 · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 752 followers View post →
Isabella R. Ghement
@IsabellaGhement · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 5,799 followers View post →
Mark Annuncio
@MarkAnnuncio · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 336 followers View post →
Garrett Bullock PT, DPT, DPhil
@DrGSBullock · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 871 followers View post →
Pavlos Msaouel
@PavlosMsaouel · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 3,537 followers View post →
Butler Scientifics
@butlersci · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 1,014 followers View post →
Steven Wambua, PhD
@StevenWambua · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 531 followers View post →
José A. Calvache
@jacalvache · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 804 followers View post →
Pedro Torres-Saavedra, PhD
@PTorresSaavedra · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 131 followers View post →
Wagner Leite .
@wagnerszleite · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 470 followers View post →
Joao R N Vissoci
@Jrvissoci · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 487 followers View post →
Scott Tagliaferri
@ScottTags · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 477 followers View post →
Juanma García
@Juanma_Garcia_T · X

RT @Richard_D_Riley: February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 22 shares · 1,123 followers View post →
Richard Riley (R²)
@Richard_D_Riley · X

February reminder to check the calibration performance of clinical prediction models

Feb 1, 2023 · 79 likes · 22 shares · 12,871 followers View post →
Maarten van Smeden
@MaartenvSmeden · X

@ayazhyder https://t.co/fCa4J6LETH

Jan 9, 2023 · 1 like · 39,855 followers View post →
Andrew Vickers
@VickersBiostats · X

@FinDataScience @rasbt @marktenenholtz note 100% sure i understand your question but have a look at https://t.co/AYCKKYXVAo

Nov 15, 2022 · 5,607 followers View post →
Ravi Mangal Patel
@ravimpatelmd · X

But this doesn't assess calibration ... Some have called calibration the "Achilles heel of predictive analytics": https://t.co/jRwf1wFpZZ It compares predicted probabilities with observed proportions ....

Oct 18, 2022 · 4,583 followers View post →
Maarten van Smeden
@MaartenvSmeden · X

@DonzelliSabrina @trumanfrancis Indeed, and we also give some arguments against it (and references) here: https://t.co/fCa4J6McJf

Oct 12, 2022 · 3 likes · 39,855 followers View post →
Percy
@psotob91 · X

El otro reto fue evaluar no solo la discriminación (p.ej., areas bajo la curva ROC o similares), sino también calibración. Si quieren entender de qué trata esto y por qué es tan pero tan importante que TRIPOD y Cochrane lo solicitan, lean esto: https://t.co/7rS56dDShS (12/n)

Sep 29, 2022 · 517 followers View post →
Jordan James Etem
@JordanJamesEtem · X

Van Calster, G., et al.: Calibration: the achilles heel of predictive analytics. BMC Med. 17(1), 1–7 (2019) @jwilluva @OracleComms

Aug 30, 2022 · 12,647 followers View post →
Gary Collins
@GSCollins · X

@AGonzalezUro @trumanfrancis two good papers on calibration https://t.co/1rdVRdtouF and https://t.co/8WNopmb8kh

Aug 30, 2022 · 1 like · 10,360 followers View post →
Gary Collins
@GSCollins · X

@trumanfrancis

Aug 28, 2022 · 7 likes · 10,360 followers View post →
Demetri (is over at the other place too)
@PhDemetri · X

@colin_fraser https://t.co/1Kn507OyBl See here. Model 2 is calibrated in the large, but Model 1 is calibrated in the small. You want in the small generally speaking.

Jun 22, 2022 · 19,544 followers View post →
The STRATOS initiative
@stratosinit · X

RT @nicolem_white: A favourite paper I share with students is "Calibration: the Achilles heel of predictive analytics" from @stratosinit…

May 20, 2022 · 6 shares · 1,375 followers View post →
LetsTalkSciComm @letstalkscicomm.bsky.social
@LetsTalkSciComm · X

RT @nicolem_white: A favourite paper I share with students is "Calibration: the Achilles heel of predictive analytics" from @stratosinit…

May 20, 2022 · 6 shares · 1,796 followers View post →
Nicole White
@nicolem_white · X

https://t.co/K99pfyAkVb

May 19, 2022 · 4 likes · 878 followers View post →
Nicholas Tierney
@nj_tierney · X

RT @nicolem_white: A favourite paper I share with students is "Calibration: the Achilles heel of predictive analytics" from @stratosinit…

May 19, 2022 · 6 shares · 3,757 followers View post →
ScienceStop
@B0tSci · X

RT @nicolem_white: A favourite paper I share with students is "Calibration: the Achilles heel of predictive analytics" from @stratosinit…

May 19, 2022 · 6 shares · 4,074 followers View post →
Nicole White
@nicolem_white · X

A favourite paper I share with students is "Calibration: the Achilles heel of predictive analytics" from @stratosinit Calibration is essential when evaluating the quality of prediction models for use in clinical decision making, but is often not reported. #MyScienceMay https://t.co/wucp3DCBaA

May 19, 2022 · 19 likes · 6 shares · 878 followers View post →
Victorio C. Carosella, MD. - ArgenSCORE
@Argenscore · X

9/ https://t.co/iWiCEtEYv9

May 11, 2022 · 1 like · 2,429 followers View post →
statsbot
@statstwitbot · X

RT @GSCollins: From a published paper Authors: "lines were almost overlapping with or close to the diagonal line". Hmm, not quite!

May 3, 2022 · 1 share · 816 followers View post →
Gary Collins
@GSCollins · X

From a published paper Authors: "lines were almost overlapping with or close to the diagonal line". Hmm, not quite!

May 3, 2022 · 17 likes · 1 share · 10,360 followers View post →
Global Health Observ
@GlobalPHObserv · X

RT @pash22: Calibration: the Achilles heel of predictive analytics https://t.co/dlqQy6CX3U via @BenVanCalster & @GSCollins & @MaartenvSmede…

Apr 9, 2022 · 1 share · 6,545 followers View post →
Ash Paul
@pash22 · X

Calibration: the Achilles heel of predictive analytics https://t.co/dlqQy6CX3U via @BenVanCalster & @GSCollins & @MaartenvSmeden et al

Apr 9, 2022 · 1 share · 32,293 followers View post →
Pilar Garcia-Iglesias
@Amleto3333 · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 198 followers View post →
rafael mello galliez
@RafaelGalliez · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 147 followers View post →
Thomas Shanahan (Dr.) MRCEM
@clifford0584 · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 6,060 followers View post →
Ewout Steyerberg
@ESteyerberg · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 4,294 followers View post →
TRIPODStatement
@TRIPODStatement · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 1,440 followers View post →
Pediatric Clinical Decision Support Collaborative
@pediatric_cds · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 78 followers View post →
Ioannis Doundoulakis
@IDoundoulakis · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 508 followers View post →
Alexis Karamanos @akara89@mas.to
@alexkara1989 · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 727 followers View post →
WillPer
@wilpertwitt · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 107 followers View post →
George E. Dafoulas MD, MBA in HSM
@GeorgeEDafoulas · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 644 followers View post →
Professor Shakila Thangaratinam
@thangaratinam · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 1,809 followers View post →
Abraham E. Gracia R.
@Abraham_RMI · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 4,680 followers View post →
Giuseppe Biondi-Zoccai
@gbiondizoccai · X

RT @GSCollins: "it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the…

Apr 9, 2022 · 15 shares · 12,164 followers View post →
Gary Collins
@GSCollins · X

"it is imperative to evaluate calibration using appropriate measures & visualizations – this helps us to understand how the algorithm performs in a particular setting, where predictions may go wrong" A must read to understand the importance of calibration https://t.co/C15LPsnxE5 https://t.co/N6jA1gkukm

Apr 9, 2022 · 78 likes · 15 shares · 10,360 followers View post →
Gary Collins
@GSCollins · X

@GaelVaroquaux Yeah, Brier score is fine, tend to prefer to report (in addition to the calibration plot), the calibration slope & calibration-in-the-large This is what we say in TRIPOD (page W34)

Apr 7, 2022 · 2 likes · 10,360 followers View post →
Professor Shakila Thangaratinam
@thangaratinam · X

RT @irfnali1: @GaelVaroquaux @Richard_D_Riley I’m also curious about the answer. This one (by @BenVanCalster, @MaartenvSmeden, @laure_wynan…

Mar 12, 2022 · 2 shares · 1,809 followers View post →
Ewout Steyerberg
@ESteyerberg · X

RT @irfnali1: @GaelVaroquaux @Richard_D_Riley I’m also curious about the answer. This one (by @BenVanCalster, @MaartenvSmeden, @laure_wynan…

Mar 12, 2022 · 2 shares · 4,294 followers View post →
Gary Collins
@GSCollins · X

@jschoeley Calibration is a key performance measure (& we recommend it in the TRIPOD Statement (https://t.co/dNuw3dWwmi, https://t.co/UUg4QwfFrt). You can do it by grouping & getting ave predicted risks (or fitting a loess smoother -> https://t.co/qk6paWdCmz) - not the case in this study

Mar 2, 2022 · 3 likes · 10,360 followers View post →
Guillaume Bonnet
@guilbon · X

RT @MaartenvSmeden: Wondering what all that talk about calibration of prediction models is about? Then this paper led by @BenVanCalster mig…

Feb 15, 2022 · 35 shares · 3,493 followers View post →
Ulrike Grittner
@Ulrike_gr · X

RT @MaartenvSmeden: Wondering what all that talk about calibration of prediction models is about? Then this paper led by @BenVanCalster mig…

Feb 10, 2022 · 35 shares · 107 followers View post →
Mariska Leeflang
@leeflang_m · X

RT @MaartenvSmeden: Wondering what all that talk about calibration of prediction models is about? Then this paper led by @BenVanCalster mig…

Feb 10, 2022 · 35 shares · 1,173 followers View post →
Ewout Steyerberg
@ESteyerberg · X

RT @MaartenvSmeden: Wondering what all that talk about calibration of prediction models is about? Then this paper led by @BenVanCalster mig…

Feb 10, 2022 · 35 shares · 4,294 followers View post →
Vadim Zipunnikov
@vadimZip · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 1,216 followers View post →
Edouard L. Fu
@FuEdouard · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 1,049 followers View post →
Juan Luis Cruz, PhD
@jotaelecruz · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 1,515 followers View post →
CJ
@cj_footeMD · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 1,273 followers View post →
Dalne Eric
@EricDalne · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 382 followers View post →
Zakia Salod, PhD
@Zaks1000 · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 340 followers View post →
Del
@KonstantFlux · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 121 followers View post →
statsbot
@statstwitbot · X

RT @GSCollins: A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to…

Jan 25, 2022 · 15 shares · 816 followers View post →
Gary Collins
@GSCollins · X

A seemingly radical idea given the published literature, but if you're developing a prediction model, my advice would be to try & show how well it predicts -> Calibration is your friend https://t.co/JArFHziq69 Focussing only on rank based measures is not enough #statstwitter https://t.co/6sEkAhCYdJ

Jan 25, 2022 · 58 likes · 15 shares · 10,360 followers View post →
Juanma García
@Juanma_Garcia_T · X

RT @ESteyerberg: And here we emphasize calibration for Predictive Analytics as a field: https://t.co/C4WtqetISU @BenVanCalster While clini…

Jan 22, 2022 · 4 shares · 1,123 followers View post →
Giuseppe Biondi-Zoccai
@gbiondizoccai · X

RT @ESteyerberg: And here we emphasize calibration for Predictive Analytics as a field: https://t.co/C4WtqetISU @BenVanCalster While clini…

Jan 21, 2022 · 4 shares · 12,164 followers View post →
Garrett Bullock PT, DPT, DPhil
@DrGSBullock · X

RT @ESteyerberg: And here we emphasize calibration for Predictive Analytics as a field: https://t.co/C4WtqetISU @BenVanCalster While clini…

Jan 21, 2022 · 4 shares · 871 followers View post →
Andrew Vickers
@VickersBiostats · X

RT @ESteyerberg: And here we emphasize calibration for Predictive Analytics as a field: https://t.co/C4WtqetISU @BenVanCalster While clini…

Jan 21, 2022 · 4 shares · 5,607 followers View post →
Ewout Steyerberg
@ESteyerberg · X

And here we emphasize calibration for Predictive Analytics as a field: https://t.co/C4WtqetISU @BenVanCalster While clinical usefulness is an overarching metric https://t.co/4ewcfoPU2t @VickersBiostats https://t.co/kgtYNzQcqH

Jan 21, 2022 · 27 likes · 4 shares · 4,294 followers View post →
Ben Van Calster
@BenVanCalster · X

@BabakWork @GSCollins @MaartenvSmeden @ESteyerberg @TRIPODStatement https://t.co/2zFkQyMtG5

Jan 20, 2022 · 2 likes · 2,111 followers View post →
alwynkotzee
@alwynkotzee · X

@BJSurgery @AmyLightnerMD @bplwijn @des_winter @ksoreide @MalinASund @evanscolorectal @nfmkok @robhinchliffe1 @young_bjs Hang on, if purpose == individual risk prediction, calibration far more important than discrimination (which isn't great in this case). Why only focus on AUROC? https://t.co/6407Gr2Txo

Dec 5, 2021 · 407 followers View post →
iordananutarusu@gmail.com
@iordananutarus1 · X

RT @pash22: @MaartenvSmeden @Argenscore @djc795 @1987Andrewk @GSCollins @rahatheart1 @mmamas1973 @VictorDayan1 @pomyers @dr_benoy_n_shah @G…

Nov 26, 2021 · 1 share · 156 followers View post →
Ash Paul
@pash22 · X

@MaartenvSmeden @Argenscore @djc795 @1987Andrewk @GSCollins @rahatheart1 @mmamas1973 @VictorDayan1 @pomyers @dr_benoy_n_shah @GilbertTangMD @DrMoritzWvB @PPibarot @DVervoort94 @gonzaeperez @drjohnm @ovidiogarciav @UniofOxford @ramspek @mpiccininni3 @escardio @UtrechtUni Maarten & @BenVanCalster et al wrote a great 2019 article "Calibration: the Achilles heel of predictive analytics" https://t.co/dlqQy6CX3U

Nov 26, 2021 · 7 likes · 1 share · 32,293 followers View post →
James Glasbey
@DrJamesGlasbey · X

RT @Richard_D_Riley: @AJAveritt Calibration, calibration & calibration (in the context of ML for prediction) https://t.co/8p9REgh5Ds

Nov 22, 2021 · 1 share · 5,060 followers View post →
Richard Riley (R²)
@Richard_D_Riley · X

@AJAveritt Calibration, calibration & calibration (in the context of ML for prediction) https://t.co/8p9REgh5Ds

Nov 22, 2021 · 16 likes · 1 share · 12,871 followers View post →
HealthBuzz.be
@HealthBuzzBE · X

Sep 22, 2021 · 1,542 followers View post →
Arshiya Sangchooli (He/They)
@Arshiya_San · X

RT @ESteyerberg: @suchisaria @narges_razavian @IAmSamFin @NEJM @_asubbaswamy @kdpsinghlab @zittrain @zakkohane Great table! Re updating of…

Jul 15, 2021 · 2 shares · 450 followers View post →
Karandeep Singh
@kdpsinghlab · X

RT @ESteyerberg: @suchisaria @narges_razavian @IAmSamFin @NEJM @_asubbaswamy @kdpsinghlab @zittrain @zakkohane Great table! Re updating of…

Jul 15, 2021 · 2 shares · 9,803 followers View post →
b_hring_m
@b_hring_m · X

RT @ESteyerberg: @MaartenvSmeden @Richard_D_Riley @GSCollins @CarlMoons @Kym_Snell @BenVanCalster @TPA_Debray @VMTdeJong @joie_ensor @f2har…

Jun 29, 2021 · 2 shares · 123 followers View post →
Roger Hilfiker
@RogerHilfiker · X

RT @ESteyerberg: @kdpsinghlab Uninformative models cannot be clinically useful (AUC=.5); neither can a 'good' model (say AUC>.7) but a thre…

Jun 26, 2021 · 4 shares · 1,620 followers View post →
Mayela HJ
@MAYELAHJ · X

RT @ESteyerberg: @kdpsinghlab Uninformative models cannot be clinically useful (AUC=.5); neither can a 'good' model (say AUC>.7) but a thre…

Jun 26, 2021 · 4 shares · 212 followers View post →
Professor Shakila Thangaratinam
@thangaratinam · X

RT @ESteyerberg: @kdpsinghlab Uninformative models cannot be clinically useful (AUC=.5); neither can a 'good' model (say AUC>.7) but a thre…

Jun 26, 2021 · 4 shares · 1,809 followers View post →
Karandeep Singh
@kdpsinghlab · X

RT @ESteyerberg: @kdpsinghlab Uninformative models cannot be clinically useful (AUC=.5); neither can a 'good' model (say AUC>.7) but a thre…

Jun 26, 2021 · 4 shares · 9,803 followers View post →
Ewout Steyerberg
@ESteyerberg · X

@kdpsinghlab Uninformative models cannot be clinically useful (AUC=.5); neither can a 'good' model (say AUC>.7) but a threshold at the extreme of the risk distribution be clinically useful; and a miscalibrated model can be harmful @BenVanCalster https://t.co/C4WtqetISU https://t.co/sUJAZ8A2kT

Jun 26, 2021 · 12 likes · 4 shares · 4,294 followers View post →
Haitao Pan
@davidhaitaopan · X

RT @triadsou: Calibration: the Achilles heel of predictive analytics. Ben Van Calster, David J. McLernon, Maarten van Smeden, Laure Wynants…

Jun 13, 2021 · 2 shares · 254 followers View post →
お薬屋
@CareerPharm · X

RT @triadsou: Calibration: the Achilles heel of predictive analytics. Ben Van Calster, David J. McLernon, Maarten van Smeden, Laure Wynants…

Jun 13, 2021 · 2 shares · 654 followers View post →
Triad sou.
@triadsou · X

Calibration: the Achilles heel of predictive analytics. Ben Van Calster, David J. McLernon, Maarten van Smeden, Laure Wynants & Ewout W. Steyerberg, On behalf of Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative. https://t.co/HhJyUIPiaj

Jun 13, 2021 · 3 likes · 2 shares · 2,216 followers View post →
rafael mello galliez
@RafaelGalliez · X

RT @ESteyerberg: @MaartenvSmeden @Richard_D_Riley @GSCollins @CarlMoons @Kym_Snell @BenVanCalster @TPA_Debray @VMTdeJong @joie_ensor @f2har…

May 28, 2021 · 2 shares · 147 followers View post →
Ewout Steyerberg
@ESteyerberg · X

@MaartenvSmeden @Richard_D_Riley @GSCollins @CarlMoons @Kym_Snell @BenVanCalster @TPA_Debray @VMTdeJong @joie_ensor @f2harrell and the important issue is that 1 validation does not suffice; rather multiple validation studies should be done to give insight in heterogeneity in performance, specifically miscalibration https://t.co/IR6wFoTfpk https://t.co/C4WtqetISU https://t.co/iP9ulRLLHx

May 28, 2021 · 7 likes · 2 shares · 4,294 followers View post →
Ash Paul
@pash22 · X

Calibration: the Achilles heel of predictive analytics https://t.co/dlqQy6CX3U via @MaartenvSmeden et al

Feb 10, 2021 · 32,293 followers View post →
Episurgeon
@episurgeon · X

RT @ESteyerberg: @kdpsinghlab @aminadibi Thanks for the fascinating story. I'm intrigued by the calibration of these models. If used for b…

Jan 6, 2021 · 2 shares · 258 followers View post →
Karandeep Singh
@kdpsinghlab · X

RT @ESteyerberg: @kdpsinghlab @aminadibi Thanks for the fascinating story. I'm intrigued by the calibration of these models. If used for b…

Jan 3, 2021 · 2 shares · 9,803 followers View post →
Ewout Steyerberg
@ESteyerberg · X

@kdpsinghlab @aminadibi Thanks for the fascinating story. I'm intrigued by the calibration of these models. If used for benchmarking, quantifying systematic miscalibration by ICU is the objective. If used for individual decision-making, miscalibration is the Achilles heel? https://t.co/C4WtqetISU

Jan 3, 2021 · 5 likes · 2 shares · 4,294 followers View post →
Ben Van Calster
@BenVanCalster · X

@DrLukeOR @VickersBiostats Difficult indeed. We call calibration the Achilles heel of clinical predictive analytics. However you quantify predictions, there will be heterogeneity. I prefer to open it up maximally: risk estimation and quantification of heterogeneity between centers. https://t.co/35PxX6faF2

Aug 18, 2020 · 2,111 followers View post →
aure
@maureviv · X

Calibration: the Achilles heel of predictive analytics https://t.co/zBGvDnGEy3

Aug 11, 2020 · 1,817 followers View post →
WillPer
@wilpertwitt · X

RT @ESteyerberg: @dt_hughes @Richard_D_Riley Congrats! Looks as a high quality study; but how can calibration-in-the-large be away from Nul…

Jun 10, 2020 · 2 shares · 107 followers View post →
Brainheartnexus
@brainheartnexus · X

RT @ESteyerberg: @dt_hughes @Richard_D_Riley Congrats! Looks as a high quality study; but how can calibration-in-the-large be away from Nul…

Jun 10, 2020 · 2 shares · 976 followers View post →
Ewout Steyerberg
@ESteyerberg · X

@dt_hughes @Richard_D_Riley Congrats! Looks as a high quality study; but how can calibration-in-the-large be away from Null at internal validation? Forget this performance aspect for internal, and make it the primary focus for external validation https://t.co/WTQpo5RIz0 https://t.co/2t3QIjDwIe

Jun 10, 2020 · 5 likes · 2 shares · 4,294 followers View post →
Zachary Cohen
@DepressionLab · X

@tkaiser_science @krstoffr @f2harrell https://t.co/FcpOrCx528 Calibration: the Achilles heel of predictive analytics https://t.co/aUL4dd5T4S

Mar 20, 2020 · 2 likes · 3,406 followers View post →
Kieran Campbell
@kieranrcampbell · X

RT @SamanthaLWilson: Great paper on predictive model tuning! I see many trying to use every variable obtained, over fitting the model. This…

Jan 8, 2020 · 1 share · 2,712 followers View post →
Samantha L Wilson Bsky:SamanthaLWilson
@SamanthaLWilson · X

Great paper on predictive model tuning! I see many trying to use every variable obtained, over fitting the model. This paper gives good suggestions to optimally calibrate predictive models. Calibration: the Achilles heel of predictive analytics. https://t.co/h6ijvgohGz

Jan 8, 2020 · 4 likes · 1 share · 3,068 followers View post →
Culture of Health Eq
@CofHEQ · X

RT @GlobalPHObserv: Calibration: the Achilles heel of predictive analytics https://t.co/RY1XOSU03q

Jan 4, 2020 · 1 share · 1,095 followers View post →
Global Health Observ
@GlobalPHObserv · X

Calibration: the Achilles heel of predictive analytics https://t.co/RY1XOSU03q

Jan 4, 2020 · 1 like · 1 share · 6,545 followers View post →
BMC Medicine
@BMCMedicine · X

RT @BMCMedicine: Just #published! A great new #opinion article from @BenVanCalster and colleagues: Calibration: the Achilles heel of pred…

Dec 18, 2019 · 2 shares · 9,194 followers View post →
Dalne Eric
@EricDalne · X

RT @BMCMedicine: Just #published! A great new #opinion article from @BenVanCalster and colleagues: Calibration: the Achilles heel of pred…

Dec 17, 2019 · 2 shares · 382 followers View post →
BMC Medicine
@BMCMedicine · X

Just #published! A great new #opinion article from @BenVanCalster and colleagues: Calibration: the Achilles heel of predictive analytics https://t.co/mwFuUIInSX https://t.co/WEx2gl6Kjr

Dec 17, 2019 · 12 likes · 2 shares · 9,194 followers View post →