Proc Natl Acad Sci U S A December 2015 PMID 26553988

Using prediction markets to estimate the reproducibility of scientific research.

Anna Dreber, Thomas Pfeiffer, Johan Almenberg, Siri Isaksson, Brad Wilson, Yiling Chen, Brian A Nosek, Magnus Johannesson

Download social card

Mentions

35
Peak: 3 in Sep 2020
across 7 sources since Nov 2015

Extended reach

98,852
potential audience

Total likes

214

Video views

2,825

Mentions per month

Nov 2015 – Mar 2025

Where it's discussed

  • X / Twitter17 posts
  • Facebook8 posts
  • Blogs4 articles
  • YouTube2 videos
  • News2 articles
  • Bluesky1 posts
  • Reddit1 posts

AI synopsis

What people are saying

The discussions show enthusiastic support for using prediction markets as efficient alternatives for assessing reproducibility, with posts mentioning their effectiveness, related experiments, and potential for faster discovery, where words like 'effective,' 'fascinating,' and 'useful' underline positive perceptions and engagement with the research's implications.

AgreementModerate agreement

Most discussions express support and recognition of the method's usefulness, indicating general agreement on the value of prediction markets in research reproducibility.

InterestHigh interest

The topic attracts high interest, with mentions of prediction markets as innovative and promising tools in scientific validation.

EngagementHigh engagement

Many posts delve into details, citing examples, related studies, and potential applications, demonstrating active engagement.

ImpactModerate impact

Some posts highlight the potential for prediction markets to improve research reliability and speed, suggesting moderate but meaningful impact.

Abstract in brief

  • Prediction markets were used to estimate the reproducibility of 44 psychology studies, and they successfully predicted replication outcomes better than individual forecasts.
  • These markets also provide probabilities about the truth of hypotheses over different testing stages, offering insights into scientific discovery over time.
  • The findings suggest that many psychological hypotheses have low initial likelihood of being true, and replication is essential to confirm statistically significant results, making prediction markets a useful and cost-effective tool for guiding research efforts.

Top posts

Most liked on X and Bluesky
finder outer
@dr_marshwiggle · X

@literalbanana @Meaningness prediction markets are actually pretty good at predicting what replicates and what doesn't https://t.co/dDtHAAvIqG

Mar 27, 2024 · 10 likes · 62 followers View post →
Sten Linnarsson
@slinnarsson · X

@arjunrajlab I recall this was tried in one of those systematic replication studies, and worked surprisingly well. I.e. people knew what was bs. Maybe this one: https://t.co/uaPMxOMyro

May 28, 2022 · 5 likes · 1 share · 8,904 followers View post →
Rotem Botvinik-Nezer
@rotembot · X

@sirileknes @TimoRoettger I think it could be useful! Though if you preregister your analysis plan and predictions with enough details, it should prevent your bias from affecting the results. Prediction markets are used to test if peers can predict studies' results (see e.g. https://t.co/SIopLDrKjL)

Sep 26, 2022 · 2 likes · 761 followers View post →
Vincent Weisser
@vincentweisser · X

@0xNN2 yes love it! @VitalikButerin touched on it here https://t.co/kzotfrmBPk https://t.co/qnovQBJUIG https://t.co/zQw3Ss9kCa

Nov 10, 2024 · 1 like · 26,298 followers View post →
高大伟 David Cowhig
@gaodawei · X

Using prediction markets to estimate the reproducibility of scientific research https://t.co/Ezx5VXUvCe

Sep 11, 2020 · 1 like · 1 share · 10,945 followers View post →
Drew Bailey
@drewhalbailey · X

@JProtzko A paper showing forecasters can in some cases spot stuff that won’t replicate: https://t.co/6DDeKqywn8 And a reflection by a forecaster who did well in a related study in which he describes his thinking along the lines you mention above: https://t.co/MjMT3RnXzi

Jun 4, 2024 · 1 like · 1,407 followers View post →

All mentions

Sathya
@9Sathya9 · X

Ref: Using prediction markets to estimate the reproducibility of scientific research https://t.co/ZfAlKsmXXs

Mar 24, 2025 · 1 like · 84 followers View post →
Xin Wei
@Xin9Xin · X

Dreber et al. (2015) reported an alternative method to examine the replicability of SBS papers: the expert-sourced prediction markets. It is effective and less time-consuming. However, it is hard to scale. https://t.co/zkHoKZgFQq

Jan 3, 2025 · 1 like · 78 followers View post →
experience
@trustlessexp · X

@tarunchitra https://t.co/JxyswTWfHW

Nov 17, 2024 · 755 followers View post →
Vincent Weisser
@vincentweisser · X

@0xNN2 yes love it! @VitalikButerin touched on it here https://t.co/kzotfrmBPk https://t.co/qnovQBJUIG https://t.co/zQw3Ss9kCa

Nov 10, 2024 · 1 like · 26,298 followers View post →
Chris Cherniakov
@candyflipline · X

@AlanaDLevin Read a cool paper on using markets to predict the reproducibility of an academic paper: https://t.co/J2VqEzR8e7 There's actually a lot of work on predictive markets out there -- some of it is really fascinating

Jul 29, 2024 · 2,255 followers View post →
Drew Bailey
@drewhalbailey · X

@JProtzko A paper showing forecasters can in some cases spot stuff that won’t replicate: https://t.co/6DDeKqywn8 And a reflection by a forecaster who did well in a related study in which he describes his thinking along the lines you mention above: https://t.co/MjMT3RnXzi

Jun 4, 2024 · 1 like · 1,407 followers View post →
finder outer
@dr_marshwiggle · X

@literalbanana @Meaningness prediction markets are actually pretty good at predicting what replicates and what doesn't https://t.co/dDtHAAvIqG

Mar 27, 2024 · 10 likes · 62 followers View post →
Jakub K. Nowak
@kubanetics · X

@Sam_J_Street You mean more recent than the 2015 Dreber et al. experiments? https://t.co/kk2oq9Pa1o

Nov 13, 2023 · 1 like · 147 followers View post →
Florian Cova
@CovaFlorian · X

@Professologue @wagneregger @Utopia_Rabbit @melex_sior @NathLarigaldie @Gobe_l_un @GeekNfit @HygieneMentale @Dumbsociologist @Scintigraphiste @MrSam144 @vieux_chene @AlexSamTG @lasagne42 @LanguesdeCha @YTCreateurs @KenzoNera @JaisTroian Celui-ci est plus pertinent : https://t.co/yQz3wfE7x0

May 3, 2023 · 5,798 followers View post →
Florian Cova
@CovaFlorian · X

@Professologue @wagneregger @Utopia_Rabbit @melex_sior @NathLarigaldie @Gobe_l_un @GeekNfit @HygieneMentale @Dumbsociologist @Scintigraphiste @MrSam144 @vieux_chene @AlexSamTG @lasagne42 @LanguesdeCha @YTCreateurs @KenzoNera @JaisTroian Et ici : https://t.co/yQz3wfE7x0

May 3, 2023 · 5,798 followers View post →
Rotem Botvinik-Nezer
@rotembot · X

@sirileknes @TimoRoettger I think it could be useful! Though if you preregister your analysis plan and predictions with enough details, it should prevent your bias from affecting the results. Prediction markets are used to test if peers can predict studies' results (see e.g. https://t.co/SIopLDrKjL)

Sep 26, 2022 · 2 likes · 761 followers View post →
Better Masks 4 Melbourne
@maskupmelb · X

Even better is if we had a betting market on key COVID hypotheses, we could discover the truth faster and observe the market’s implied probability of each hypotheses being true. https://t.co/IAc0PqPsFP

Jul 17, 2022 · 1 like · 968 followers View post →
Arjun Raj
@arjunrajlab · X

RT @slinnarsson: @arjunrajlab I recall this was tried in one of those systematic replication studies, and worked surprisingly well. I.e. pe…

May 28, 2022 · 1 share · 29,778 followers View post →
Sten Linnarsson
@slinnarsson · X

@arjunrajlab I recall this was tried in one of those systematic replication studies, and worked surprisingly well. I.e. people knew what was bs. Maybe this one: https://t.co/uaPMxOMyro

May 28, 2022 · 5 likes · 1 share · 8,904 followers View post →
ODHIAMBO
@RevoltKenya · X

Using prediction markets to estimate the reproducibility of scientific research https://t.co/bsJSACHQYB

Sep 14, 2020 · 733 followers View post →
Jim Jones (鍾傑穆)
@han4yu3 · X

RT @gaodawei: Using prediction markets to estimate the reproducibility of scientific research https://t.co/Ezx5VXUvCe

Sep 11, 2020 · 1 share · 1,099 followers View post →
高大伟 David Cowhig
@gaodawei · X

Using prediction markets to estimate the reproducibility of scientific research https://t.co/Ezx5VXUvCe

Sep 11, 2020 · 1 like · 1 share · 10,945 followers View post →
Ruben C. Arslan
@ruben.the100.ci · Bluesky

Using prediction markets to estimate the reproducibility of scientific research. @StatModeling @bahniks @R__INDEX http://andrewgelman.com/2015/11/09/using-prediction-markets-to-estimate-the-reproducibility-of-scientific-research/

Nov 10, 2015 View post →
Evaluating the Replicability of Social Science Experiments in Nature ...
Blogs

10.1073/pnas.1516179112. 30. Benjamin, D, et al. Redefine statistical significance. Nat. Hum. Behav. 1 (2017). DOI: 10.1038/s41562-017-0189-z. 31. Jeffreys, H ...

Dec 26, 2025 View →
Using prediction markets to estimate the reproducibility of scientific ...
Blogs

1516179112/-/DCSupplemental. www.pnas.org/cgi/doi/10.1073/pnas.1516179112. PNAS Early Edition | 1 of 5.

Dec 26, 2025 View →
Failure to Replicate in Preregistered Studies and the Effect of ...
Facebook

https://www.pnas.org/doi/10.1073/pnas.2402315121 New study. Does ... 1516179112.abstract) and summary news stories in Atlantic (http://www.

Jul 25, 2024 View →
Como podemos decidir quais pesquisas são confiáveis? : r ... - Reddit
Reddit

https://www.pnas.org/doi/10.1073/pnas.1516179112. Eu mesmo fiz um desses estudos. Então, o sistema pelo menos meio que funciona. O que ...

Sep 1, 2023 View →
Pitfalls of amateur regression: The Dutch New Herring controversies ...
Facebook

... pnas.org/doi/10.1073/pnas. 1313495110 http://www.unm.edu/~mbeb ... 1516179112.abstract) and summary news stories in Atlantic (http://www.

Apr 15, 2023 View →
Replication report for UT Austin psych department
Facebook

... 1516179112.abstract) and summary news stories in Atlantic (http://www. ... 10.1073/pnas. 2120668119 About the project: https://replications ...

Mar 13, 2022 View →
Replication in psychological networks: new pre-print available
Facebook

org/content/early/2015/11/04/1516179112. abstract ... org/replication-2022pnas119-43/ The original study: https://doi.org/10.1073/pnas.

Jan 29, 2020 View →
Let's just say this is about as niche a niche study as you can get:
Facebook

https://www.pnas.org/doi/10.1073/pnas. 2402315121 New study. Does ... 1516179112.abstract) and summary news stories in Atlantic (http://www.

Apr 29, 2018 View →
Non-replication published in JAMA Psychiatry. Blog post describes ...
Facebook

org/replication-2022pnas119-43/ The original study: https://doi.org/10.1073/pnas. ... org/content/early/2015/11/04/1516179112. abstract ...

Dec 2, 2017 View →
Una solución estadística para la crisis de la replicación en ciencia ...
Blogs

DOI: 10.1073/pnas.1516179112. Available from: http://www.pnas.org/content/112/50/15343. Referências. American Statistical Association releases ...

Oct 25, 2017 View →
Uma solução estatística para a crise de replicação na ciência ...
Blogs

DOI: 10.1073/pnas.1516179112. Available from: http://www.pnas.org/content/112/50/15343. Referências. American Statistical Association releases ...

Oct 25, 2017 View →
The Journal of Experimental Social Psychology's special issue on ...
Facebook

org/replication-2022pnas119-43/ The original study: https://doi.org/10.1073/pnas. ... org/content/early/2015/11/04/1516179112. abstract ...

Jul 15, 2016 View →
Evaluating replicability of laboratory experiments in economics
News

Medline doi:10.1073/pnas.1516179112. 27. J. Wolfers, E. Zitzewitz, Interpreting Prediction Market Prices as Probabilities. (Working Paper No ...

Mar 3, 2016 View →
Here is the problem with politeness: it helps to keep you alive in an ...
Facebook

org/replication-2022pnas119-43/ The original study: https://doi.org/10.1073/pnas. ... org/content/early/2015/11/04/1516179112. abstract ...

Mar 2, 2016 View →
Prediction markets can help identify research results that are too ...
News

... pnas.org/cgi/doi/10.1073/pnas.1516179112. Journal information: Proceedings of the National Academy of Sciences. Provided by Harvard University.

Nov 9, 2015 View →