The discussions show enthusiastic support for using prediction markets as efficient alternatives for assessing reproducibility, with posts mentioning their effectiveness, related experiments, and potential for faster discovery, where words like 'effective,' 'fascinating,' and 'useful' underline positive perceptions and engagement with the research's implications.
AgreementModerate agreement
Most discussions express support and recognition of the method's usefulness, indicating general agreement on the value of prediction markets in research reproducibility.
InterestHigh interest
The topic attracts high interest, with mentions of prediction markets as innovative and promising tools in scientific validation.
EngagementHigh engagement
Many posts delve into details, citing examples, related studies, and potential applications, demonstrating active engagement.
ImpactModerate impact
Some posts highlight the potential for prediction markets to improve research reliability and speed, suggesting moderate but meaningful impact.
Abstract in brief
Prediction markets were used to estimate the reproducibility of 44 psychology studies, and they successfully predicted replication outcomes better than individual forecasts.
These markets also provide probabilities about the truth of hypotheses over different testing stages, offering insights into scientific discovery over time.
The findings suggest that many psychological hypotheses have low initial likelihood of being true, and replication is essential to confirm statistically significant results, making prediction markets a useful and cost-effective tool for guiding research efforts.
@arjunrajlab I recall this was tried in one of those systematic replication studies, and worked surprisingly well. I.e. people knew what was bs. Maybe this one: https://t.co/uaPMxOMyro
May 28, 2022 · 5 likes · 1 share · 8,904 followersView post →
RN
Rotem Botvinik-Nezer
@rotembot · X
@sirileknes @TimoRoettger I think it could be useful! Though if you preregister your analysis plan and predictions with enough details, it should prevent your bias from affecting the results.
Prediction markets are used to test if peers can predict studies' results (see e.g. https://t.co/SIopLDrKjL)
@0xNN2 yes love it! @VitalikButerin touched on it here https://t.co/kzotfrmBPk https://t.co/qnovQBJUIG https://t.co/zQw3Ss9kCa
Nov 10, 2024 · 1 like · 26,298 followersView post →
高C
高大伟 David Cowhig
@gaodawei · X
Using prediction markets to estimate the reproducibility of scientific research https://t.co/Ezx5VXUvCe
Sep 11, 2020 · 1 like · 1 share · 10,945 followersView post →
DB
Drew Bailey
@drewhalbailey · X
@JProtzko A paper showing forecasters can in some cases spot stuff that won’t replicate:
https://t.co/6DDeKqywn8
And a reflection by a forecaster who did well in a related study in which he describes his thinking along the lines you mention above:
https://t.co/MjMT3RnXzi
Impact is often overvalued in science, with citations and prominence overshadowing quality. This video explores why nonreplicable studies are cited more than replicable ones and discusses the influence of journal…
Open Science promotes transparency and integrity using practices like preprints, preregistrations, and open software, improving research robustness in neuroscience and psychology. This series explores how these methods…
Dreber et al. (2015) reported an alternative method to examine the replicability of SBS papers: the expert-sourced prediction markets. It is effective and less time-consuming. However, it is hard to scale.
https://t.co/zkHoKZgFQq
@0xNN2 yes love it! @VitalikButerin touched on it here https://t.co/kzotfrmBPk https://t.co/qnovQBJUIG https://t.co/zQw3Ss9kCa
Nov 10, 2024 · 1 like · 26,298 followersView post →
CC
Chris Cherniakov
@candyflipline · X
@AlanaDLevin Read a cool paper on using markets to predict the reproducibility of an academic paper: https://t.co/J2VqEzR8e7
There's actually a lot of work on predictive markets out there -- some of it is really fascinating
@JProtzko A paper showing forecasters can in some cases spot stuff that won’t replicate:
https://t.co/6DDeKqywn8
And a reflection by a forecaster who did well in a related study in which he describes his thinking along the lines you mention above:
https://t.co/MjMT3RnXzi
@sirileknes @TimoRoettger I think it could be useful! Though if you preregister your analysis plan and predictions with enough details, it should prevent your bias from affecting the results.
Prediction markets are used to test if peers can predict studies' results (see e.g. https://t.co/SIopLDrKjL)
Even better is if we had a betting market on key COVID hypotheses, we could discover the truth faster and observe the market’s implied probability of each hypotheses being true. https://t.co/IAc0PqPsFP
RT @slinnarsson: @arjunrajlab I recall this was tried in one of those systematic replication studies, and worked surprisingly well. I.e. pe…
May 28, 2022 · 1 share · 29,778 followersView post →
SL
Sten Linnarsson
@slinnarsson · X
@arjunrajlab I recall this was tried in one of those systematic replication studies, and worked surprisingly well. I.e. people knew what was bs. Maybe this one: https://t.co/uaPMxOMyro
May 28, 2022 · 5 likes · 1 share · 8,904 followersView post →
O
ODHIAMBO
@RevoltKenya · X
Using prediction markets to estimate the reproducibility of scientific research https://t.co/bsJSACHQYB
Using prediction markets to estimate the reproducibility of scientific research https://t.co/Ezx5VXUvCe
Sep 11, 2020 · 1 like · 1 share · 10,945 followersView post →
RA
Ruben C. Arslan
@ruben.the100.ci · Bluesky
Using prediction markets to estimate the reproducibility of scientific research. @StatModeling @bahniks @R__INDEX http://andrewgelman.com/2015/11/09/using-prediction-markets-to-estimate-the-reproducibility-of-scientific-research/