Discussions emphasize the value of the checklist, with comments like 'a very useful checklist' and references to improving preregistration, reflecting strong interest and recognition of its impact. The tone is generally analytical and motivated for methodological rigor, with examples discussing how the checklist helps prevent p-hacking and improves research transparency, showcasing deep engagement with the material and its potential influence on research standards.
AgreementModerate agreement
Most discussions recognize the importance of the checklist and its practical utility in improving research transparency.
InterestHigh interest
Topics about research practices, p-hacking, and preregistration generate high curiosity and engagement among scholars.
EngagementModerate engagement
Participants reference specific aspects of the checklist, compare tools, and discuss methodological improvements, indicating meaningful interaction.
ImpactHigh impact
The widespread sharing and emphasis on better research practices suggest the discussions could influence research norms and practices.
Abstract in brief
The article discusses how many decisions made during the planning, running, analyzing, and reporting of psychological studies are often arbitrary, which can lead to biased results and false positives if researchers try to find significant outcomes intentionally.
It provides an extensive list of 34 "degrees of freedom" that highlight the choices researchers have at various stages of research, emphasizing the importance of transparency and careful decision-making.
This list can be used as an educational tool, a checklist to evaluate study quality, and a way to identify potential biases caused by arbitrary choices in both registered and unregistered studies.]
@PersonalityPat @siminevazire @improvingpsych @AnuABee This checklist is useful to see if you covered everything when writing the pre-registration: https://t.co/Oy6r2d0FmW
MB: By pre-registration of a research study, you can define the path you will take through the "garden of forking paths". By specifying which choices the research will take, such degrees of freedom can be reduced.
See:
https://t.co/ZkdPBy2tGd
and
https://t.co/uFl0h1qBss
Aug 25, 2021 · 13 likes · 3 shares · 144,373 followersView post →
DL
Daniël Lakens
@lakens · X
@RickCarlsson @KhandisBlake That is all the analysis plan (at least in frequentist statistics). Sample size, stopping rule, conditions, measures, outlier removal, etc. Anything that impacts false positives (which pre-reg intends to control). A useful checklist is at https://t.co/Oy6r2d0FmW
Mar 31, 2019 · 6 likes · 2 shares · 30,675 followersView post →
DL
Daniël Lakens
@lakens · X
@Heinonmatti @RogertheGS @olivier_klein @raklein3 @CameronBrick @NHankonen I typically recommend the checklist in https://t.co/Oy6r2d0FmW. Regarding how to deal with deviations, there are no articles on this, and thus no teaching material. Most researchers don't know what to do with these.
@LauraMKoenig @annemscheel Andy Field (both R and SPSS available), for Open Science ican recommend the Open Science Committee of LMU Munich (https://t.co/DM0v2WwtqQ) of the one and only @nicebread303
Furthermore this paper: https://t.co/E0k7M79WCN
@jpsimmon @donandrewmoore @PNASNews It would be great if the preregistration used all the items from this checklist. https://t.co/yNiGp8XvB2 With 800 citations, I think is the best starting point to guide people to a complete preregistration.
This video discusses the influence of arbitrary decision-making in psychological studies and provides a checklist of 34 'degrees of freedom' to enhance transparency and reduce bias during planning, analysis, and…
This video discusses how choices made during psychological studies can lead to bias and false positives. It offers a checklist of 34 degrees of freedom to enhance transparency, reduce bias, and improve research quality…
This video discusses how arbitrary decisions in planning, running, analyzing, and reporting psychological studies can lead to bias and false positives. It presents a checklist of 34 degrees of freedom to promote…
This video discusses how arbitrary decisions during planning, running, analyzing, and reporting psychological studies can lead to biased results. It emphasizes transparency and provides a checklist of 34 degrees of…
This video explores how arbitrary decisions in planning, running, analyzing, and reporting psychological studies can lead to biases and false positives. It offers a checklist of 34 degrees of freedom to promote…
The article discusses how many decisions made during the planning, running, analyzing, and reporting of psychological studies are often arbitrary, which can lead to biased results and false positives if researchers try…
The article discusses how many decisions made during the planning, running, analyzing, and reporting of psychological studies are often arbitrary, which can lead to biased results and false positives if researchers try…
This video discusses how arbitrary decisions in planning, running, analyzing, and reporting psychological studies can lead to bias and false positives. It introduces a checklist of 34 "degrees of freedom" to promote…
This video discusses how many decisions in psychological research are arbitrary, potentially causing bias and false positives. It provides a checklist of 34 choices researchers make, emphasizing transparency and careful…
9/10 times I see an amazing methodological paper, its done by a dutch!
What is in the water or air of Netherlands? Cool discussion by Wicherts and colleagues on researcher degrees of freedom distinguishing when you can justify it and when you cant
https://t.co/FiKSQ6j9k6
@uri_sohn The brevity that is required is 1 way. The prompts above the questions is another. Why don't you include the checklist by Wicherts et al (https://t.co/yNiGp8WXLu) as a nudge to be more complete?
@jpsimmon @donandrewmoore @PNASNews It would be great if the preregistration used all the items from this checklist. https://t.co/yNiGp8XvB2 With 800 citations, I think is the best starting point to guide people to a complete preregistration.
@rmwillen This is doing a much better job of verbalizing the tests you will perform. I do not think it captures all the items on this checklist https://t.co/yNiGp8WXLu
but it looks relatively close. The best way to tell is to compare it with the final report - was everything in there?
@OxACClab So it is not worthless, but it is not worth much, and it is very obviously too far removed from best practice. Why didn't you follow this checklist? https://t.co/yNiGp8WXLu That would already have improved things!
Jul 24, 2024 · 1 like · 30,675 followersView post →
AG
Anders Granholm
@andersgranholm · X
#MethodMonday 44: Researcher degrees of freedom and variation in analyses/results
https://t.co/myyGAHP5ZK
https://t.co/PbUaL2AvW3
Two papers from psychology, but the central points apply to most fields 1/..
Dec 4, 2023 · 4 likes · 1,192 followersView post →
RZ
Rafał Zwolak
@RafaZwolak · X
@JaroslawKuzniar @pisorgpl Temat zasługuje na poważniejsze traktowanie, bo tu analiza statystyczna jest bardziej alarmująca, niż wnioski.
https://t.co/eAFdrWtK4O
RT @syeducation: @olivier_klein This is not exactly what you are asking for, but is certainly related: Wicherts et al.'s (2016) p-hacking c…
Feb 18, 2023 · 1 share · 15,568 followersView post →
MS
moin syed
@syeducation · X
@olivier_klein This is not exactly what you are asking for, but is certainly related: Wicherts et al.'s (2016) p-hacking checklist, https://t.co/kYADykJbBN
Feb 9, 2023 · 1 like · 1 share · 2,529 followersView post →
DH
Darías Holgado
@DariasHolgado · X
@MaxPrimbs https://t.co/SHQYsEtx2c and https://t.co/wG1NtjPn03
@aufdroeseler @AaronCharlton @uni_bamberg_of Not a template, but a very useful checklist https://t.co/Oy6r2d0FmW
May 16, 2022 · 4 likes · 1 share · 30,675 followersView post →
KP
Krzysztof Potempa
@BRAINCURES · X
RT @JelteWicherts: Degrees of freedom in planning, running, analyzing & reporting psychological studies. A checklist to avoid p-hacking
htt…
Feb 21, 2022 · 49 shares · 2,657 followersView post →
NS
NanoBubbles (nanobubbles.bsky.social)
@_Nano_Bubbles · X
RT @MicrobiomDigest: MB: By pre-registration of a research study, you can define the path you will take through the "garden of forking path…
Aug 25, 2021 · 3 shares · 656 followersView post →
N
Not
@MichaelPaulEdw1 · X
RT @MicrobiomDigest: MB: By pre-registration of a research study, you can define the path you will take through the "garden of forking path…
Aug 25, 2021 · 3 shares · 1,441 followersView post →
FC
F.Chasteland
@fchasteland · X
RT @MicrobiomDigest: MB: By pre-registration of a research study, you can define the path you will take through the "garden of forking path…
Aug 25, 2021 · 3 shares · 936 followersView post →
EB
Elisabeth Bik
@MicrobiomDigest · X
MB: By pre-registration of a research study, you can define the path you will take through the "garden of forking paths". By specifying which choices the research will take, such degrees of freedom can be reduced.
See:
https://t.co/ZkdPBy2tGd
and
https://t.co/uFl0h1qBss
Aug 25, 2021 · 13 likes · 3 shares · 144,373 followersView post →
DL
Daniël Lakens
@lakens · X
@rfarmer27 @eba_psych @OSFramework This is a good list of items to address in your preregistration: https://t.co/Oy6r2d0FmW It can be helpful to look at existing preregistrations (but you will want to do better than most of them, see also the related paper in this tweet https://t.co/hbTS9NgxSz
@yorl Here's another example: https://t.co/yPDrGulLmr "This entails D2: Measuring additional variables that can be selected later as covariates, independent variables, mediators, or moderators. "
@LauraMKoenig @annemscheel Andy Field (both R and SPSS available), for Open Science ican recommend the Open Science Committee of LMU Munich (https://t.co/DM0v2WwtqQ) of the one and only @nicebread303
Furthermore this paper: https://t.co/E0k7M79WCN
RT @JelteWicherts: Degrees of freedom in planning, running, analyzing & reporting psychological studies. A checklist to avoid p-hacking
htt…
Jun 19, 2020 · 49 shares · 1,202 followersView post →
DH
Dr. Helena Hartmann
@helenahhartmann · X
@iamscicomm I am sad to say I didn't invent it haha
May 14, 2020 · 1 like · 7,964 followersView post →
GP
Gordon Pennycook
@GordPennycook · X
Example of an approach that takes (a) https://t.co/qaP6Cld3Oa
Example of an approach that takes (b) https://t.co/xTMwP4Kb4v
Apr 11, 2020 · 1 like · 10,586 followersView post →
DL
Daniël Lakens
@lakens · X
@Heinonmatti @RogertheGS @olivier_klein @raklein3 @CameronBrick @NHankonen I typically recommend the checklist in https://t.co/Oy6r2d0FmW. Regarding how to deal with deviations, there are no articles on this, and thus no teaching material. Most researchers don't know what to do with these.
I'm amazed at how easy it is to p-hack by mistake. As @jpsimmon mentions, "good (and bad) people are p-hackers." As we learned in seminar, here are some ways to avoid it: https://t.co/f1RpGa2lfh #AcademicTwitter #Psyc4180 @julie_conder @chantelle0090
@PersonalityPat @siminevazire @improvingpsych @AnuABee This checklist is useful to see if you covered everything when writing the pre-registration: https://t.co/Oy6r2d0FmW
RT @lakens: @RickCarlsson @KhandisBlake That is all the analysis plan (at least in frequentist statistics). Sample size, stopping rule, con…
Mar 31, 2019 · 2 shares · 312 followersView post →
GM
Gabriel González Medina
@Gabriel00602805 · X
RT @lakens: @RickCarlsson @KhandisBlake That is all the analysis plan (at least in frequentist statistics). Sample size, stopping rule, con…
Mar 31, 2019 · 2 shares · 196 followersView post →
DL
Daniël Lakens
@lakens · X
@RickCarlsson @KhandisBlake That is all the analysis plan (at least in frequentist statistics). Sample size, stopping rule, conditions, measures, outlier removal, etc. Anything that impacts false positives (which pre-reg intends to control). A useful checklist is at https://t.co/Oy6r2d0FmW
Mar 31, 2019 · 6 likes · 2 shares · 30,675 followersView post →
BD
Berna Devezer
@devezer.bsky.social · Bluesky
when in doubt, check this out: 34 ways to p-hack your data http://journal.frontiersin.org/article/10.3389/fpsyg.2016.01832/full
ICYMI: Degrees of Freedom in Planning, Running, Analyzing, & Reporting Psych Studies: A Checklist to Avoid p-Hacking http://journal.frontiersin.org/article/10.3389/fpsyg.2016.01832/full
doi: 10.3389/fpsyg.2016.01832 doi: 10.1177/1745691612459060 Yamada, Y. (2018). How to crack pre-registration: toward transparent and open. Belcher, W. L. ...
2020 Book RadicalSolutionsAndOpenScience PDF | PDF | Massive ...
News
https://doi.org/10.3389/fpsyg. 2016.01832. Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., et al. (2016). The FAIR ...
Burning out, fading away, and the sophomore slump: Critics' versus ...
News
https://doi.org/10.3389/ · fpsyg.2016.01832. Zackery, S. M. (2014). Musical missteps: The severity of the sophomore slump in the music industry. (Unpublished ...