May 2024 PMID 38353690

ChatGPT for generating multiple-choice questions: Evidence on the use of artificial intelligence in automatic item generation for a rational pharmacotherapy exam.

Yavuz Selim Kıyak, Özlem Coşkun, Işıl İrem Budakoğlu, Canan Uluoğlu

Download social card

Mentions

30
Peak: 23 in Feb 2024
across 3 sources since Feb 2024

Extended reach

46,034
potential audience

Total likes

74

Video views

1,403

Mentions per month

Feb 2024 – Aug 2024

Where it's discussed

  • X / Twitter26 posts
  • YouTube3 videos
  • News1 articles

AI synopsis

What people are saying

Discussions consistently praise the study’s findings, such as mentions of ChatGPT producing questions with acceptable psychometric properties, and highlight how AI can revolutionize exam development, using words like 'transformative' and 'remarkable.' The tone and choice of examples reflect a high level of engagement and recognition of the research’s importance in education and assessment fields.

AgreementModerate agreement

Most discussions acknowledge the potential and usefulness of AI-generated questions, emphasizing the positive findings related to psychometric quality and efficiency.

InterestHigh interest

Posts demonstrate high interest by highlighting studies, results, and the transformative potential of AI in exam creation, indicating enthusiasm and engagement with the topic.

EngagementHigh engagement

Through references to specific research findings, discussions of quality, and the importance of prompts, the posts show a deep level of analysis and active involvement.

ImpactHigh impact

The discussions suggest this research may significantly influence educational assessments and testing practices by showcasing AI's capabilities and limitations.

Top posts

Most liked on X and Bluesky
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

Our study published today! "#ChatGPT for generating MCQs: Evidence on the use of #artificialintelligence in automatic item generation for a rational pharmacotherapy exam" It presents the first psychometric evidence for ChatGPT-generated case-based MCQs. https://t.co/NxhAuoeGA4

Feb 14, 2024 · 9 likes · 2 shares · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"... we found that ChatGPT is able to generate multiple choice questions with acceptable levels of item psychometrics. Our findings showed that the questions can effectively differentiate between students who perform at high and low levels." Ref: 10.1007/s00228-024-03649-x

Feb 19, 2024 · 8 likes · 2 shares · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

There is a group of authors who has published 100+ commentaries on many diverse fields only in the last 8 months. They didn't miss our #ChatGPT paper (free read-only access via this link: https://t.co/oPTuS5E9qR) as well. Here is our reply to them: https://t.co/ADcaQxa29e https://t.co/zrwarhYhM4

Mar 18, 2024 · 7 likes · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"The efficiency achieved through #AI would appear remarkable to test developers from a decade ago, who were engaged in the effortful task of manually writing multiple-choice questions." #MedEd Ref: 10.1007/s00228-024-03649-x

Feb 21, 2024 · 3 likes · 2 shares · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"Despite the presence of studies that focus on providing human-generated questions to ChatGPT, there has been a lack of research focused to asking ChatGPT-generated questions to humans." Ref: 10.1007/s00228-024-03649-x This is why we conducted this study, btw.

Feb 18, 2024 · 3 likes · 1 share · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"... [in generating MCQs using #ChatGPT] it is essential to recognize that the quality of multiple-choice questions is closely tied to prompt quality." Ref: 10.1007/s00228-024-03649-x

Feb 22, 2024 · 3 likes · 2 shares · 1,343 followers View post →

All mentions

Tıp Eğitimi
@tipegitim · X

“Bir diğer önemli dezavantaj ise yapay zeka modellerinin “kara kutu” olmasıdır. Bu, her seferinde benzersiz sorular üretmemizi sağlasa da, girdimizin çıktıyı tam olarak nasıl etkileyeceğini bilememekteyiz.” Kaynak: 10.1007/s00228-024-03649-x

Aug 26, 2024 · 662 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"Another significant drawback is the “black box” nature of AI models. Although this enables us to generate unique questions each time, we cannot know how our input will precisely affect the output." Ref: 10.1007/s00228-024-03649-x

May 13, 2024 · 1 like · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

There is a group of authors who has published 100+ commentaries on many diverse fields only in the last 8 months. They didn't miss our #ChatGPT paper (free read-only access via this link: https://t.co/oPTuS5E9qR) as well. Here is our reply to them: https://t.co/ADcaQxa29e https://t.co/zrwarhYhM4

Mar 18, 2024 · 7 likes · 1,343 followers View post →
Prof. Nasser Alzerwi
@NAlzerwi · X

RT @MedEdFlamingo: Couldn't agree more! "Despite the presence of studies that focus on providing human-generated questions to ChatGPT, the…

Feb 26, 2024 · 1 share · 1,755 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

Couldn't agree more! "Despite the presence of studies that focus on providing human-generated questions to ChatGPT, there has been a lack of research focused to asking ChatGPT-generated questions to humans." Ref: 10.1007/s00228-024-03649-x https://t.co/DQuHuIzIyq

Feb 26, 2024 · 1 like · 1 share · 1,343 followers View post →
Teresa Hartman
@thartman2u · X

RT @MedEdFlamingo: "... while it is efficient to generate questions with #ChatGPT, constructing an entire exam without any revisions can be…

Feb 24, 2024 · 1 share · 3,513 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"... while it is efficient to generate questions with #ChatGPT, constructing an entire exam without any revisions can be challenging." #MedEd Ref: 10.1007/s00228-024-03649-x

Feb 23, 2024 · 1 like · 1 share · 1,343 followers View post →
SANDRAJOHNSTON
@sandra04850184 · X

RT @MedEdFlamingo: "... [in generating MCQs using #ChatGPT] it is essential to recognize that the quality of multiple-choice questions is c…

Feb 22, 2024 · 2 shares · 957 followers View post →
Teresa Hartman
@thartman2u · X

RT @MedEdFlamingo: "... [in generating MCQs using #ChatGPT] it is essential to recognize that the quality of multiple-choice questions is c…

Feb 22, 2024 · 2 shares · 3,513 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"... [in generating MCQs using #ChatGPT] it is essential to recognize that the quality of multiple-choice questions is closely tied to prompt quality." Ref: 10.1007/s00228-024-03649-x

Feb 22, 2024 · 3 likes · 2 shares · 1,343 followers View post →
Teresa Hartman
@thartman2u · X

RT @MedEdFlamingo: "The efficiency achieved through #AI would appear remarkable to test developers from a decade ago, who were engaged in t…

Feb 21, 2024 · 2 shares · 3,513 followers View post →
Prof. Nasser Alzerwi
@NAlzerwi · X

RT @MedEdFlamingo: "The efficiency achieved through #AI would appear remarkable to test developers from a decade ago, who were engaged in t…

Feb 21, 2024 · 2 shares · 1,755 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"The efficiency achieved through #AI would appear remarkable to test developers from a decade ago, who were engaged in the effortful task of manually writing multiple-choice questions." #MedEd Ref: 10.1007/s00228-024-03649-x

Feb 21, 2024 · 3 likes · 2 shares · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

Ref: ChatGPT for generating multiple‑choice questions: Evidence on the use of artificial intelligence in automatic item generation for a rational pharmacotherapy exam Here is the link for accessing to a view-only version of the paper: https://t.co/p3OLk1mxQT

Feb 20, 2024 · 1 like · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"This [#AI] transformation is readily observable in our new ability, as humans, to produce appropriate case-based MCQs with minimal human efforts, accomplished by the simple process of inputting a prompt and hitting the enter key." #MedEd Ref: 10.1007/s00228-024-03649-x

Feb 20, 2024 · 2 likes · 1,343 followers View post →
SANDRAJOHNSTON
@sandra04850184 · X

RT @MedEdFlamingo: "... we found that ChatGPT is able to generate multiple choice questions with acceptable levels of item psychometrics. O…

Feb 19, 2024 · 2 shares · 957 followers View post →
MedEd Models
@MedEdModels · X

RT @MedEdFlamingo: "... we found that ChatGPT is able to generate multiple choice questions with acceptable levels of item psychometrics. O…

Feb 19, 2024 · 2 shares · 2,290 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

Ref: ChatGPT for generating multiple‑choice questions: Evidence on the use of artificial intelligence in automatic item generation for a rational pharmacotherapy exam Here is the link for accessing to a view-only version of the paper: https://t.co/p3OLk1mxQT

Feb 19, 2024 · 1 like · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"... we found that ChatGPT is able to generate multiple choice questions with acceptable levels of item psychometrics. Our findings showed that the questions can effectively differentiate between students who perform at high and low levels." Ref: 10.1007/s00228-024-03649-x

Feb 19, 2024 · 8 likes · 2 shares · 1,343 followers View post →
Ikemen
@HandsomeKinoko · X

ChatGPT for generating multiple-choice questions: Evidence on the use of artificial intelligence in automatic i @HandsomeKinoko #note https://t.co/WbwL3tTsyp

Feb 18, 2024 · 1,256 followers View post →
Teresa Hartman
@thartman2u · X

RT @MedEdFlamingo: "Despite the presence of studies that focus on providing human-generated questions to ChatGPT, there has been a lack of…

Feb 18, 2024 · 1 share · 3,513 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

ChatGPT for generating multiple‑choice questions: Evidence on the use of artificial intelligence in automatic item generation for a rational pharmacotherapy exam Here is the link for accessing to a view-only version of the paper: https://t.co/oPTuS5E9qR

Feb 18, 2024 · 1 like · 1,343 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

"Despite the presence of studies that focus on providing human-generated questions to ChatGPT, there has been a lack of research focused to asking ChatGPT-generated questions to humans." Ref: 10.1007/s00228-024-03649-x This is why we conducted this study, btw.

Feb 18, 2024 · 3 likes · 1 share · 1,343 followers View post →
Robert Trevino, MD, PhD
@trevy13 · X

RT @MedEdFlamingo: Our study published today! "#ChatGPT for generating MCQs: Evidence on the use of #artificialintelligence in automatic i…

Feb 14, 2024 · 2 shares · 1,665 followers View post →
Prof. Nasser Alzerwi
@NAlzerwi · X

RT @MedEdFlamingo: Our study published today! "#ChatGPT for generating MCQs: Evidence on the use of #artificialintelligence in automatic i…

Feb 14, 2024 · 2 shares · 1,755 followers View post →
Medical Education Flamingo, MD, PhD
@MedEdFlamingo · X

Our study published today! "#ChatGPT for generating MCQs: Evidence on the use of #artificialintelligence in automatic item generation for a rational pharmacotherapy exam" It presents the first psychometric evidence for ChatGPT-generated case-based MCQs. https://t.co/NxhAuoeGA4

Feb 14, 2024 · 9 likes · 2 shares · 1,343 followers View post →
AMEE Guide No. 178 - AI in HPE Assessment | PDF | Artificial ...
News

doi:10. professions education. Part 2: tools and practical application: 1007/s00228-024-03649-x ... doi:10.1007/978-3-031-72315-5_20 review. Smart Learn ...

Dec 12, 2025 View →