Comparing ChatGPT's and Surgeon's Responses to Thyroid-related Questions From Patients
Authors' conclusion
ChatGPT performed better than a junior specialist and senior specialist in answering common thyroid-related questions, but further research is needed to validate the logical ability of the ChatGPT for complex thyroid questions.
Study context
- Population
- 30 thyroid-related questions (28 from the Huayitong app plus 2 interfering questions); responses evaluated by 26 patients and 11 thyroid surgeons
- Participants
- 30 questions
- Exposure
- ChatGPT (GPT-4.0) vs junior specialist vs senior specialist responses to thyroid-related questions
- Outcome
- Accuracy, comprehensiveness, compassion and satisfaction of responses
Methodological detailsResponses obtained July 19–20, 2023; evaluated by 26 patients and 11 thyroid surgeons
- Responses obtained July 19–20, 2023; evaluated by 26 patients and 11 thyroid surgeons
Main finding
As reported in the abstract
Accuracy, comprehensiveness, compassion and satisfaction
- ChatGPT received higher scores than the junior and senior specialist in accuracy, comprehensiveness, compassion and satisfaction
Observational data: associations, not effects.
Further findings
| Outcome | ChatGPT | Junior specialist | Senior specialist | P value |
|---|---|---|---|---|
| Word count of responsesChatGPT vs Junior specialist · greater with ChatGPT | 341.50 (range 301.00–384.25) | 74.50 (range 51.75–84.75) | 104.00 (range 63.75–177.75) | P < 0.001 |
| Word count of responsesChatGPT vs Senior specialist · greater with ChatGPT | P < 0.001 |
- Speed of response · ChatGPT vs Junior specialist · faster with ChatGPT; 8.69 [7.53-9.48] vs 4.33 [4.05-4.60]
- P < 0.001
- Speed of response · ChatGPT vs Senior specialist · faster with ChatGPT; 8.69 [7.53-9.48] vs 4.22 [3.36-4.76]
- P < 0.001
Original publication
Comparing ChatGPT's and Surgeon's Responses to Thyroid-related Questions From Patients. J Clin Endocrinol Metab. Published 10 April 2024. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →