ChatGPT-4's Accuracy in Estimating Thyroid Nodule Features and Cancer Risk From Ultrasound Images
Authors' conclusion
While GPT-4 and GPT-4o models show potential for improving the efficiency of thyroid nodule triage, their performance remains suboptimal, particularly in higher-risk categories. Further refinement and validation of these models are necessary before clinical implementation.
Study context
- Population
- Thyroid nodule US images from 3 open-access databases, assessed with ACR Thyroid Imaging Reporting and Data System (TI-RADS)
- Participants
- 202 thyroid US images
- Comparison
- GPT-4 and GPT-4o vs expert radiologists' reference standard for TI-RADS features and categories
- Outcome
- Accuracy in identifying TI-RADS features and categories vs the reference standard
Methodological detailscomparative validation study, 3 open-access databases, Conducted between October 2023 and May 2024
- Comparative validation study
- 3 open-access databases
- Conducted between October 2023 and May 2024
Performance
As reported in the abstract
TI-RADS categories, complete images
| Test or model | Sensitivity, low-risk (benign) | Specificity, low-risk (benign) | Accuracy, low-risk (benign) | Sensitivity, moderately suspicious | Specificity, moderately suspicious | Accuracy, moderately suspicious |
|---|---|---|---|---|---|---|
| GPT-4 | 25.0% | 99.5% | 93.6% | 75% | 22.2% | 42.1% |
GPT-4
Sensitivity, low-risk (benign)25.0%
Specificity, low-risk (benign)99.5%
Accuracy, low-risk (benign)93.6%
Sensitivity, moderately suspicious75%
Specificity, moderately suspicious22.2%
Accuracy, moderately suspicious42.1%
Further findings
TI-RADS features, complete images
- Smooth margins, GPT-4 vs reference standard
- 73% vs 65%
- Isoechoic echogenicity, GPT-4 vs reference standard
- 5% vs 46%
- Echogenic foci, GPT-4 vs reference standard
- 3% vs 27%
Cropped images, GPT-4 and GPT-4o
- Similar patterns, with slight deviations indicating decreased performance vs GPT-4's assessment of complete images
Original publication
ChatGPT-4's Accuracy in Estimating Thyroid Nodule Features and Cancer Risk From Ultrasound Images. Endocr Pract. Published 24 March 2025. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →