Artificial Intelligence Model Assisting Thyroid Nodule Diagnosis and Management: A Multicenter Diagnostic Study
Authors' conclusion
AI-Thyroid can improve diagnostic performance and interobserver agreement in thyroid cancer diagnosis, especially in less-experienced physicians.
Study context
- Population
- Patients with thyroid nodules imaged for thyroid cancer diagnosis; training at a tertiary hospital (AUMC), validation in 24 hospitals and AUMC6,163 training patients
- Participants
- 4,820 / 2,367 / 4 / 2
- Comparison
- AI-Thyroid diagnostic performance in validation sets; physicians before vs after AI-Thyroid assistance
- Test set 1n = 4,820
- Test set 2n = 2,367
- Group 1n = 4
- Group 2n = 2
- Outcome
- Diagnostic performance of AI-Thyroid for thyroid cancer (AUROC, sensitivity, specificity)
Methodological detailsmulticenter, deep-learning-based artificial intelligence (AI) model, simulated reader study before and after AI assistance
- Multicenter
- Deep-learning-based AI model
- Simulated reader study before and after AI assistance
Performance
As reported in the abstract
Diagnostic performance of AI-Thyroid
| Test or model | Area under the receiver operating characteristic (AUROC) | Sensitivity | Specificity |
|---|---|---|---|
| Test set 1 | 0.922 | 87.0% | 81.5% |
| Test set 2 | 0.938 | 89.9% | 81.6% |
Area under the receiver operating characteristic (AUROC)
Test set 10.922
Test set 20.938
Sensitivity
Test set 187.0%
Test set 289.9%
Specificity
Test set 181.5%
Test set 281.6%
- AUROC of AI-Thyroid
- 0.939
Further findings
AUROC of AI-Thyroid by prevalence of malignancies (>15.0% vs ≤15.0%)
- AUROCs did not differ significantly according to prevalence of malignancies
- P = 0.226
Group 1 (trainees) performance before vs after AI-Thyroid assistance (simulated scenario)
- AUROC changed from 0.854 to 0.945
- P < 0.001
- Sensitivity changed from 84.2% to 92.7%
- P < 0.001
- Specificity changed from 72.9% to 86.6%
- P < 0.001
- All P < .001
Group 2 (faculty radiologists) performance before vs after AI-Thyroid assistance (simulated scenario)
- AUROC changed from 0.914 to 0.939
- P = 0.022
- Sensitivity changed from 78.6% to 85.5%
- P = 0.053
- Specificity changed from 91.9% to 92.5%
- P = 0.683
Interobserver agreement
- Improved from moderate to substantial in both groups
Original publication
Artificial Intelligence Model Assisting Thyroid Nodule Diagnosis and Management: A Multicenter Diagnostic Study. J Clin Endocrinol Metab. Published 25 August 2023. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →