Evaluating Large Language Models-Generated Health Education Materials for Discharged Patients with Diabetes: A Comparative Analysis
Authors' conclusion
While ChatGPT-4o, Doubao 1.5, and DeepSeek R1 generate accurate and comprehensible materials, concerns remain regarding their actionability and safety. These findings suggest that LLMs should be used as auxiliary tools in diabetes education, requiring further refinement for personalized and actionable content.
Study context
- Population
- De-identified medical records of discharged patients with diabetes; materials rated by experienced diabetes nursing experts
- Participants
- 10 de-identified medical records
- Comparison
- ChatGPT-4o vs Doubao 1.5 vs DeepSeek R1 for generating health education materials
- Outcome
- Quality of generated materials
Main finding
As reported in the abstract
- Comprehensibility score pass rates >70% for all models, DeepSeek R1 best
- Actionability score pass rates <70% for all models, no significant differences (P > .01)
- Accuracy scores ≥98% for all models, no significant differences (P > .01)
- No significant differences in personalization and effectiveness scores (P > .01)
- Highest safety score with DeepSeek R1, lowest with Doubao 1.5 (P < .01)
- Between ChatGPT-4o and Doubao 1.5: P < 0.01
Original publication
Evaluating Large Language Models-Generated Health Education Materials for Discharged Patients with Diabetes: A Comparative Analysis. J Diabetes Sci Technol. Published 10 March 2026. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →