Evaluation of large language models for carbohydrate counting in type 1 diabetes mellitus: a comparative study with patient estimations
Authors' conclusion
LLMs, particularly Gemini-3 Pro, may support CC when detailed inputs are provided; however, findings should be interpreted cautiously given the small, highly selected sample.
Study context
- Population
- Adults with T1D with advanced carbohydrate counting training; 18 main meals and 4 snacks prepared by dietitians (dietitian values as reference)adults with T1DM
- Participants
- 14 adults
- Exposure
- Large language models (LLMs) vs patients with T1D for carbohydrate counting
- Patientsn = 14reference
- Outcome
- CHO counting accuracy (MAE, MAPE, CA ≤10 g) vs dietitian reference
Methodological detailstwo-stage LLM evaluation: image-only (stage-1) and image-plus-detailed-description (stage-2)
- Two-stage LLM evaluation: image-only (stage-1) and image-plus-detailed-description (stage-2)
Main finding
As reported in the abstract
| Group | Mean absolute error (MAE) | Mean absolute percentage error (MAPE) | Counting accuracy (CA) | MAE (stage-2) | MAPE (stage-2) | CA (stage-2) | Mean bias (stage-2) |
|---|---|---|---|---|---|---|---|
| Patients | 8.31 g | 13.82% | 73.74% | ||||
| Gemini | 2.14 g | 3.21% | 100% | - 1.05 g |
Mean absolute error (MAE)
Patients8.31 g
Gemini— not stated in the abstract
Mean absolute percentage error (MAPE)
Patients13.82%
Gemini— not stated in the abstract
Counting accuracy (CA)
Patients73.74%
Gemini— not stated in the abstract
MAE (stage-2)
Patients— not stated in the abstract
Gemini2.14 g
MAPE (stage-2)
Patients— not stated in the abstract
Gemini3.21%
CA (stage-2)
Patients— not stated in the abstract
Gemini100%
Mean bias (stage-2)
Patients— not stated in the abstract
Gemini- 1.05 g
- Gemini vs Patients: P < 0.001 — stage-2 (image plus detailed description); Gemini outperformed patients
- Stage-1 (image only): lowest MAE, Gemini
- 6.55 g
- Stage-1 (image only): lowest MAPE, Gemini
- 12.54%
- Stage-1 (image only): highest CA, Claude
- 81.8%
Observational data: associations, not effects.
Original publication
Evaluation of large language models for carbohydrate counting in type 1 diabetes mellitus: a comparative study with patient estimations. Diabetes Res Clin Pract. Published 25 April 2026. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →