Acute cardiac care & resuscitation
Suitability of GPT-4o as an evaluator of cardiopulmonary resuscitation skills examinations
Authors' conclusion
GPT-4o demonstrated a level of accuracy that was similar to that of senior experts in examining CPR skills examination videos.
Study context
- Population
- Cardiopulmonary resuscitation (CPR) skills test videos of examinees in the national medical licensing examination
- Participants
- 103 examinees
- Comparison
- GPT-4o vs junior and senior experts scoring CPR skills test videos
- Junior expertsn = 3reference
- Senior expertsn = 3reference
- Outcome
- Accuracy (agreement of GPT-4o vs experts' scores: intraclass correlation coefficient for patient assessment, chest compressions, rescue breathing; Fleiss' Kappa for repeated operations)
Performance
As reported in the abstract
Mean accuracy scores of GPT-4o vs experts, by section
| Test or model | Patient assessment | Chest compressions | Rescue breathing | Repeated operation |
|---|---|---|---|---|
| Junior experts | 0.65 | 0.58 | 0.60 | 0.31 |
| Senior experts | 0.75 | 0.65 | 0.72 | 0.41 |
Patient assessment
Junior experts0.65
Senior experts0.75
Chest compressions
Junior experts0.58
Senior experts0.65
Rescue breathing
Junior experts0.60
Senior experts0.72
Repeated operation
Junior experts0.31
Senior experts0.41
Further findings
| Outcome | Junior experts | Senior experts |
|---|---|---|
| Experts' rating of GPT-4o reliability (5-point Likert scale; 1 completely unreliable, 5 completely reliable)Median | 4.00 (IQR 3.66–4.33) | 4.33 (IQR 4.00–4.67) |
| Mean (SD) | 3.95 [0.55] | 4.29 [0.50] |
Original publication
Suitability of GPT-4o as an evaluator of cardiopulmonary resuscitation skills examinations. Resuscitation. Published 28 September 2024. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →