Heart failure & cardiomyopathiesDiagnostic or prognostic study
Automated Identification of Heart Failure With Reduced Ejection Fraction Using Deep Learning-Based Natural Language Processing
- Diagnostic accuracy study
- external validation
- deep-learning NLP model
Authors' conclusion
The authors developed a language model that identifies HFrEF from clinical notes with high precision and accuracy, representing a key element in automating quality assessment for individuals with HFrEF.
Study context
- Population
- Hospitalizations with HF at Yale New Haven Hospital 2015–2019; external validation at Northwestern Medicine, Yale community hospitals and MIMIC-III13,251 notes
- Participants
- 5,392 hospitalizations
- Comparison
- Deep-learning language model for heart failure with reduced ejection fraction (HFrEF) identification from discharge summaries
- Outcome
- Detection of HFrEF (LVEF <40% on antecedent echocardiography)
Performance
As reported in the abstract
Detection of HFrEF (held-out set)
| Test or model | AUROC | Area under precision recall curve (AUPRC) |
|---|---|---|
| Model | 0.97 | 0.97 |
Further findings
| Outcome | Values | net reclassification improvement | P value |
|---|---|---|---|
| External validation: Northwestern Medicine (19,242 notes) | Model AUROC 0.94 · AUPRC 0.91 | ||
| External validation: Yale community hospitals (139 manually abstracted notes) | Model AUROC 0.95 · AUPRC 0.96 | ||
| External validation: MIMIC-III (146 manually reviewed notes) | Model AUROC 0.91 · AUPRC 0.92 | ||
| Net reclassification improvement vs diagnosis codes · ± 1.9% | 60.2% | P < 0.001 |
Original publication
Automated Identification of Heart Failure With Reduced Ejection Fraction Using Deep Learning-Based Natural Language Processing. JACC Heart Fail. Published 23 October 2024. doi:10.1016/j.jchf.2024.08.012
Summarised from the published abstract; every number is checked against it