ASA Main Site  |  Past and Future Meetings
American Surgical Association

Back to 2020 Abstracts


Automated Surveillance Of Surgical Site Infections Using Natural Language Processing: Development And Validation
*Brian T Bucher1, *Jianlin Shi1, *Jeffrey P Ferraro2, *David E Skarda1, *Matthew H Samore1, *Adi V Gundlapalli1, *Wendy W Chapman1, Samuel RG Finlayson1
1University of Utah, Salt Lake City, UT;2Intermountain Healthcare, Salt Lake City, UT

OBJECTIVES: We present the development and validation of a portable natural language processing (NLP) approach for automated surveillance of surgical site infections (SSIs).
METHODS: We abstracted patient clinical text notes after surgical procedures from two independent healthcare systems using different electronic healthcare records. A SSI detected as part of the American College of Surgeons’ National Surgical Quality Improvement Program was used as the reference standard. A rules-based NLP system (EasyCIE-SSI) was developed for operative event-level detection of SSIs using an internal development corpus (4515 operative events, 98,797 notes) from one healthcare system and then validated on a blind corpus from each healthcare system as internal (1475 operative events, 51,691 notes) and external (15,360 operative events, 216,820 notes) validation. Performance was measured using sensitivity, specificity, and area under the receiver-operating-curve (AUC).
RESULTS: The prevalence of SSI was 3.6% and 4.7% in the internal and external validation corpora. In internal validation, EasyCIE-SSI had a sensitivity, specificity, AUC of 92.5%, 88.5%, 0.905 for the detection of SSI, respectively. (Table) In external validation, EasyCIE-SSI had sensitivity, specificity, AUC of 78.2%, 92.2%, 0.851 for the detection of SSI, respectively. In external validation, EasyCIE-SSI performed best for appendectomy (AUC 0.927) and worst for vascular procedures (AUC 0.721).
CONCLUSION: Automated surveillance of surgical site infections in independent healthcare systems can be achieved using natural language processing of clinical notes with high accuracy.
EasyCIE-SSI Performance on the Validation Corpora
Internal ValidationExternal Validation
Procedure TypeSensitivity
(95%CI)
Specificity
(95%CI)
AUC
(95%CI)
Sensitivity
(95%CI)
Specificity
(95%CI)
AUC
(95%CI)
Appendectomy100%
(100%-100%)
93.3%
(90.0%-96.3%)
0.967
(0.951-0.982)
89.7%
(81.0%- 96.6%)
95.8%
(95.0%-96.6%)
0.927
(0.888-0.967)
Breast100%
(100%-100%)
88.7%
(83.0%-93.6%)
0.943
(0.915-0.968)
74.0%
(55.6%-88.9%)
94.7%
(93.6%- 95.9%)
0.844
(0.760-0.929)
Colon and Rectal86.4%
(72.7-100%)
85.6%
(80.5%-90.2%)
0.860
(0.782-0.937)
82.7%
(77.6%-87.3%)
86.3%
(84.4%-88.3%)
0.845
(0.820-0.871)
Hernia100%
(100%-100%)
94.4%
(91.1%-96.8%)
0.972
(0.956-0.986)
79.7
(70.3%-87.8%)
94.7%
(93.9%-95.5%)
0.872
(0.826-0.917)
Hepatobiliary100%
(100%-100%)
89.4%
(84.1%-93.9%)
0.947
(0.921-0.973)
81.6%
(69.4%-91.8%)
91.6%
(89.9%-93.2%)
0.866
(0.810-0.922)
Vascular100%
(100%-100%)
80.9%
(72.3%-88.3%)
0.904
(0.864-0.944)
51.7%
(34.5%-69.0%)
92.4%
(90.9%- 93.4%)
0.721
(0.628-0.814)
All92.5%
(85.2%-98.2%)
88.5%
(86.8%-90.1%)
0.905
(0.869-0.942)
78.2%
(75.3%-81.1%)
92.2%
(91.7%-92.6%)
0.851
(0.837-0.867)


Back to 2020 Abstracts