Computer Vision Analysis of Intraoperative Video: Automated Recognition of Operative Steps in Laparoscopic Sleeve Gastrectomy
Daniel A. Hashimoto*1, Guy Rosman*1, Elan R. Witkowski*1, David W. Rattner1, Keith D. Lillemoe1, Daniela L. Rus*2, Ozanan R. Meireles*1
1Massachusetts General Hospital, Boston, MA;2Massachusetts Institute of Technology, Cambridge, MA
OBJECTIVE: Computer vision, a form of artificial intelligence (AI), allows for quantitative analysis of video by computers for identification of objects and patterns, such as in autonomous driving. This study developed and assessed AI algorithms to identify operative steps in laparoscopic surgery.
METHODS: Intraoperative video from laparoscopic sleeve gastrectomy (LSG) from an academic institution were annotated by two fellowship-trained, board-certified bariatric surgeons. Videos were segmented into the following steps: 1) port placement, 2) liver biopsy, 3) gastrocolic dissection, 4) stapling, 5) bagging. Analysis was focused on steps 2-4 as key visual identifiers of LSG. Deep neural networks (NN) were used to analyze visual data from videos; hidden Markov modeling (hMM) was used to analyze temporal data from videos. Accuracy of operative step identification by the AI was determined by comparing to surgeon annotations.
RESULTS: 37 cases of LSG were analyzed to generate 168 video clip segments of operative steps. A random 70% sample of these clips were used to train the AI, and 30% were used to test the AIís performance. Accuracy of the AI in identifying operative steps using visual data only was 72%; use of hMM added a minimum of 2% to identification accuracy. Mean concordance correlation coefficient for human annotators was 0.862, suggesting excellent agreement.
CONCLUSIONS: AI can extract quantitative surgical data from video with minimum of 72% accuracy. This suggests operative video could be used as a quantitative data source for research in intraoperative clinical decision support, risk prediction, or outcomes studies.
* By Invitation
Back to 2019 Abstracts