Is Risk Adjusted Mortality An Indicator Of Quality Of Care In General Surgery? A Comparison Of Risk Adjustment To Peer Review
Steven Shackford, *Talia Ben-Jacob, *John Ratliff, Neil Hyman
University of Vermont College of Medicine, Burlington, VT
OBJECTIVE(S): Profiling of hospitals using risk-adjusted mortality rates as a measure of quality is becoming increasingly frequent. We aimed to determine the validity of this approach by comparing the risk adjusted predicted mortality to the findings of concurrent peer review and retrospective chart review of deaths that occur on a general surgery service.
METHODS: Consecutive patients admitted to a busy general surgery service from 1/00-1/06 were prospectively entered into the Surgical Activity Tracking System. Rigorous, systematic peer review was performed concurrently by service members on all deaths. Adjudication was later validated by an independent senior surgeon. Three methodologies of risk adjustment (University Health Consortium [UHC], Physiological and Operative Severity Score for the enUmeration of Mortality [POSSUM], and the Charlson index) and compared the “excess mortality” predicted by each to the number of potentially preventable deaths determined by peer review.
RESULTS: 9623 patients were admitted and 75 died (0.7%). UHC and POSSUM predicted an excess mortality of 62 and 65 deaths, respectively; Charlson predicted that 73% of the cohort would be dead in 1 year. Concurrent and retrospective peer review found that death was potentially preventable in only 22 and 21 patients, respectively.
CONCLUSIONS: Peer adjudication and extensive clinical review adds much to the analysis of an adverse outcome, similar to the “black box” in an airplane crash. While methods of risk adjustment may be helpful in identifying patients for peer review, they should be used for internal process improvement and not published as metrics of hospital or provider performance.