American Surgical Association Annual Meeting
Search Meeting Site Only
 
Main ASA Website
Current Meeting Home
Final Program
Past & Future Meetings

 

 

Back to 2016 Annual Meeting


Is the Publicly Available ProPublica Surgeon Scorecard Valid? An Empirical Evaluation of their Outcome Measure and Methods
Kristen A Ban*1, Mark E. Cohen*1, Clifford Y Ko2, Mark W. Friedberg*3, Lynn Zhou*1, Bruce L. Hall4, David B. Hoyt1, Karl Y Bilimoria*5
1American College of Surgeons, Chicago, IL;2University of California Los Angeles, Los Angeles, CA;3RAND Corporation, Santa Monica, CA;4Washington University in St Louis, St. Louis, MO;5Northwestern University, Chicago, IL

OBJECTIVE(S): The ProPublica Surgeon Scorecard is the first nationwide, multispecialty public reporting of individual surgeon outcomes. However, many question ProPublica’s previously undescribed outcome measure (composite of in-hospital Mortality and 30-Day Readmission) which excludes all inpatient complications. Our objective was to examine the validity of ProPublica’s measure by (1) comparing performance on ProPublica’s outcome measure to well-established postoperative outcome measures and (2) assessing the proportion of complications missed by ProPublica’s exclusion of inpatient complications.
METHODS: Using ACS-NSQIP data (2011-2014) for ProPublica operations and all general surgery cases, risk-adjusted performance on the ProPublica measure was compared to established outcome measure performance (e.g., Death/Serious Morbidity, Mortality) using hierarchical regression models. In addition, the proportion of 30-day complications missed by ProPublica’s measure was assessed by determining the proportion of complications which occur as an inpatient versus after discharge.
RESULTS: For all general surgery cases, the ProPublica adjusted complication measure correlated poorly with Death/Serious Morbidity (Pearson r=0.324, p<0.001), Mortality (r=0.163, p<0.001), Serious Morbidity (r=0.182, p<0.001), and SSI (r=0.210, p<0.001). ProPublica’s complication measure correlated strongly with the Readmission measure (r=0.978, p <0.001). Results were similar when examining individual operations separately (laparoscopic cholecystectomy, prostatectomy, knee replacement). For all eight ProPublica operations, the ProPublica outcome measure missed, on average, the 87% of complications occurring during inpatient hospitalization (range: 57% missed for TURP to 90% missed for knee replacement).
CONCLUSIONS: As ProPublica’s new complication measure correlates poorly with well-established postoperative outcomes and misses nearly 90% of postoperative complications, the validity of the ProPublica Surgeon Scorecard is suspect.


Back to 2016 Annual Meeting


© 2022 American Surgical Association. All Rights Reserved. Privacy Policy.