Racial bias in widely used hospital algorithm, study finds


>>Sreenivasan: A RECENT STUDY PUBLISHED IN “SCIENCE MAGAZINE” FOUND SIGNIFICANT RACIAL BIAS IN AN ALGORITHM USED BY HOSPITALS ACROSS THE NATION TO DETERMINE WHO NEEDS FOLLOW UP CARE AND WHO DOES NOT. MEGAN THOMPSON RECENTLY SPOKE WITH “STATS” SHRADDHA CHAKRADHAR, WHO EXPLAINED WHAT THE RESEARCHERS FOUND.>>Thompson: WHERE EXACTLY WAS THIS BIAS COMING FROM?>>THERE ARE TWO WAYS THAT WE CAN IDENTIFY HOW SICK A PERSON IS. ONE IS HOW MANY DOLLARS ARE SPENT ON THAT PERSON. YOU KNOW, THE ASSUMPTION BEING THE MORE HEALTHCARE THEY COME IN FOR, THE MORE TREATMENT THAT THEY GET, THE MORE DOLLARS THEY SPEND. AND PRESUMABLY THE SICKER THEY ARE IF THEY’RE GETTING ALL THAT TREATMENT. AND THE OTHER WAY IS THAT, YOU KNOW, WE CAN MEASURE, ACTUAL BIOPHYSICAL THINGS, YOU KNOW, FROM LAB TESTS TO WHAT KIND OF CONDITIONS OR DISEASES THEY MIGHT HAVE. SO IT SEEMS LIKE THIS ALGORITHM WAS RELYING ON THE COST PREDICTION DEFINITION. IN OTHER WORDS, THE MORE DOLLARS A PATIENT WAS PROJECTED TO SPEND ON THE PART OF AN INSURANCE COMPANY OR A HOSPITAL, THEN THAT WAS A SIGN OF HOW SICK THEY WERE GOING TO BE. UM, AND THAT SEEMS TO BE WHERE THE BIAS EMERGED.>>Thompson: I UNDERSTAND THAT THE RESEARCHERS THEN SORT OF RE– USE THE ALGORITHM USING A DIFFERENT TYPE OF DATA. CAN YOU JUST TELL US A LITTLE BIT MORE ABOUT THAT? WHAT DID THEY USE?>>YEAH, SO INSTEAD OF RELYING ON JUST COSTS TO PREDICT WHICH PATIENTS ARE GOING TO NEED FOLLOW UP CARE THEY ACTUALLY USE BIOMETRIC DATA, PHYSICAL– BIOPHYSICAL DATA, PHYSIOLOGICAL DATA. UM, AND THEY SAW A DRAMATIC DIFFERENCE. YOU KNOW, IN THE PREVIOUS MODEL, THE ALGORITHM MISSED SOME 48,000 EXTRA CHRONIC CONDITIONS THAT AFRICAN AMERICAN PATIENTS HAD. BUT WHEN THEY REJIGGERED THE ALGORITHM TO LOOK MORE AT ACTUAL BIOLOGICAL DATA, THEY BROUGHT THAT DOWN TO ABOUT 7,700. SO IT WAS ABOUT AN 84% REDUCTION IN BIAS.>>Thompson: DO WE KNOW ANYTHING ABOUT HOW THE USE OF THIS BIASED ALGORITHM ACTUALLY AFFECTED PATIENT CARE?>>UM, WE DON’T ACTUALLY KNOW THAT BUT, AS I MENTIONED, THE ALGORITHM IS USED BY HOSPITALS TO HELP THEM FLAG PATIENTS WHO MIGHT NEED EXTRA CARE IN THE COMING YEAR. WHETHER IT’S, YOU KNOW, AN AT HOME NURSE OR MAKING SURE THAT THEY COME IN FOR REGULARLY SCHEDULED DOCTOR’S APPOINTMENTS. SO, WE CAN ONLY PRESUME THAT IF BLACK PATIENTS– SICKER BLACK PATIENTS WEREN’T BEING FLAGGED ACCURATELY THAT THEY ALSO MISSED OUT ON THIS FOLLOW UP CARE.>>Thompson: ARE THERE ANY CONSEQUENCES FOR THE COMPANY, OPTUM, THAT WAS BEHIND THIS ALGORITHM?>>YEAH. SO, THE DAY AFTER THE STUDY CAME OUT, ACTUALLY, NEW YORK REGULATORS, THE DEPARTMENT OF FINANCIAL SERVICES AND THE DEPARTMENT OF HEALTH SENT A LETTER TO THE COMPANY SAYING THEY WERE INVESTIGATING THIS ALGORITHM AND THAT THE COMPANY HAD TO SHOW THAT THE WAY THE ALGORITHM WORKED WASN’T IN VIOLATION OF ANTI DISCRIMINATION LAWS IN NEW YORK. SO THAT INVESTIGATION IS PENDING. ONE ENCOURAGING THING IS THAT WHEN THE RESEARCHERS DID THE STUDY, THEY ACTUALLY REACHED BACK TO OPTUM AND LET THEM KNOW ABOUT THE DISCREPANCY IN THE DATA, AND THE COMPANY WAS GLAD TO BE TOLD ABOUT IT. AND, UM, I’M TOLD THAT THEY’RE WORKING ON A FIX, UM, AND THE OTHER, THE OTHER ENCOURAGING THING IS THAT THE RESEARCHERS HAVE ACTUALLY NOW LAUNCHED AN INITIATIVE TO HELP OTHER COMPANIES WHO MAY BE BEHIND SIMILAR ALGORITHMS, TO HELP THEM FIX ANY BIASES IN THEIR PROGRAMS. SO, THEY’VE LAUNCHED A PROGRAM BASED OUT OF THE UNIVERSITY OF CHICAGO’S BOOTH SCHOOL TO DO THIS WORK ON A PRO BONO BASIS SO THAT THEY CAN SORT OF CATCH THESE THINGS IN OTHER ALGORITHMS THAT MIGHT BE USED ACROSS THE COUNTRY.>>Thompson: ALL RIGHT. SHRADDHA CHAKRADHAR OF “STAT.” THANK YOU SO MUCH FOR BEING WITH US.>>THANK YOU FOR HAVING ME.