Is your computer doctor secretly a racist? Two US senators want to find out the truth

Garbage in, garbage out

US Senators Ron Wyden (D-OR) and Cory Booker (D-NJ) are examining how federal agencies and healthcare companies are tackling algorithmic biases – after a recent study found that black patients were less likely to be referred to care programs by software than white patients, despite being sicker.

On Tuesday, Wyden (D-OR) and Booker (D-NJ) wrote a series of letters to the Federal Trade Commission and the Centers for Medicare and Medicaid Services, as well as the top five healthcare insurers, including UnitedHealth Group and Blue Cross Blue Shield. Each letter lists specific requests for information, asking if the agencies or companies have any policies, investigations, or internal tools to help them audit the impact of biases in data used to create algorithms.

These algorithms are increasingly being used to calculate an individual’s risk to certain diseases, before automatically deciding what level of care patients should be given. Such software is often guided by historical medical data that may contain implicit racial or gender biases, these are then carried forward in the decision making process to affect marginalized groups.

Wyden and Booker’s efforts were sparked by a paper published in Science last month. Eggheads at the University of California, Berkeley, the University of Chicago, and the Brigham and Women’s Hospital and Massachusetts General Hospital in Boston, studied a commercial algorithm widely used by the American healthcare system to decide whether to further treat people with complex medical problems.

They found that black patients were less likely than white patients to be recommended additional care by the software due to that old chestnut: money. The code was led to believe white people were less healthy and more at risk than black people because more money was spent on treating white people. Therefore, a black person had to be a lot sicker a white person to trigger the same level of additional care. Here's how the paper's abstract put it:

Less money is spent on Black patients who have the same level of need, and the algorithm thus falsely concludes that Black patients are healthier than equally sick White patients. Reformulating the algorithm so that it no longer uses costs as a proxy for needs eliminates the racial bias in predicting who needs extra care.

Ziad Obermeyer, first author of the paper and a researcher at UC Berkeley, further explained: "The algorithms encode racial bias by using health care costs to determine patient ‘risk,’ or who was mostly likely to benefit from care management programs.

"Because of the structural inequalities in our health care system, blacks at a given level of health end up generating lower costs than whites. As a result, black patients were much sicker at a given level of the algorithm’s predicted risk."


Oh dear... AI models used to flag hate speech online are, er, racist against black people


After the researchers altered the algorithm to take into account other variables, such as extra costs that could be cut by having access to other forms of preventative care, the percentage of black patients flagged for further medical help increased from 17.7 per cent to 46.5 per cent.

“In using algorithms, organizations often attempt to remove human flaws and biases from the process," Wyden and Booker said in a statement.

“Unfortunately, both the people who design these complex systems, and the massive sets of data that are used, have many historical and human biases built in. Without very careful consideration, the algorithms they subsequently create can further perpetuate those very biases.”

The FTC, the Centers for Medicare and Medicaid Services, and the five healthcare insurers addressed in the Senators’ letters have until 31 December to respond. ®

Other stories you might like

  • Assange can go to UK Supreme Court (again) to fend off US extradition bid

    Top Brit judges may consider whether an American prison is just too much

    Julian Assange has won a technical victory in his ongoing battle against extradition from the UK to the United States, buying him a few more months in the relative safety of Her Majesty's Prison Belmarsh.

    Today at London's High Court, the Lord Chief Justice of England and Wales Lord Burnett approved a question on a technical point of law, having refused Assange immediate permission to appeal to the UK Supreme Court. The WikiLeaker's lawyers had asked for formal permission to pose this legal conundrum about Assange's likely treatment in US prisons to the Supreme Court:

    Continue reading
  • They see us Cinnamon Rolling, they're rating: GeckoLinux incorporates kernel 5.16 with familiar installation experience

    A nice, clean community distro that works well

    Most distros haven't got to 5.15 yet, but openSUSE's downstream project GeckoLinux boasts 5.16 of the Linux kernel and the latest Cinnamon desktop environment.

    Some of the big-name distros have lots of downstream projects. Debian has been around for decades so has umpteen, including Ubuntu, which has dozens of its own, including Linux Mint, which is arguably more popular a desktop than its parent. Some have only a few, such as Fedora. As far as we know, openSUSE has just the one – GeckoLinux.

    The SUSE-sponsored community distro has two main editions, the stable Leap, which has a slow-moving release cycle synched with the commercial SUSE Linux Enterprise; and Tumbleweed, its rolling-release distro, which gets substantial updates pretty much every day. GeckoLinux does its own editions of both: its remix of Leap is called "GeckoLinux Static", and its remix of Tumbleweed is called "GeckoLinux Rolling".

    Continue reading
  • Running Windows 10? Microsoft is preparing to fire up the update engines

    Winter Windows Is Coming

    It's coming. Microsoft is preparing to start shoveling the latest version of Windows 10 down the throats of refuseniks still clinging to older incarnations.

    The Windows Update team gave the heads-up through its Twitter orifice last week. Windows 10 2004 was already on its last gasp, have had support terminated in December. 20H2, on the other hand, should be good to go until May this year.

    Continue reading
  • Throw away your Ethernet cables* because MediaTek says Wi-Fi 7 will replace them

    *Don't do this

    MediaTek claims to have given the world's first live demo of Wi-Fi 7, and said that the upcoming wireless technology will be able to challenge wired Ethernet for high-bandwidth applications, once available.

    The fabless Taiwanese chip firm said it is currently showcasing two Wi-Fi 7 demos to key customers and industry collaborators, in order to demonstrate the technology's super-fast speeds and low latency transmission.

    Based on the IEEE 802.11be standard, the draft version of which was published last year, Wi-Fi 7 is expected to provide speeds several times faster than Wi-Fi 6 kit, offering connections of at least 30Gbps and possibly up to 40Gbps.

    Continue reading
  • Windows box won't boot? SystemRescue 9 may help

    An ISO image you can burn or drop onto a USB key

    The latest version of an old friend of the jobbing support bod has delivered a new kernel to help with fixing Microsoft's finest.

    It used to be called the System Rescue CD, but who uses CDs any more? Enter SystemRescue, an ISO image that you can burn, or just drop onto your Ventoy USB key, and which may help you to fix a borked Windows box. Or a borked Linux box, come to that.

    SystemRescue 9 includes Linux kernel 5.15 and a minimal Xfce 4.16 desktop (which isn't loaded by default). There is a modest selection of GUI tools: Firefox, VNC and RDP clients and servers, and various connectivity tools – SSH, FTP, IRC. There's also some security-related stuff such as Yubikey setup, KeePass, token management, and so on. The main course is a bunch of the usual Linux tools for partitioning, formatting, copying, and imaging disks. You can check SMART status, mount LVM volumes, rsync files, and other handy stuff.

    Continue reading

Biting the hand that feeds IT © 1998–2022