Machine learning the hard way: IBM Watson's fatal misdiagnosis

The doctor won't see you now


Opinion It started in Jeopardy and ended in loss. IBM's flagship AI Watson Health has been sold to venture capitalists for an undisclosed sum thought to be around a billion dollars, or a quarter of what the division cost IBM in acquisitions alone since it was spun off in 2015.

Not the first nor the last massively expensive tech biz screw-up, but isn't AI supposed to be the future? Isn't IBM supposed to be good at this?

It all started so well. One of Watson's early set pieces was taking a complex set of symptoms and finding the most probable diagnosis out of an encyclopaedic knowledge of rare diseases. A different challenge marked its demise. Like a corpse with a broken neck, 15 bullet holes and a strong smell of cyanide, it raised the question: which massive failure actually finished it off?

A good doctor takes a comprehensive history, so let's start at birth. The first most people knew of IBM Watson was in 2011, when it used its natural language processing and capacious knowledge models to win Jeopardy, an American TV game show.

A PR and marketing coup, IBM lost no time in launching Watson Health off the back of it. With this sort of automated analytics, the company promised, doctors could diagnose more accurately, more swiftly and at less expense. Treatments would be more precisely targeted for more patients. A revolution in healthcare was underway.

That's a lot to promise off the back of a gameshow, but IBM was bullish. Shiny new Manhattan headquarters were unveiled with an "immersion room" that put punters and journalists inside a planetarium-like dome where Watson could display its fabulous talents. Institutional and commercial partners were signed up for co-development, and the magic of turning hype into reality kicked off.

IBM tried to bypass that by buying other companies with successful AI medical products, to absorb their goodness, but Watson consistently rejected the transplants.

In 2019, IEEE Spectrum, the flagship mag of the American professional body for electronic engineering, reported that around 50 partnerships had been announced since launch between IBM Watson and healthcare organisations.

It listed 20 of the highest profile ones, with such august bodies as the Mayo Clinic, American national organisations for cancer, cardiology and oncological research, and numerous hospitals and companies. None had produced usable tools or apps.

At the time (we can't speak to today), Watson Health didn't seem to work. It is usually possible to fudge such misfortune in business technology, because stats can be dressed up, returns on investment left agreeably fuzzy, and sufficient figleaves plucked and donned for CIOs to move on without shame to the next failure. Medicine uses real statistics. It publishes. It checks outcomes, because it's not selling widgets, it's trying to keep people not dead, ideally happily so.

When clinical trials were published, Watson came up short every time. It didn't matter what field it was in, it consistently scored less well than human clinicians – sometimes under 50 per cent – and demonstrated some alarming blind spots in suggested treatments. Medical professionals had enough to worry about without babysitting a broken AI: it got dropped.

You can learn the rules for Jeopardy in a minute. Becoming a doctor takes 10 years. Becoming the best doctor you can be takes a lifetime. Medical data, whether in the literature or in test results, is meaningless or misleading without a lot of implied context. Watson Health couldn't work across lots of fields at once; that needs a general intelligence which AI currently does not have. It needed to evolve from the ground up with experts in each specialism, letting them set the rules, the ways of working, what knowledge mattered and why. You can't do that with an immersion room. If you promise and do not deliver, you won't get the help you need to make things better. Once you've blown trust, you're toast.

IBM tried to bypass that by buying other companies with successful AI medical products, to absorb their goodness, but Watson Health consistently rejected the transplants. Companies that could prosper on their own against small, nimble, focused competition couldn't thrive when sewn into Watson's marketing-led system. Customers left, and the newly acquired employees dumped when the numbers fell apart.

IBM's Watson Health failed at the time, like so much AI/ML, because it didn't know what the question was – ironic, since the game of Jeopardy at which it excelled is all about deducing questions from data. It wanted to automate the highest skilled aspects of healthcare, diagnosis and treatment, but the problem wasn't one of getting the most data and the best algorithm. Rather, the problem was one of meaning.

A good doctor sees the patient, not the symptoms. Watson saw the symptoms of inefficiency and lack of capability. It did not see the process of care and making whole, where doctors, not data, were what needed to be understood.

Fortunately, this sad case does not mean AI-based tools can't work in medicine, nor even that Watson Health won't do so, nor that AIs' evaluation and adoption has been slowed down. Ask the right questions, such as "why do some patients get readmitted" and can you spot them early?, and you get the right answers. AI in healthcare is being adopted, and it shows every sign of deserving its place.

It only wins that place when it's held to the same standards as any other aspect of healthcare. AI/ML in general business will only succeed once we learn to spot the Watsons. Set early tests with clear goals and clear results. Set time limits – it took Watson Health many years to die, but it was failing within three. If you can't deliver on proof of concept, stop. If you can't even decide on a proof of concept, don't even start.

Perhaps the best legacy Watson Health has left from its short, troubled time with us is a game in the spirit of Jeopardy. Let's call it Hyperspace. Just enter "IBM Watson" and a year between 2011 and 2021 into the search engine of your choice, and see if you can work out just how badly things had gone by that year from the mix of IBM-led hype and reality-led news reports. There's a pattern there. It's worth learning – after all, it cost enough. ®

Similar topics


Other stories you might like

  • Lonestar plans to put datacenters in the Moon's lava tubes
    How? Founder tells The Register 'Robots… lots of robots'

    Imagine a future where racks of computer servers hum quietly in darkness below the surface of the Moon.

    Here is where some of the most important data is stored, to be left untouched for as long as can be. The idea sounds like something from science-fiction, but one startup that recently emerged from stealth is trying to turn it into a reality. Lonestar Data Holdings has a unique mission unlike any other cloud provider: to build datacenters on the Moon backing up the world's data.

    "It's inconceivable to me that we are keeping our most precious assets, our knowledge and our data, on Earth, where we're setting off bombs and burning things," Christopher Stott, founder and CEO of Lonestar, told The Register. "We need to put our assets in place off our planet, where we can keep it safe."

    Continue reading
  • Conti: Russian-backed rulers of Costa Rican hacktocracy?
    Also, Chinese IT admin jailed for deleting database, and the NSA promises no more backdoors

    In brief The notorious Russian-aligned Conti ransomware gang has upped the ante in its attack against Costa Rica, threatening to overthrow the government if it doesn't pay a $20 million ransom. 

    Costa Rican president Rodrigo Chaves said that the country is effectively at war with the gang, who in April infiltrated the government's computer systems, gaining a foothold in 27 agencies at various government levels. The US State Department has offered a $15 million reward leading to the capture of Conti's leaders, who it said have made more than $150 million from 1,000+ victims.

    Conti claimed this week that it has insiders in the Costa Rican government, the AP reported, warning that "We are determined to overthrow the government by means of a cyber attack, we have already shown you all the strength and power, you have introduced an emergency." 

    Continue reading
  • China-linked Twisted Panda caught spying on Russian defense R&D
    Because Beijing isn't above covert ops to accomplish its five-year goals

    Chinese cyberspies targeted two Russian defense institutes and possibly another research facility in Belarus, according to Check Point Research.

    The new campaign, dubbed Twisted Panda, is part of a larger, state-sponsored espionage operation that has been ongoing for several months, if not nearly a year, according to the security shop.

    In a technical analysis, the researchers detail the various malicious stages and payloads of the campaign that used sanctions-related phishing emails to attack Russian entities, which are part of the state-owned defense conglomerate Rostec Corporation.

    Continue reading
  • FTC signals crackdown on ed-tech harvesting kid's data
    Trade watchdog, and President, reminds that COPPA can ban ya

    The US Federal Trade Commission on Thursday said it intends to take action against educational technology companies that unlawfully collect data from children using online educational services.

    In a policy statement, the agency said, "Children should not have to needlessly hand over their data and forfeit their privacy in order to do their schoolwork or participate in remote learning, especially given the wide and increasing adoption of ed tech tools."

    The agency says it will scrutinize educational service providers to ensure that they are meeting their legal obligations under COPPA, the Children's Online Privacy Protection Act.

    Continue reading
  • Mysterious firm seeks to buy majority stake in Arm China
    Chinese joint venture's ousted CEO tries to hang on - who will get control?

    The saga surrounding Arm's joint venture in China just took another intriguing turn: a mysterious firm named Lotcap Group claims it has signed a letter of intent to buy a 51 percent stake in Arm China from existing investors in the country.

    In a Chinese-language press release posted Wednesday, Lotcap said it has formed a subsidiary, Lotcap Fund, to buy a majority stake in the joint venture. However, reporting by one newspaper suggested that the investment firm still needs the approval of one significant investor to gain 51 percent control of Arm China.

    The development comes a couple of weeks after Arm China said that its former CEO, Allen Wu, was refusing once again to step down from his position, despite the company's board voting in late April to replace Wu with two co-chief executives. SoftBank Group, which owns 49 percent of the Chinese venture, has been trying to unentangle Arm China from Wu as the Japanese tech investment giant plans for an initial public offering of the British parent company.

    Continue reading
  • SmartNICs power the cloud, are enterprise datacenters next?
    High pricing, lack of software make smartNICs a tough sell, despite offload potential

    SmartNICs have the potential to accelerate enterprise workloads, but don't expect to see them bring hyperscale-class efficiency to most datacenters anytime soon, ZK Research's Zeus Kerravala told The Register.

    SmartNICs are widely deployed in cloud and hyperscale datacenters as a means to offload input/output (I/O) intensive network, security, and storage operations from the CPU, freeing it up to run revenue generating tenant workloads. Some more advanced chips even offload the hypervisor to further separate the infrastructure management layer from the rest of the server.

    Despite relative success in the cloud and a flurry of innovation from the still-limited vendor SmartNIC ecosystem, including Mellanox (Nvidia), Intel, Marvell, and Xilinx (AMD), Kerravala argues that the use cases for enterprise datacenters are unlikely to resemble those of the major hyperscalers, at least in the near term.

    Continue reading

Biting the hand that feeds IT © 1998–2022