Google DeepMind's use of 1.6m Brits' medical records to test app was 'legally inappropriate'

Privacy watchdog scolds London hospital


Updated Google's use of Brits' real medical records to test a software algorithm that diagnoses kidney disease was legally "inappropriate," says Dame Fiona Caldicott, the National Data Guardian at the UK's Department of Health.

In April 2016 it was revealed the web giant had signed a deal with the Royal Free Hospital in London to build an application called Streams, which can analyze patients' details and identify those who have acute kidney damage. The app uses a fixed algorithm, developed with the help of doctors, rather than anything fancy like AI. It essentially takes things like your blood pressure, age, symptoms, and other personal information, and computes the chance you've suffered acute kidney damage. This is then used to alert docs to provide treatment.

The software – developed by DeepMind, Google's highly secretive machine-learning nerve center – was first tested with simulated data. Then it was tested using 1.6 million sets of real NHS medical files provided by the London hospital. However, not every patient was aware that their data was being given to Google to test the Streams software. Streams has been deployed in wards, and thus now handles real people's details, but during development, it also used live medical records – as well as simulated inputs – and that's what's got the Royal Free into trouble.

Dame Fiona has told the hospital's medical director Professor Stephen Powis that he overstepped the mark, and that there was no consent given by people to have their information used in this way pre-deployment.

"It is my view, and that of my panel, that purpose for the transfer of 1.6 million identifiable patient records to Google DeepMind was for the testing of the Streams application, and not for the provision of direct care to patients," she wrote in a letter dated February, which was leaked to Sky News on Monday.

The pact between the hospital and Google raised eyebrows at the time, but was sold as a legal way to develop apps using sensitive data. It now appears that the US tech goliath and the Royal Free blew it.

"This letter shows that Google DeepMind must know it had to delete the 1.6 million patient medical records it should never have had in the first place," said Phil Booth, coordinator for medical privacy pressure group medConfidential.

"There were legitimate ways for DeepMind to develop the app they wanted to sell. Instead they broke the law, and then lied to the public about it. Such gross disregard of medical ethics by commercial interests – whose vision of 'patient care' reaches little further than their business plan – must never be repeated."

The UK Information Commissioner's Office is investigating the matter and is expected to rule shortly on the Google-Royal Free agreement after academic outcry over the software project.

"We have been very grateful to Dame Fiona for her support [and] advice during this process, and we would absolutely welcome further guidance on this issue," Professor Powis told Sky News.

DeepMind denies any wrongdoing. Its spokespeople stressed to The Register on Monday night that Streams has real benefits for patients.

"Nurses and doctors have told us that Streams is already speeding up urgent care at the Royal Free and saving hours every day. The data used to provide the app has always been strictly controlled by the Royal Free and has never been used for commercial purposes or combined with Google products, services or ads – and never will be," said a DeepMind spokesman.

"Clinicians at the Royal Free put patient safety first by testing Streams with the full set of data before using it to treat patients. Safety testing is essential across the NHS, and no hospital would turn a new service live without testing it first." ®

Editor's note: This article was updated after publication after we learned that Streams is not an AI program, but a fixed algorithm developed with the help of medical professionals. We're happy to clarify this point.


Other stories you might like

  • AMD claims its GPUs beat Nvidia on performance per dollar
    * Terms, conditions, hardware specs and software may vary – a lot

    As a slowdown in PC sales brings down prices for graphics cards, AMD is hoping to win over the market's remaining buyers with a bold, new claim that its latest Radeon cards provide better performance for the dollar than Nvidia's most recent GeForce cards.

    In an image tweeted Monday by AMD's top gaming executive, the chip designer claims its lineup of Radeon RX 6000 cards provide better performance per dollar than competing ones from Nvidia, with all but two of the ten cards listed offering advantages in the double-digit percentages. AMD also claims to provide better performance for the power required by each card in all but two of the cards.

    Continue reading
  • Google opens the pod doors on Bay View campus
    A futuristic design won't make people want to come back – just ask Apple

    After nearly a decade of planning and five years of construction, Google is cutting the ribbon on its Bay View campus, the first that Google itself designed.

    The Bay View campus in Mountain View – slated to open this week – consists of two office buildings (one of which, Charleston East, is still under construction), 20 acres of open space, a 1,000-person event center and 240 short-term accommodations for Google employees. The search giant said the buildings at Bay View total 1.1 million square feet. For reference, that's less than half the size of Apple's spaceship. 

    The roofs on the two main buildings, which look like pavilions roofed in sails, were designed that way for a purpose: They're a network of 90,000 scale-like solar panels nicknamed "dragonscales" for their layout and shimmer. By scaling the tiles, Google said the design minimises damage from wind, rain and snow, and the sloped pavilion-like roof improves solar capture by adding additional curves in the roof. 

    Continue reading
  • Pentester pops open Tesla Model 3 using low-cost Bluetooth module
    Anything that uses proximity-based BLE is vulnerable, claim researchers

    Tesla Model 3 and Y owners, beware: the passive entry feature on your vehicle could potentially be hoodwinked by a relay attack, leading to the theft of the flash motor.

    Discovered and demonstrated by researchers at NCC Group, the technique involves relaying the Bluetooth Low Energy (BLE) signals from a smartphone that has been paired with a Tesla back to the vehicle. Far from simply unlocking the door, this hack lets a miscreant start the car and drive away, too.

    Essentially, what happens is this: the paired smartphone should be physically close by the Tesla to unlock it. NCC's technique involves one gadget near the paired phone, and another gadget near the car. The phone-side gadget relays signals from the phone to the car-side gadget, which forwards them to the vehicle to unlock and start it. This shouldn't normally happen because the phone and car are so far apart. The car has a defense mechanism – based on measuring transmission latency to detect that a paired device is too far away – that ideally prevents relayed signals from working, though this can be defeated by simply cutting the latency of the relay process.

    Continue reading

Biting the hand that feeds IT © 1998–2022