The UK’s data protection watchdog has ruled that DeepMind’s first deal with the NHS “failed to comply with data protection law.”
The Information Commissioner’s Office (ICO) announced its verdict on the controversial data sharing-agreement between DeepMind and the Royal Free London NHS Trust on Monday after a year long investigation.
The agreement — revealed in full by New Scientist in April 2016 — gave the Google-owned artificial intelligence (AI) lab access to 1.6 million NHS patient records across three North London hospitals without patient’s prior knowledge.
The deal was signed to help DeepMind develop an app called Streams, which sends an alert to a clinician’s smartphone if a patient’s condition deteriorates. It also allows clinicians to view a patient’s medical records and see where patients are being looked after. It doesn’t use any of the AI that DeepMind is known for. Through the agreement, DeepMind was able to see whether people are HIV-positive as well as details of drug overdoses and abortions.
DeepMind and the Royal Free tried to justify the data-sharing deal by saying that “implied consent” was assumed because the Streams app was delivering “direct care” to patients.
But the ICO, which launched its investigation last May after receiving at least one complaint from the public, said it found several shortcomings in how the data was handled, including that patients were not adequately informed that their data would be used as part of the test.
Elizabeth Denham, Information Commissioner, said in a statement:
“There’s no doubt the huge potential that creative use of data could have on patient care and clinical improvements, but the price of innovation does not need to be the erosion of fundamental privacy rights.
“Our investigation found a number of shortcomings in the way patient records were shared for this trial. Patients would not have reasonably expected their information to have been used in this way, and the Trust could and should have been far more transparent with patients as to what was happening.
“We’ve asked the Trust to commit to making changes that will address those shortcomings, and their co-operation is welcome. The Data Protection Act is not a barrier to innovation, but it does need to be considered wherever people’s data is being used.”
The ICO was advised by the National Data Guardian (NDG) Dame Fiona Caldicott, the UK health data regulator, during its investigation.
A leaked letter from Caldicott to the Royal Free in May revealed that the she considered the deal to be “inappropriate”.
“It is my view,” Caldicott wrote in the letter dated 20 February, “that the purpose for the transfer of 1.6 million identifiable patient records to Google DeepMind was for the testing of the Streams application, and not for the provision of direct care to patients.”
“Given that Streams was going through testing and therefore could not be relied up for patient care, any role the application might have played in supporting the provision of direct care would have been limited and secondary to the purpose of the data transfer.”
The deal has attracted criticism from academics, lawyers, and privacy campaigners.
For example, the “Google DeepMind and healthcare in an age of algorithms” paper — coauthored by Cornell University’s Julia Powles and The Economist’s Hal Hodson — questioned why DeepMind was given permission to process millions of NHS patient records so easily and without patient approval. It concluded that the deal was riddled with “inexcusable” mistakes.DeepMind set up its own independent review panel last year to scrutinise the work it is doing with the NHS.
The panel is reviewing DeepMind’s data sharing agreements, its privacy and security measures, and its product roadmaps. It is due to brief journalists about the findings of the report at the Science Media Centre in London on Tuesday, before releasing it on Wednesday.
“Working in healthcare requires regular and independent oversight,” DeepMind writes on its website. “We have asked a number of respected public figures to act in the public interest as unpaid Independent Reviewers of DeepMind Health.”
DeepMind cofounder Mustafa Suleyman defended the data-sharing agreement last year, saying: “As Googlers, we have the very best privacy and secure infrastructure for managing the most sensitive data in the world.”
Two other NHS trusts have signed deals with DeepMind to use its Streams app. Imperial College NHS Foundation Trust announced a deal with DeepMind in ??? and Taunton and Somerset NHS Foundation Trust announced one in ???
While these deals also give DeepMind access to patient records, the purposes for which DeepMind can use data are far better constrained than in first deal. The amount of data being shared is also more proportional.