Google DeepMind may have received personally identifying medical records of over a million NHS patients on a “legally inappropriate” basis.
That’s according to a letter from National Data Guardian (NDG) Dame Fiona Caldicott to NHS Royal Free Trust medical director Stephen Powis on February 20 that was leaked to Sky News.
DeepMind, a London-based artificial intelligence (AI) research lab acquired by Google in 2014 for £400 million, was quietly given access to 1.6 million NHS patient records in 2015 to help it develop a mobile app called Streams on the legal basis that it was offering “direct care.”
Legally speaking, patients are “implied” to have consented to their medical records being shared if it was shared for the purpose of “direct care.”
But Caldicott wrote in her letter: “When I wrote to you in December, I said that I did not believe that when the patient data was shared with Google DeepMind, implied consent for direct care was an appropriate legal basis.”
Caldicott, who does not dispute the app’s ability to help clinicians save lives today, added: “Given that Streams was going through testing and therefore could not be relied upon for patient care, any role the application may have played in supporting the provision of direct care would have been limited and secondary to the purpose of the data transfer.
“My considered opinion therefore remains that it would not have been within this reasonable expectation of patients that their records would have been shared for this purpose.”
The letter, which DeepMind was expecting to be published last Friday, was sent by email to people including DeepMind cofounder Mustafa Suleyman and DeepMind strategy lead, Will Cavendish.
A spokesperson for the National Data Guardian confirmed to Business Insider that the letter is real.
The National Data Guardian, which was set up to help ensure that citizens’ confidential medical information is safeguarded securely and used properly, explained at the end of the letter that she would be passing on her views to the Information Commissioner, Elizabeth Denham, who is the data protection officer of the UK.
The Information Commissioner’s Office is carrying out its own investigation into whether the data transfer between from Royal Free to DeepMind was legal under the Data Protection Act. A verdict is expected to be made public in the coming weeks.
Streams is a clinical app designed to help clinicians identify and treat patients as quickly as possible. It pulls together patient information and generates an automatic alert when a problem is detected. It does not make use of DeepMind’s AI or machine learning technologies.
Following the initial testing period, the Royal Free rolled out the Streams app to a number of clinicians in its hospitals as part of a subsequent deal, which is not being investigated.
DeepMind did not immediately respond to Business Insider’s request for comment but Dominic King, the clinical lead at Google DeepMind, told Sky News: “It’s really important to say that DeepMind is a British company, and although acquired by Google, operates independently. At no point has any patient data been shared with other Google products or services, or used for commercial purposes.
“I think one thing that we do recognise that we could have done better is make sure that the public are really informed about how their data is used.”
Google DeepMind’s first deal with Royal Free London was torn apart in an academic paper published in March. The “Google DeepMind and healthcare in an age of algorithms” paper — coauthored by Cambridge University’s Julia Powles and The Economist’s Hal Hodson — questioned why DeepMind was given permission to process millions of NHS patient records so easily and without patient approval. It concluded that the deal was riddled with “inexcusable” mistakes,
A spokesperson for the Royal Free London sent the following statement to Business Insider:
“The Streams app was built in close collaboration with clinicians to help prevent unnecessary deaths by alerting them to patients in need in a matter of seconds. It is now in use at the Royal Free, and is helping clinicians provide better, faster care to our patients. Nurses report that it is saving them hours each day.
“We took a safety-first approach in testing Streams using real data. This was to check that the app was presenting patient information accurately and safely before being deployed in a live patient setting. Real patient data is routinely used in the NHS to check new systems are working properly before turning them fully live. No responsible hospital would ever deploy a system that hadn’t been thoroughly tested. The NHS remained in full control of all patient data throughout.
“This project, designed to help prevent unnecessary deaths using new technology, is one of the first of its kind in the NHS and there are always lessons we can learn from pioneering work. We take seriously the conclusions of the NDG, and are pleased that they have asked the Department of Health to look closely at the regulatory framework and guidance provided to organisations taking forward this type of innovation, which is essential to the future of the NHS.
“We are proud of the work we have done with DeepMind and will continue to be bold and brave for the benefit of our patients.”
A spokesperson for the National Data Guardian for Health and Care (NDG) told Business Insider:
“Dame Fiona Caldicott, and her panel of advisors have been considering how patient data was shared by the Royal Free London NHS Foundation Trust with DeepMind under the ‘Streams’ project to improve the detection and management of acute kidney failure. In discussions with the ICO about this, the NDG agreed to provide advice on the use of implied consent for direct care as a legal basis for the sharing of data by the Royal Free with DeepMind. While the ICO investigation is ongoing the NDG will provide any further assistance to the ICO as required, but will not be commenting further on the matter at this point.”
Full copies of National Data Guardian’s letter sent to @SkyNews warning of legal basis re: @DeepMindAI + 1.6 million NHS patients’ data. pic.twitter.com/QjLRDGs1xr
— Alexander J. Martin (@lexanderjmartin) May 15, 2017
Business Insider Emails & Alerts
Site highlights each day to your inbox.