What a Hancock-up: Excel spreadsheet blunder blamed after England under-reports 16,000 COVID-19 cases
All amid a second wave of the bio-nasty
Updated As the UK heads into a troubling second wave of coronavirus cases, those in contact with thousands of people who just tested positive for COVID-19 in England went about their lives for up to a week unaware they had rubbed shoulders with a carrier.
It's estimated as many as 48,000 people were not informed they had come in close contact with people who had tested positive, and by this morning, half of them still hadn't been warned or told to self-isolate.
At the heart of this mess is the under-reporting of 15,841 novel coronavirus cases between 25 September and 2 October, as revealed on Sunday by Michael Brodie, interim chief executive of Public Health England (PHE), who said:
A technical issue was identified overnight on Friday 2 October in the data load process that transfers COVID-19 positive lab results into reporting dashboards. After rapid investigation, we have identified that 15,841 cases between 25 September and 2 October were not included in the reported daily COVID-19 cases. The majority of these cases occurred in most recent days.
This latest cock-up in England's NHS coronavirus contact-tracing system is why the government blamed "computer issues" for cases seemingly doubling overnight, leading to widespread concern among disease modelers about what this meant for the spread of the virus. Just yesterday a record number of new cases were logged in the UK: 7,982.
The under-reporting was widely reported to be down to the use of Microsoft's Excel spreadsheet program in transferring test results from labs to the health service to total up. It was suggested PHE used columns to log each case and that the maximum number had been reached – Excel has a 16,384 column limit – leading many to suggest it should instead have used rows for entries as Excel's row limit exceeds one million. Or that it should have used, you know, database software.
Then a more plausible explanation emerged: test results were automatically fetched in CSV format by PHE from various commercial testing labs, and stored in rows in an older .XLS Excel format that limited the number of rows to 65,536 per spreadsheet, rather than the one-million row limit offered by the modern .XLSX file format. Each test result, according to the BBC, took up several rows, so the real case limit was about 1,400 per sheet, and after that cutoff point, records were simply left off and not counted when imported.
That led to results going missing, stats not being totaled up correctly, and contact-tracing alerts not going out. For now, we're told, the solution, for want of a better word, is to break the data into two or more spreadsheets.
A Reg source confirmed widespread use of the spreadsheet software as "human middleware" in the sector, scathingly describing it as the "default for all tech in all of the NHS and related quangos and other bodies... to bridge all the gaps that the 'proper' tech hasn't been designed to cope with."
Wags on Twitter noted, among other things, that the testing catastrophe was "the most shambolic thing to happen to a Microsoft program since Kelly Rowland sent Nelly a text on Excel in the Dilemma video and got mad that he didn't reply."
Dido 'Queen of Carnage' Harding to lead UK's Institute for Health Protection because Test and Trace went so wellREAD MORE
The Reg asked PHE for comment and confirmation that Excel was behind the cock-up, and was told to watch out for a ministerial statement from the Secretary of State for Health, Matthew Hancock.
At the time of publication, Hancock was reassuring the House of Commons that the "data upload" issue didn't "directly affect outbreaks in care homes and schools", going over some of the points included in last night's missive including that the patients themselves knew of their results, which is completely besides the point. He did not mention Excel. The link is here if you're interested in what he had to say.
PHE is an executive agency of the Department for Health and Social Care and therefore under direct ministerial control.
The "issue did not affect people receiving their COVID-19 test results and all people who tested positive received their COVID-19 test result in the normal way," the agency said last night, but clearly something else did not take place – proper tracing of all those positive patients' unwitting contacts, leaving thousands of Brits none the wiser.
The following paragraph of the statement rapidly made this clear: "All outstanding cases were immediately transferred to the contact tracing system by 1am on 3 October and a thorough public health risk assessment was undertaken to ensure outstanding cases were prioritised for contact tracing effectively."
This comes just days after multiple users were erroneously told to "self-isolate" during a "test" of the much-delayed contact tracing app for England and Wales, NHS COVID-19, which was meant to launch on 1 June but instead launched on 24 September (Scotland and Northern Ireland already had their own COVID-19 awareness apps by this point).
Just two months back, Baroness Dido Harding, Queen of Carnage head of the UK's COVID-19 Test and Trace organisation and former CEO of hacker-magnet ISP TalkTalk, was announced as head of the new National Institute for Health Protection, the agency that replaces Public Health England.
The test-and-trace programme in England was outsourced to Serco for £45m in June. The outsourcer is not responsible for the design or overall management of NHS Test and Trace, nor its IT systems. Nonetheless the outsourcer managed to leak the email addresses of around 300 human contact tracers, the people paid to find out who an infected person was in touch with, back in May.
The government's response to track and trace also came under fire last week over privacy issues, with, as El Reg exclusively revealed, the UK Information Commissioner's Office investigating a mass email-and-text blast.
If you know something no one else does about this issue, contact us here. ®
Updated to add
The UK government has pretty much confirmed the data went AWOL after imported test results hit the row limit in Excel's .XLS format, though in a statement officials characterized it as a "file size" issue:
The technical issue was caused by the fact that some files containing positive test results exceeded the maximum file size that takes these data files and loads then into central systems. A rapid mitigation has been put in place that splits large files and a full end to end review of all systems has also been instigated to mitigate the risk of this happening again. There are already a number of automated and manual checks that happen throughout.
The solution, as you can see, is to split the spreadsheets over multiple files.