A system crash and subsequent bungling of a digital certificate caused California to fall behind in reporting the results of around 300,000 COVID-19 coronavirus tests last week.
Governor Gavin Newsom said on Monday that after a weekend-long slog, officials had managed to get through hundreds of thousands tests that, thanks to a pair of IT screw-ups, had not been counted up and added to the Golden State's totals in a timely manner.
This may well explain the abrupt resignation on Sunday of the US state's public health director Dr Sonia Angell after less than a year on the job.
To be clear these aren't delays that kept individual doctors and patients from getting their virus results – that's still an issue albeit a separate one – rather it's the test result figures that labs give to Cali's public health office, so that the number of positive and negative results can be added up, and the amount of testing across the state monitored. Those numbers are also handed to county governments, who can break them down by race, age, and occupation to map out virus hotspots.
In other words, the computer blunders slowed the totting up of coronavirus test stats, both positive and negative, in California.
According to Health and Human Services secretary Dr Mark Ghaly, the backlog began on July 25, when one of the servers collecting testing records crashed and a temporary workaround was used to keep the numbers flowing. Specifically, the California Reportable Disease Information Exchange, aka CalREDIE, was unable to accept case data. The temporary fix remained in place, rather than the system being properly fixed, which led to further errors and holdups.
"These changes were supposed to be temporary but they were not disabled," said Ghaly, who noted that the systems being used to handle the reports are not purpose-build for this pandemic, but were designed prior for everyday use.
Internet use up 40 per cent in San Francisco Bay Area – but you know what’s even higher? Yep, alcohol, weed useREAD MORE
On top of the wonky server fix, the health department found itself with a second forehead-slapper when, on July 31, a digital certificate on a system that, from what we can tell, receives statistics from a major medical lab, said to be Quest Diagnostics, was not renewed by the state. It thus expired, and sat invalid until August 4. This further contributed to the backlog of unprocessed records: data couldn't be received until the cert was renewed.
As a result, the state under-reported the number of tests, both positive and negative, in California, and Gov Newsom was working with bad info when he earlier boasted a 20 per cent decline in positive results.
On Monday, Newsom said that fortunately, these 300,000 new reports did not skew the overall data, and the state as a whole continues to see a good, downward trend in new infections. Individual counties, however, will have to wait longer – around 34 to 72 hours – for data they can process and use to inform their decisions on reopening towns and cities.
As for the borked systems, the state plans to replace them with new hardware and software intended specifically to handle COVID-19 caseloads. More immediately, both the server mitigation and expired cert responsible for the backlog have been replaced.
At least one official, Dr Angell, has also resigned.
"This was an IT issue in many respects," said Newsom, who refused to comment specifically on the reasons for Angell's resignation, "there were some personal judgement issues as well." ®