The UK's public authorities slurped up more than 750,000 items of communications data during 2016, with more than 1,000 reported errors – of which 29 were deemed serious.
The figures were revealed today in the Interception of Communications Commissioner's Office (IOCCO) annual report.
Although Commissioner Stanley Burnton said the standard of compliance was generally high, he raised particular concern about incorrect IP address resolution, which is "far more common than is acceptable".
Overall, the report sets out the state of comms data slurping across the UK, showing that public authorities acquired 754,599 items of communication – for instance, 30 days of incoming and outgoing call data from a phone – this year.
The nation's three spy agencies snaffled up more than 45,000 items of data, but the biggest haul was gathered by the police.
The most slurp-happy force was the Met – its communications intelligence unit alone pulled in 103,602 items. That was followed by the West Midlands (55,250), Police Scotland (44,158) and Greater Manchester Police (40,857).
There needs to be a change of mindset away from the assumption that technical intelligence, such as an IP address resolution, is always correct
The IOCCO's report revealed some 1,101 errors in the collection of communications data, which might include authorities getting the wrong info, during 2016.
Most of these – some 43.5 per cent – were the fault of the single point of contact (the people responsible for helping authorities lawfully get their hands on comms data), but this could be for a number of reasons.
The single biggest cause of error (31.8 per cent) was applicants submitting the wrong communications addresses.
Of the 1,101 errors, 29 were defined as serious. In seven cases, someone was wrongfully arrested or a warrant was wrongfully granted, while in nine cases someone unconnected to the investigation was "visited" by police. In five cases agencies got the wrong or too much data, and a further five caused a welfare check on a vulnerable person to be delayed.
'Make sure authorities can cut and paste'
Twenty of the 29 were human errors, and the commissioner said that many of the most serious cock-ups were caused by mistakes in the resolution of IP addresses.
"These have resulted in the wrong people being arrested for extremely serious crimes," Burnton wrote.
"People have been arrested for crimes relating to child sexual exploitation. Their children have been taken into care, and they have had to tell their employers."
Essentially, authorities want IP addresses to link to the specific online activity on a specific device either owned by an individual or at a specific location.
But things aren't so simple, as Burnton pointed out, because service providers can reassign IP addresses between active customers or route multiple users through the same address.
"All of this means that turning an IP address into a specific location is increasingly complex," he said, and relies on the authorities providing a precise time for the online activity. But massive variations in how time is recorded in date stamps makes this tough.
This "greatly increases the risk of error", he said – especially when so much of the data processing is done manually.
Indeed, most errors are the result of someone mistyping a number. That's because, while some applicants the commish worked with have dual-screen terminals with access to all systems, and others use USB sticks to transfer data, some are literally retyping addresses into their applications.
How to fix the fat-fingered problem? Well – and it seems almost incomprehensible that the body had to dedicate an entire chapter to pointing this out – give people the power of CTRL+C/CTRL+V.
"Make it easier for applicants to be able to electronically transfer (i.e. copy/paste) communications addresses and timestamps into their applications," the report said.
Other calls are for more caution and double-checks, before sending the troops in all guns blazing.
Burnton acknowledged that some of the biggest, most harmful errors come as a result of overzealous authorities wanting to protect children that may be at risk of sexual exploitation, but being too quick to hand out arrest or search warrants.
If authorities link an address to a place where children are present, they don't always do the "usual investigative work" to corroborate it before taking executive action, Burnton said – but he added this can't be an excuse.
"There needs to be a change of mindset away from the assumption that technical intelligence, such as an IP address resolution, is always correct."
In particular, Burnton expressed his admiration for Nigel Lang, who was arrested in such circumstances and has made his ordeal public.
"On confirmation of the error, all the power of the state, which comes into force to protect children, needs to be turned around and switched off," he said.
"I would have no hesitation in using my powers of notification to enable victims to make applications to the Investigatory Powers Tribunal."
The report also revealed that some 3,007 interception warrants were doled out to agencies during 2016, and that the security service made 19,995 applications to access communications data, related to 97,382 items. ®
This year's annual report is the last, as the IOCCO has been replaced by the Investigatory Powers Commissioner's Office, which started work in September. ®