Government-favoured child safety app warned it could violate the UK's Investigatory Powers Act with message-scanning tech
Redesigned SafeToNet feature highlights tech law mess
A company repeatedly endorsed by ministers backing the UK's Online Safety Bill was warned by its lawyers that its technology could breach the Investigatory Powers Act's ban on unlawful interception of communications, The Register can reveal.
SafeToNet, a content-scanning startup whose product is aimed at parents and uses AI to monitor messages sent to and from children's online accounts, had to change its product after being warned that a feature developed for the government-approved app would break the law.
SafeToNet was hailed this week by senior politicians as an example of "new tech in the fight against online child abuse," having previously featured in announcements from the Department for Digital, Culture, Media and Sport over the past 12 months.
Chief exec Richard Pursey recounted, during an online seminar at the CogX conference in March this year, how his company's lawyers warned SafeToNet its technology was unlawful.
"I don't think I've ever broken out in such a sweat in all my life," Pursey told the seminar.
I'll never forget our CFO and I, we were called into a meeting with our lawyers – we've got some pretty heavyweight lawyers in London – and we told them about how our technology worked and one of the things we were doing, is we were intercepting incoming messages, without the authority of the person that had sent it in the first place.
Intercepting data without the permission of the sender is a civil offence under section 3 of the Investigatory Powers Act 2016 (aka the Snoopers' Charter). Breaches are investigated and judged by the Investigatory Powers Commissioner.
Pursey continued: "We were doing it for good, you know, it's a social impact, we were doing it to safeguard children, until the lawyers said... 'you realise you could go to prison for doing that'. And so what seemed a pretty obvious thing to do – why wouldn't you be allowed to do that – you know, it just put the fear of God in me."
Such interception may also breach section 1 of the Computer Misuse Act 1990, which criminalises accessing data without authorisation.
Pursey told The Register "development of that particular feature" had stopped when the company was told of the legal compliance problem, adding:
We consider ourselves pioneers of safety tech and were concerned about existing and future data protection legislation (GDPR) and so asked for a legal review of our proposed tech architecture to ensure we would be fully compliant. The review led to a detailed analysis against a range of legislation from the Computer Misuse Act, the Defamation Act, RIPA, Data Protection etc. It became clear that our plans for interception could have been challenged as illegal – albeit there would have been a defence that related to the way spam emails are filtered before the user sees them.
This is a reference to section 3((2)(b) of the Snoopers' Charter, which allows message interception with the "express or implied consent" of a system owner or manager.
- MI5 still risks breaking the law on surveillance data through poor controls – years after it was first warned
- Snowden was right, rules human rights court as it declares UK spy laws broke ECHR
- UK terror law reviewer calls for expanded police powers to imprison people who refuse to hand over passwords
- Thou shalt not hack indiscriminately, High Court of England tells Britain's spy agencies
SafeToNet's website says today: "For legal reasons, SafeToNet does not analyze incoming messages before a child has read them."
Pursey added that Britain's unique tech law environment throws this problem up regularly, telling us: "We were a very young startup then but it worries me that those that don't have the finance to get professional advice will cut corners and innocently/naively breach laws like [the Computer Misuse Act] etc. We see that all the time, especially with international safety tech providers entering the UK market. They often have no idea these laws exist."
The revelation that a government-approved company's product fell foul of Britain's laws highlights the ongoing campaign to reform the Computer Misuse Act, and may well prompt further reforms of Britain's convoluted surveillance legislation.
Civil servants and government ministers are engaged in a bitter war against social media platforms' moves towards end-to-end encryption (E2EE) for user messages. Law enforcement bodies such as the National Crime Agency claim that wider adoption of E2EE will stop them from detecting paedophiles preying on children through messaging apps. Tech platforms and privacy advocates say E2EE is a vital tool to prevent and deter unlawful surveillance.
Avoiding the E2EE problem by scanning messages on children's devices after delivery seems like it might help preserve adults' internet privacy while allowing police agencies to focus on actual harms instead of indiscriminate platform surveillance.
Part of the government war on social media platforms is the Online Safety Bill, renamed from Online Harms Bill at the last moment perhaps in a crude attempt to disassociate it from mountains of well-informed criticism.
While Britain's current surveillance laws were designed to place GCHQ and other spy agencies above and beyond the criminal law (following former NSA sysadmin Edward Snowden 2013's revelations about the extent of nation-states' spying and public revulsion at unchecked domestic mass surveillance), their authors may not have intended to cause difficulties for people making child safety apps. ®