Security

Government-favoured child safety app warned it could violate the UK's Investigatory Powers Act with message-scanning tech

Redesigned SafeToNet feature highlights tech law mess


A company repeatedly endorsed by ministers backing the UK's Online Safety Bill was warned by its lawyers that its technology could breach the Investigatory Powers Act's ban on unlawful interception of communications, The Register can reveal.

SafeToNet, a content-scanning startup whose product is aimed at parents and uses AI to monitor messages sent to and from children's online accounts, had to change its product after being warned that a feature developed for the government-approved app would break the law.

SafeToNet was hailed this week by senior politicians as an example of "new tech in the fight against online child abuse," having previously featured in announcements from the Department for Digital, Culture, Media and Sport over the past 12 months.

Chief exec Richard Pursey recounted, during an online seminar at the CogX conference in March this year, how his company's lawyers warned SafeToNet its technology was unlawful.

"I don't think I've ever broken out in such a sweat in all my life," Pursey told the seminar.

I'll never forget our CFO and I, we were called into a meeting with our lawyers – we've got some pretty heavyweight lawyers in London – and we told them about how our technology worked and one of the things we were doing, is we were intercepting incoming messages, without the authority of the person that had sent it in the first place.

Intercepting data without the permission of the sender is a civil offence under section 3 of the Investigatory Powers Act 2016 (aka the Snoopers' Charter). Breaches are investigated and judged by the Investigatory Powers Commissioner.

Pursey continued: "We were doing it for good, you know, it's a social impact, we were doing it to safeguard children, until the lawyers said... 'you realise you could go to prison for doing that'. And so what seemed a pretty obvious thing to do – why wouldn't you be allowed to do that – you know, it just put the fear of God in me."

Such interception may also breach section 1 of the Computer Misuse Act 1990, which criminalises accessing data without authorisation.

Pursey told The Register "development of that particular feature" had stopped when the company was told of the legal compliance problem, adding:

We consider ourselves pioneers of safety tech and were concerned about existing and future data protection legislation (GDPR) and so asked for a legal review of our proposed tech architecture to ensure we would be fully compliant. The review led to a detailed analysis against a range of legislation from the Computer Misuse Act, the Defamation Act, RIPA, Data Protection etc. It became clear that our plans for interception could have been challenged as illegal – albeit there would have been a defence that related to the way spam emails are filtered before the user sees them.

This is a reference to section 3((2)(b) of the Snoopers' Charter, which allows message interception with the "express or implied consent" of a system owner or manager.

SafeToNet's website says today: "For legal reasons, SafeToNet does not analyze incoming messages before a child has read them."

Pursey added that Britain's unique tech law environment throws this problem up regularly, telling us: "We were a very young startup then but it worries me that those that don't have the finance to get professional advice will cut corners and innocently/naively breach laws like [the Computer Misuse Act] etc. We see that all the time, especially with international safety tech providers entering the UK market. They often have no idea these laws exist."

The revelation that a government-approved company's product fell foul of Britain's laws highlights the ongoing campaign to reform the Computer Misuse Act, and may well prompt further reforms of Britain's convoluted surveillance legislation.

Civil servants and government ministers are engaged in a bitter war against social media platforms' moves towards end-to-end encryption (E2EE) for user messages. Law enforcement bodies such as the National Crime Agency claim that wider adoption of E2EE will stop them from detecting paedophiles preying on children through messaging apps. Tech platforms and privacy advocates say E2EE is a vital tool to prevent and deter unlawful surveillance.

Avoiding the E2EE problem by scanning messages on children's devices after delivery seems like it might help preserve adults' internet privacy while allowing police agencies to focus on actual harms instead of indiscriminate platform surveillance.

Part of the government war on social media platforms is the Online Safety Bill, renamed from Online Harms Bill at the last moment perhaps in a crude attempt to disassociate it from mountains of well-informed criticism.

While Britain's current surveillance laws were designed to place GCHQ and other spy agencies above and beyond the criminal law (following former NSA sysadmin Edward Snowden 2013's revelations about the extent of nation-states' spying and public revulsion at unchecked domestic mass surveillance), their authors may not have intended to cause difficulties for people making child safety apps. ®

Send us news
63 Comments

China crisis is a TikToking time bomb

ByteDance with the devil if you dare

Cop warrant orders Ring to cough up footage from inside this guy's home

Don't say you weren't warned

Dems offer ban on Feds using facial, voice recognition

Second time before Congress a charm?

Secret Service, ICE break the law over and over with fake cell tower spying

Investigations 'at risk' from sloppy surveillance uncovered by audit probe

US teases more China tech sanctions, this time to deflate balloon-makers

State Dept already has one target, FBI is identifying sources of floating surveillance platform's components

Australia gives made-in-China CCTV cams the boot

The usual suspects - Hikvision and Dahua - named as a risk to national security, prompting the usual denials

French lawmakers say <i>oui</i> to Olympic video surveillance, but <i>non</i> for faces

Pas le visage, merci

Americans have the right to livestream police traffic stops … probably

Handy thing, that First Amendment. Now, about the Second …

Cedars-Sinai hospital's website shares patient info with Meta, lawsuit claims

Facebook parent could then offer that data to other advertising clients, complaint alleges

Seriously, what's with FBI, DEA vacuuming up people's money transfer records?

Warrantless surveillance branded illegal, said to unfairly target the poor, immigrants, minorities

Happy Lunar New Year: Beijing warns of enhanced surveillance during celebrations

Censors are on the lookout for showering under a waterfall of money, overeating, and more conventional sins

Canadian owes bosses for 'time theft' after work-tracking app sinks tribunal bid

She hoped to score thousands but laptop app had other ideas