What's 2 + 2? Personal info, sniffs Twitter: Anti-doxxing AI goes off the rails, bans tweets with numbers in them

Innocuous images deemed a threat, netizens silenced until twitterings terminated


Final update Netizens are being locked out of their Twitter accounts for tweeting innocuous posts and images, such as math equations, that trigger the social network's system that prevents the sharing of private personal information.

Revealing personally identifiable non-public information, such as someone’s home address or cellphone number, also known as doxxing, is against the site’s rules. Doxxing is often used to harass people, as it invites strangers to stalk them, for instance.

“You may not publish or post other people's private information without their express authorization and permission,” states Twitter’s policy. “We also prohibit threatening to expose private information or incentivizing others to do so.”

Some users, however, have been wrongly punished for breaking the rules after they shared what appear to be ordinary tweets and pictures that contained no personal info. A Reg reader alerted us to the problem after he was barred from using two of his accounts when he tweeted an image of the front page of The Evening Standard, a British newspaper, and, again, when he posted an screenshot of a Wikipedia article.

It didn’t give any indication of what the private information was determined to be

Louis Maddox, a programmer based in London, showed The Register both of the offending images and neither of them contain any private personal information about anyone. Nevertheless, Twitter asked him to delete the tweets or appeal its request.

Maddox opted to appeal both times, and was subsequently temporarily barred from using Twitter. Although his account is visible online, he cannot tweet or view his timeline at the moment whilst Twitter processes both of his appeals. He reckons there might be a bug in the automated methods Twitter uses to analyze images, which we presume involves some kind of artificial intelligence due to the speed at which it works – faster than a human moderator – and it involves computer vision.

“It didn’t give any indication of what the private information was determined to be,” Maddox told El Reg.

"I have no idea how this works on the backend other than that Twitter feeds all images it receives through neural networks to do things like automatically cropping an image. Since the tweet gets flagged immediately, there is no chance it was due to human intervention from a moderator or a bad actor trolling through the report feature, then it must be down to automated image recognition."

Another Twitter user, who goes by the handle AltentX, also discovered that accounts were being automatically, and incorrectly, flagged for doxxing when they themselves and others posted images on the social media platform.

“The images I uploaded were in Indonesian, and the first image just reads, 'oops sorry, the website is under construction, please try again later.' Another image is an algebra problem,” AltentX told us. “I don't think Twitter’s AI models detect the meaning of words.”

What’s common with all the offending images is that they involve a white box against a dark background. “The key is a centered white box, and a dark color around the box. Then your account will be locked,” AltentX added.

Numerous tweeters have been thrown in the platform's jail for supposedly sharing private information on Twitter, requiring them to delete their posts to continue using the site, even if their posts contained no personal info.

A spokesperson for Twitter was not available to comment. ®

Updated to add

"This shouldn’t be happening and is not expected behavior," a Twitter spokesperson told The Register. "We’re looking into this issue and how we can fix it. We'll keep you apprised as we learn more."

We understand Maddox has now had his Twitter accounts restored.

Final update

A spokesperson for Twitter tells us the glitch has been fixed: "This issue has been resolved, but we’ll continue to keep a close eye on it. Thanks again for raising it.”

Similar topics


Other stories you might like

  • Battlefield 2042: Please don't be the death knell of the franchise, please don't be the death knell of the franchise

    Another terrible launch, but DICE is already working on improvements

    The RPG Greetings, traveller, and welcome back to The Register Plays Games, our monthly gaming column. Since the last edition on New World, we hit level cap and the "endgame". Around this time, item duping exploits became rife and every attempt Amazon Games made to fix it just broke something else. The post-level 60 "watermark" system for gear drops is also infuriating and tedious, but not something we were able to address in the column. So bear these things in mind if you were ever tempted. On that note, it's time to look at another newly released shit show – Battlefield 2042.

    I wanted to love Battlefield 2042, I really did. After the bum note of the first-person shooter (FPS) franchise's return to Second World War theatres with Battlefield V (2018), I stupidly assumed the next entry from EA-owned Swedish developer DICE would be a return to form. I was wrong.

    The multiplayer military FPS market is dominated by two forces: Activision's Call of Duty (COD) series and EA's Battlefield. Fans of each franchise are loyal to the point of zealotry with little crossover between player bases. Here's where I stand: COD jumped the shark with Modern Warfare 2 in 2009. It's flip-flopped from WW2 to present-day combat and back again, tried sci-fi, and even the Battle Royale trend with the free-to-play Call of Duty: Warzone (2020), which has been thoroughly ruined by hackers and developer inaction.

    Continue reading
  • American diplomats' iPhones reportedly compromised by NSO Group intrusion software

    Reuters claims nine State Department employees outside the US had their devices hacked

    The Apple iPhones of at least nine US State Department officials were compromised by an unidentified entity using NSO Group's Pegasus spyware, according to a report published Friday by Reuters.

    NSO Group in an email to The Register said it has blocked an unnamed customers' access to its system upon receiving an inquiry about the incident but has yet to confirm whether its software was involved.

    "Once the inquiry was received, and before any investigation under our compliance policy, we have decided to immediately terminate relevant customers’ access to the system, due to the severity of the allegations," an NSO spokesperson told The Register in an email. "To this point, we haven’t received any information nor the phone numbers, nor any indication that NSO’s tools were used in this case."

    Continue reading
  • Utility biz Delta-Montrose Electric Association loses billing capability and two decades of records after cyber attack

    All together now - R, A, N, S, O...

    A US utility company based in Colorado was hit by a ransomware attack in November that wiped out two decades' worth of records and knocked out billing systems that won't be restored until next week at the earliest.

    The attack was detailed by the Delta-Montrose Electric Association (DMEA) in a post on its website explaining that current customers won't be penalised for being unable to pay their bills because of the incident.

    "We are a victim of a malicious cyber security attack. In the middle of an investigation, that is as far as I’m willing to go," DMEA chief exec Alyssa Clemsen Roberts told a public board meeting, as reported by a local paper.

    Continue reading

Biting the hand that feeds IT © 1998–2021