Better filters won't cure this: YouTube's kids nightmare

What has been seen?

124 Reg comments Got Tips?

Comment For the "smartest guys in the room", Google often seems to be the last to know what’s going on in its own front room. And something very strange indeed is going on over at YouTube.

The artist James Bridle – who created the witty "self-driving car trap" (spoiler: it's a chalk circle) – has been investigating the outer reaches of kids' content. He describes how channels attracting millions of subscribers are producing very strange things indeed.

These range from the "decidedly off" to the nasty. The BBC has reported on the latter. The most disturbing may not be the obvious but predictable 4chan-derived pirate nasties, but Bridle describes them as "things which aren't overtly disturbing to adults, just incredibly dark and weird". Basement content factories are churning out imbecilic content that looks like this:

"This is not a simple matter of 'won't somebody think of the children' hand-wringing. Obviously this content is inappropriate, obviously there are bad actors out there, obviously some of these videos should be removed. Obviously too this raises questions of fair use, appropriation, free speech and so on. But reports which simply understand the problem through this lens fail to fully grasp the mechanisms being deployed, and thus are incapable of thinking its implications in totality, and responding accordingly."

We find a phenomenon so extensive that it cannot be easily dismissed as the product of a specific culture, or trolls.

For what Bridle is describing, he explains, is a disturbing new kind of "art", mass-produced only because a huge ad-supported platform such as YouTube rewards its creation. The "such as" caveat is redundant here, since no other video platform has scale. It has to be YouTube. Just as no other social network (at least outside China or Russia) has the scale of Facebook, so for fake news and political ads, it has to be Facebook. This is as much a characteristic of the internet in 2017 as fake news or spam.

As Bridle explains, thinking of "better filters" as a solution doesn't even begin to solve the problem.

An explanation of the production side gives us a clue. "Online kids' content is one of the few alternative ways of making money from 3D animation because the aesthetic standards are lower and independent production can profit through scale. It uses existing and easily available content (such as character models and motion-capture libraries) and it can be repeated and revised endlessly and mostly meaninglessly because the algorithms don't discriminate — and neither do the kids."

But the giant indiscriminate advertising engine is what incentivises it.

YouTube's Good Baby Toys channel

YouTube's "Good Baby Toys" channel. Note the number of views.

A sweatshop can produce endless huge amounts of material and doesn't care if the millions of "viewers" are robots or children. ("This is content production in the age of algorithmic discovery – even if you're a human, you have to end up impersonating the machine," Bridle notes.)

"It's not about trolls, but about a kind of violence inherent in the combination of digital systems and capitalist incentives. It's down to that level of the metal. And right now, right here, YouTube and Google are complicit in that system... The system is complicit in the abuse.. The architecture they have built to extract the maximum revenue from online video is being hacked by persons unknown to abuse children, perhaps not even deliberately, but at a massive scale."

We asked Google yesterday about guidelines for acceptable content for children on YouTube – but they have not responded.

Years ago, your reporter attempted to pacify a toddler by showing a clip of In The Night Garden from YouTube, only to find a pre-roll lavish advertisement for the Church of Scientology. Several more children's videos were being used to promote the cult: a use of BBC Worldwide material that the licensee perhaps did not anticipate. But that seems almost innocent now. ®

SUBSCRIBE TO OUR WEEKLY TECH NEWSLETTER


Keep Reading

Escobar lines up COO... for lawsuit: Controversial bendy phone slinger sues exec over 'missing cash, YouTube hijack'

Dust up after sales money vanishes from bank account, it is claimed

Facebook does the right thing for once: Joins Google, LinkedIn, Microsoft, Reddit, Twitter, YouTube to clean out dodgy COVID-19 info

What took you so long, eh?

Forget toilet roll, bandwidth is the new ration: Amazon, YouTube also degrade video in Europe to keep 'net running amid coronavirus crunch

Pair join Netflix to turn it down for what? Connectivity stability

Hypochondriacs – are your eyes all blurry? It's just YouTube trying to cut video-stream quality worldwide amid the coronavirus pandemic

You can still watch in HD, but, please, maybe not?

India crowdsources COVID-19 response – startups told to make YouTube vids to win

Online hackfest designed to help government response, provided entries arrive in 12-point Arial

Instagram, YouTube 'iron man' marketer first to be nabbed by Feds cracking down on fake coronavirus web cures

Videos claiming to have magic savior pills racked up millions of views amid allegedly fraudulent investment scheme

Google, YouTube cough up $170m to make that trifling little thing about slurping kids' info without consent go away

Ad giant gets slap on the wrist, promises not to do it again

Google, YouTube, Twitter tell face-rec upstart Clearview to stop harvesting people's content – that's their job

Updated Tech-for-cops CEO claims First Amendment rights as a legal defense

Biting the hand that feeds IT © 1998–2020