Software

AI + ML

Boffins build AI that can detect cyber-abuse – and if you don't believe us, YOU CAN *%**#* *&**%* #** OFF

Alternatively, you can try to overpower it with your incredibly amazing sarcasm


Trolls, morons, and bots plaster toxic crap all over Twitter and other antisocial networks. Can machine learning help clean it up?

A team of computer scientists spanning the globe think so. They've built a neural network that can seemingly classify tweets into four different categories: normal, aggressor, spam, and bully – aggressor being a deliberately harmful, derogatory, or offensive tweet; and bully being a belittling or hostile message. The aim is to create a system that can filter out aggressive and bullying tweets, delete spam, and allow normal tweets through. Pretty straight forward.

The boffins admit it's difficult to draw a line between so-called cyber-aggression and cyber-bullying. And the line between normal and aggressive tweets is often blurred: after all, people enjoy ranting about things, from feminism to Brexit to tabs-versus-spaces. Having said that, the goal was to craft a system that can automatically and fairly – and by fairly, we mean consistently – draw a line between each category.

After analyzing more than two million tweets that discussed touchy topics such as Gamergate and gender pay inequality at the BBC, as well as more neutral matters like the NBA, the eggheads selected a sample containing 9,484 tweets, and hand labelled them as normal, aggressor, spam, and bully. Obviously, this means the academics' definition of what is aggressive or bullying forms the basis of the model.

About 80 per cent of these tweets were used to train the recurrent neural network, and the remaining 20 or so per cent was used to test it, according to one of the scientists: Jeremy Blackburn, an assistant computer science professor at Binghamton University in New York. We're told the code could sort the test tweets into the four categories with over 80 per cent accuracy. That is to say, 8 of 10 times, the AI would categorize a tweet as expected by the human boffins.

Their research was published in the journal ACM Transactions on the Web – here's the Arxiv version [PDF]. While it's not revolutionary, it's a good introduction text analysis – and don't forget, it is an academic study.

Cyber-bullying is 'dehumanizing'

The neural network analyses not just the content of a tweet but also the tweeter's profile, and the rate at which they tweeted. All the words are encoded as vectors, and various algorithms were used to determine the overall sentiment or emotion of the message and its sender, and whether or not any curse words were used, plus how many hashtags were included, and the number of followers someone has. All of this information is fed into the network so it can predict the category its human masters would have assigned the tweet.

It’s not easy to pinpoint which features are more indicative of online harassment, Blackburn told El Reg on Monday. “It is not straight forward to describe this because we are differentiating between several categories," he said. "For example, we saw that bullies and aggressors had less time between their tweets than spammers, but similar to spammers, bullies used fewer adjectives than normal users and aggressors.”

Aggressors and bullies were more likely to tweet multiple times and use more hashtags than spammers and normal accounts. Bullies tend to harm others by directing their messages at specific people, whereas aggressors were more likely insult groups of folks. Spammers, on the other hand, are less likely to use abusive language and tend to sell things like smut pics and videos.

“Normal users tend to discuss a variety of topics, such as political and social issues, whereas bully users seem to organize their attacks against important and sensitive issues, such as feminism, religion, and pedophiles, using aggressive and in some cases insulting language. Aggressive users express their negativity on popular topics, such as the ‘brexit’ case, ‘maga’, and the spread of the Zika virus. Spammers typically post inappropriate content in an effort to gain more followers or attract victims to malicious sites of questionable or malicious content,” the team wrote in their paper.

Blackburn said cyberbullying has only recently been taken seriously. “This type of behavior is often dehumanizing, which most rational people would consider a bad thing," he said. "From a more pragmatic point of view, the intensity of this behavior, enabled by the scale of the Web and social media, can and has led to acts of real world violence.

'I radically update my course module almost every year to keep up with the rate of change'

READ MORE

“I think that efforts are being made, but that much more can be done. There are difficult decisions for social media companies to consider. For example, some of the biggest offenders also have large followings on social media; simply silencing these people can have unforeseen consequences.”

The researchers hope to use their model on other platforms, such as YouTube or Facebook, though there are other challenges when using machine learning to tackle hate speech and harassment online.

In previous studies, computer scientists discovered that abusive messages bypassed classifiers if the text contained spelling mistakes. Not all rude comments are hateful, either.

“Understanding sarcasm is difficult, especially because different people and cultures express it in different ways,” noted Blackburn. "We did not examine sarcasm in particular, however our algorithm is designed to be extensible, and we intend to add more linguistic features to it in the future." ®

Send us news
49 Comments

Tech world forms AI Alliance to promote open and responsible AI

Everyone from Linux Foundation to NASA and Intel ... but some big names in AI are MIA

Brits turn off Twitter, although teens and tweens keen on generative AI

Bing grows but Google remains top dog, according to Ofcom report

Creating a single AI-generated image needs as much power as charging your smartphone

PLUS: Microsoft to invest £2.5B in UK datacenters to power AI, and more

Mere minority of orgs put GenAI in production after year of hype

Folks are dipping their toes in without a full commitment

Elon Musk's xAI wants $1B cash infusion in exchange for equity shares

What, spent all your liquid assets on Twitter, Elon?

AWS unveils core-packed Graviton4 and beefier Trainium accelerators for AI

Also hedging its bets with a healthy dose of Nvidia chips too

Exposed Hugging Face API tokens offered full access to Meta's Llama 2

With more than 1,500 tokens exposed, research highlights importance of securing supply chains in AI and ML

The AI everything show continues at AWS: Generate SQL from text, vector search, and more

Invisible watermarks on AI-generated images? Sure. But major tools in the stack matter most

HPE targets enterprises with Nvidia-powered platform for tuning AI

'We feel like enterprises are either going to become AI powered, or they're going to become obsolete'

EU running in circles trying to get AI Act out the door

Bloc risks missing out on first-to-legislate status if timetable slips

UK and US lead international efforts to raise AI security standards

17 countries agree to adopt vision for artificial intelligence security as fears mount over pace of development

Fake views for the win: Text-to-image models learn more efficiently with made-up data

Future neural nets will probably use a mix of AI-generated and real info