Oh no, you're thinking, yet another cookie pop-up. Well, sorry, it's the law. We measure how many people read us, and ensure you see relevant ads, by storing cookies on your device. If you're cool with that, hit “Accept all Cookies”. For more info and to customize your settings, hit “Customize Settings”.

Review and manage your consent

Here's an overview of our use of cookies, similar technologies and how to manage them. You can also change your choices at any time, by hitting the “Your Consent Options” link on the site's footer.

Manage Cookie Preferences
  • These cookies are strictly necessary so that you can navigate the site as normal and use all features. Without these cookies we cannot provide you with the service that you expect.

  • These cookies are used to make advertising messages more relevant to you. They perform functions like preventing the same ad from continuously reappearing, ensuring that ads are properly displayed for advertisers, and in some cases selecting advertisements that are based on your interests.

  • These cookies collect information in aggregate form to help us understand how our websites are being used. They allow us to count visits and traffic sources so that we can measure and improve the performance of our sites. If people say no to these cookies, we do not know how many people have visited and we cannot monitor performance.

See also our Cookie policy and Privacy policy.

This article is more than 1 year old

AI trained to sniff out fake news online may itself be fake news: Bot has mixed results in classifying legit titles

Be careful who you read

The best way to tackle fake news using artificial intelligence is to go straight to the source, according to a new study.

Specifically, smartypants at the Massachusetts Institute of Technology (MIT) in America, Hamad Bin Khalifa University (HBKU) in Qatar, and Sofia University in Bulgaria, trained a classifier to determine how accurate and biased news sources are. This software could, therefore, be used to automatically rank publishers on how factual their content is, letting readers know if they can be trusted.

“If a website has published fake news before, there’s a good chance they’ll do it again,” said Ramy Baly, lead author of the paper and a postdoctoral researcher at MIT, on Thursday. “By automatically scraping data about these sites, the hope is that our system can help figure out which ones are likely to do it in the first place.”

The researchers looked at five different variables – including a sample of articles from a particular publisher, its Wikipedia page if it has one, its Twitter account, the URL of its website, and how much web traffic it receives.

online_troll

AI sucks at stopping online trolls spewing toxic comments

READ MORE

All the different variables are good indicators of how likely, or unlikely, an organisation will publish fake news, the academics claimed. For example, articles peppered with lies tended to contain hyperbolic and emotive language. Netizens are also easily manipulated by clickbait, and are more likely to share dodgy or too-good-to-be-true articles on social media. A previous study showed that people are worse than Twitter bots for spreading fake news.

Reputable sources are also more likely to have Wikipedia pages and verified Twitter accounts, and short URLs. For example, ABCnews.com.co is a fake version of ABCnews.com.

The researchers investigated 1,066 different websites and amassed a total of 94,814 articles to measure factuality and bias. Factuality was split into three categories: Low, Mixed, and High.

Associated Press, a large not-for-profit news agency, scored high for accuracy. Organisations like Russia Insider, known for having a pro-Kremlin stance, was ranked as medium, whereas lesser known niche websites like Patriots Voice, set up by a Republican husband and wife team, was low.

The classifier, based on a Support Vector Machine algorithm, also looks at political biases. Fox News was considered Right-wing, and surprise, surprise, Die Hard Democrat came up as Left.

Overall, the system was about 65 per cent accurate at measuring how factual a news source was and 70 per cent accurate at detecting its political bias.

“It’s interesting to think about new ways to present the news to people,” said Preslav Nakov, co-author of the paper and a senior scientist at HBKU. “Tools like this could help people give a bit more thought to issues and explore other perspectives that they might not have otherwise considered."

The researchers have made the datasets and code public on GitHub. It would be interesting to see how well El Reg measures up against the machine... ®

 

Similar topics

TIP US OFF

Send us news


Other stories you might like