AI-powered IT security seems cool – until you clock miscreants wielding it too

Field both embraced, feared by enterprise

20 Got Tips?

Comment We're hearing more about AI or machine learning being used in security, monitoring, and intrusion-detection systems. But what happens when AI turns bad?

Two interesting themes emerged from separate recent studies: the growth of artificial intelligence coupled with concerns about their potential impact on security.

A survey of 5,000 IT professionals released late last month revealed three major threats techies believe they will face over the next five years: malicious AI attacks in the form of social engineering, computer-manipulated media content, and data poisoning. Just four in 10 pro quizzed believed their organizations understood how to accurately assess the security of artificially intelligent systems.

That was according to the Information Systems Audit and Control Association's (ISACA) second annual Digital Transformation Barometer, which named AI and machine learning among the top three technologies likely to be deployed in the next year.

They were also listed in the top five technologies likely to face resistance.

Interestingly, ISACA highlighted the different perceptions of AI risk between the digitally informed and business leaders who are technically illiterate.

"For AI, having digitally literate leaders correlates to lower perceived risks, which can be key when making the case for deploying technologies," ISACA noted. "33 per cent of companies whose leaders do not possess technological expertise perceive AI to be high-risk, while just 25 per cent of companies with digitally literate leaders perceive AI to be high-risk. Organisations led by digitally literate leaders were almost twice as likely to deploy AI than other organizations (33 per cent compared to 18 per cent)."

When it came to emerging technologies, a decision on whether or not to deploy was found to be largely affected by familiarity. Using AI as an example, 76 per cent of enterprises testing it said that it was worth the risk, with just nine per cent saying it was not. In enterprises that were not testing AI, the confidence in it being worth the risk dropped by a third, while the proportion of respondents who said it is not worth the risk more than doubled.

Rise of the Machines

Are the ISACA members right to be concerned about AI security risks, or does simply understanding a tech make you fear it less?

A paper published earlier this year, titled The New Frontiers of Cybersecurity, backed by the National Natural Science Foundation of China, sided with the former statement.

AI quickly cooks malware that AV software can't spot


It asserted that machine-learning is capable of transforming security by mining information and learning from various types of data – such as spam emails, messages and videos – and then evolving an autonomous detection or defense system. Continuous self-training will continue to promote the performance of AI-powered systems, including their stability, accuracy, efficiency, and scalability. But this also works the other way round.

"AI is pushing the boundaries of the abilities of hackers," the paper noted. "Autonomous hacking machines powered by AI can craft sensitive information and find vulnerabilities in computer systems, thus making it much more difficult to fight hackers. Worse yet, AI is able to learn sensitive information, such as personal preferences, from a vast amount of seemingly insensitive data.

"These facts lead us to believe that hackers weaponized by AI will create more sophisticated and increasingly stealthy automated attacks that will demand effective detection and mitigation techniques."

Knowing AI and not fearing it has its place; understanding it as an tool in the hands of the enemy, however, is also worthwhile. Luckily, so far, miscreants prefer to run relatively simple attacks, usually involving phishing or automated exploitation of known vulnerabilities, than training and developing sophisticated machine-learning cyber-weapons. ®

We'll be examining machine learning, artificial intelligence, and data analytics, and what they mean for you, at Minds Mastering Machines in London, between October 15 and 17. Head to the website for the full agenda and ticket information.

Sign up to our NewsletterGet IT in your inbox daily


Keep Reading

US govt: Julian Assange tried to recruit hacker to steal hush-hush dirt and we should know – the hacker was an informant

WikiLeaker accused of tapping up LulzSec's Sabu as a source

Russian hacker selling how-to vid on exploiting unsupported Magento installations to skim credit card details for $5,000

Nearly 2,000 e-commerce shops pwned over weekend so it's time to migrate

Move along, nothing to see here: Auditors say £100k grant to Hacker House was 'appropriate' handout scrutinised due to boss's friendship with PM

We know what you did last summer: MGM's hotel spinoff lost 10.7m guest records and now they're on hacker forums

What happens in Vegas... gets leaked on the internet

Hey there, want to break into computers like an Iranian hacker crew? IBM finds 40GB of videos that include how-tos

In Brief Plus: BitTorrent CEO puts a $1m bounty on Twitter hackers

Hacker House shoved under UK Parliament's spotlight following Boris Johnson funding allegs

Half of government grant to infosec training biz suspended as MPs demand probe

'Ethical' hackers say: It's just hacker. To be one is no longer a bad thing

Great and good of pentesting chew the fat with El Reg

Remember that competition for non-hoodie hacker pics? Here's their best entries

And we invite you to grab your easel and brush

Researchers unmask Indian 'infosec' firm to reveal hacker-for-hire op that targeted pretty much anyone clients wanted

And we would have gotten away with it if it wasn't for that meddling Citizen Lab

Hacker swipes customer list from controversial face-recog-for-Feds Clearview. Its reaction? 'A part of life'

AI biz tells clients miscreant 'gained unauthorized access', tells us 'servers were never accessed, flaw patched'

Tech Resources

Webcast Slide Deck | Zero trust strategies to zap ransomware peril

Join industry veteran and security pro Mike Wronski of Nutanix as he explains to Tim Phillips about zero trust strategies combined with HCI can improve your security posture, defend against threats, help prevent your business from being the next victim of ransomware.

Cybersecurity Best Practices for Business Resilience Guide

Download Rapid7's Cybersecurity Best Practices for Business Resilience series.

IBM and Nvidia® Solutions Power Insights with the New AI

IBM is well-positioned to help organizations incorporate high-performance solutions for AI into the enterprise landscape.

2020 SANS Enterprise Cloud Incident Response Survey

SANS surveyed 218 enterprises to discover the factors that are affecting their ability to respond to incidents in the cloud.