This article is more than 1 year old
Eggheads demo how to fool share-trading bots with carefully crafted retweets
Good thing no one's buying or selling stock from Twitter posts, right? Um, right?
For years, Twitter posts and other online messages have been gathered and analyzed by financial algorithms in an effort to anticipate stock market movements. But, it turns out, these smart systems are rather dumb.
Financially minded developers have created programs that ingest the tweets of widely followed public figures, such as Donald Trump and Elon Musk, and conduct trades based on the sentiment of their musings.
Financial professionals tend to rely on more sophisticated software to analyze online posts from a broader set of people to avoid over-reliance on a single source. Their goals, however, remain the same – to make profitable market predictions.
That becomes more complicated when data is afforded too much trust. As six boffins at three US universities and IBM have found, subversive tweets can make Twitter-based stock prediction a losing proposition.
The researchers – Yong Xie and Sanmi Koyejo (University of Illinois Urbana-Champaign), Dakuo Wang (IBM), Pin-Yu Chen (IBM), Jinjun Xiong (State University of New York at Buffalo), Sijia Liu (IBM/Michigan State University) – describe their approach in a paper titled, "A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction."
The researchers argue that investors and machine-learning models increasingly rely on social media to gather real-time information and sentiment to predict stock price movements.
- Rise of the Machines: How computers took over the stock market
- Japan (lightly) regulates high-frequency algorithmic trading
- Robbing-some Caruso and pal lived life of luxury using victims' millions in crypto-Ponzi scam, say prosecutors
- Alleged Ponzi mastermind on the run from FBI hid in lake with sea-scooter, collared after he surfaced half-hour later
To illustrate what's at stake, they point to misinformation posted by a miscreant to an Associated Press Twitter account on April 23, 2013 – "Two Explosions in the White House and Barack Obama is Injured" – which briefly erased $136bn in market value.
But rather than exploring the damage that can be done by malicious tweets distributed through a compromised, widely followed and trusted account, or the market mayhem that can occur when a popular Twitter personality makes a speculative statement about a public company as if it's a fact, the boffins propose a concatenation attack.
An example from the paper of a quote tweet engineered to change a positive trading tip to a negative one in the eyes of Twitter-watching bots
Their technique effectively poisons benign tweets by joining them with manipulative text designed to fool financial analysis models. Manipulating language models with synonyms is a well-known attack in the machine learning world, the researchers say they believe this hasn't been previously explored in the context of finance.
The researchers looked at three particular models: Stocknet, FinGRU, and FinLSTM.
"Our task is inspired and mimics the retweet function on social media, and uses it to feed the adversarial samples into the dataset," the paper explains.
The primary challenge, the researchers say, is to craft new and effective adversarial tweets. They claim they managed to do so by matching the semantics – the analyzed meaning – of the benign tweet and the concatenated malicious tweet in a way that isn't obvious to potential human and machine readers.
For example, the researchers suggest that this authentic tweet from the Twitter account wallstreetbet7821...
$BHP announces the demerger of its noncore assets – details expected to be filled in on Tuesday.
…could be joined with this semantically similar manipulative tweet…
$BHP announces the demerger of its non-core assets – details expected to be exercised in on Tuesday.
Combined into a retweet, the merged Twitter posts would change the targeted tweet analysis model from a positive prediction to a negative prediction. The result potentially would be a bad stock market trade.
The point of piggybacking on a retweet rather than composing a new tweet with slightly altered wording is to affect how the victim AI model interprets the whole. Naturally, the affected model needs to ingest the retweet, which might not happen if the retweeting account isn't followed. But that's more likely if the financial model ingests data by topic (e.g. mentions of a company name) as opposed to by specific followed accounts.
The researchers tested their technique with a simulated portfolio of $10,000 and found it could produce a loss ranging from 23 percent to 32 percent, using a Long-Only, Buy-Hold-Sell strategy over two years.
"The proposed attack method with a retweet with only a single word replacement can cause the investor an additional $3.2K loss to their portfolio after about two years," the paper explains. The code to replicate the boffins' experiment is available on GitHub.
The researchers say the goal of their work is to raise the financial community's awareness of the risks of relying on AI models rather than providing a specific implementation of an attack. ®