How one bad algorithm cost traders $440m

A look at the worst software testing day ever


Knight Capital, a firm that specialises in executing trades for retail brokers, took $440m in cash losses Wednesday due to a faulty test of new trading software. This morning reports were calling it a trading “glitch", which isn’t nearly as accurate as the term I’d use: “f**king disaster".

The broad outline of the story is here and more colourful, bloody details are here.

Briefly, here’s what happened: Knight Capital’s worst day in IT started Wednesday morning with a test run of its new trading software. An old pal of mine who’s following the story closely (and is also deep in both IT and trading) told me that the company set up the software to work with only a few stocks. They also set the buy/sell points well outside where the markets were currently trading to ensure that nothing would actually execute.

But somehow – and this will probably the be the subject of several lawsuits, books, and maybe even a Broadway musical – the software didn’t behave as expected. It went out and did what it was designed to do: execute lots and lots of trades very, very quickly.

Unfortunately, the trading algorithm the program was using was a bit eccentric as well. On every stock exchange, there is a "bid" and an "ask" price. The bid price is what you’d like to pay the holder of the stock if you want to buy their shares. The ask price is what they’ll pay to buy those same shares from you. There’s always a spread between the two prices, with the "ask" being a few cents or more above the "bid". If the stock is thinly traded, then the spread between the ask and the bid is higher than what you’d see for, say, IBM.

Knight Capital’s software went out and bought at the "market", meaning it paid ask price and then sold at the bid price – instantly. Over and over and over again. One of the stocks the program was trading, electric utility Exelon, had a bid/ask spread of 15 cents. Knight Capital was trading blocks of Exelon common stock at a rate as high as 40 trades per second – and taking a 15 cent per share loss on each round-trip transaction. As one observer put it: "Do that 40 times a second, 2,400 times a minute, and you now have a system that’s very efficient at burning money".

As the program continued its ill-fated test run, Knight’s fast buys and sells moved prices up and attracted more action from other trading programs. This only increased the amount of losses resulting from their trades to the point where, at the end of the debacle 45 minutes later, Knight Capital had lost $440m and was teetering on the brink of insolvency.

They may get at least a partial reprieve. The NYSE will reverse trades in six stocks during the time period when the prices were at least 30 per cent outside the normal trading range for the stocks. This will significantly defray much of Knight Capital’s losses for the day, but we don’t know if it’s enough to allow the firm to survive the blow.

We also don’t yet know exactly what happened. I find it hard to believe that the software was so faulty that it could just go into berserker mode and start wildly trading. Seems to me that that’s the type of issue that ISV regression testing would find and correct right away. On the other hand, I don’t think that the Knight Capital IT guys would just fire the program up for testing and not put in solid parameters to ensure that it doesn’t play with real money.

I think we’ll find that the culprit was a combination of ISV software bugs, bad documentation, and human error from Knight Capital. In short, plenty of blame to go around. But apportioning blame will have to wait until the forensics are complete. Then will come the lawsuits, settlements, high-level blue-ribbon commission hearings, and, finally, the insider tell-all books. Put me down for a Kindle version of the books, please. ®

Similar topics


Other stories you might like

  • UK watchdogs ask how they can better regulate algorithms
    We have bad news: you probably can't... but good luck anyway

    UK watchdogs under the banner of the Digital Regulation Cooperation Forum (DRCF) have called for views on the benefits and risks of how sites and apps use algorithms.

    While "algorithm" can be defined as a strict set of rules to be followed by a computer in calculations, the term has become a boogeyman as lawmakers grapple with the revelation that they are involved in every digital service we use today.

    Whether that's which video to watch next on YouTube, which film you might enjoy on Netflix, who turns up in your Twitter feed, search autosuggestions, and what you might like to buy on Amazon – the algorithm governs them all and much more.

    Continue reading
  • Indian government accuses Uber of jacking up prices for loyal customers
    Six ride sharing companies forced into consumer redress scheme

    India has accused ride-sharing companies of over-charging loyal customers who regularly take the same route, and directed six platforms to become part of a scheme that offers third-party grievance handling services.

    The directive to join the scheme was issued during a meeting with officials of India's Department of Consumer Affairs, attended by Ola, Uber, Rapido, Meru Cabs and Jugnoo. The platforms were advised to improve responses to customer concerns and rights and directed to become "convergence partners" in India's National Consumer Helpline. Such partners are required to accept and resolve consumer grievances reported to the Helpline.

    The Department said ride-sharing companies need to sign up for the helpline for reasons including that their algorithms set fares in ways that are not easy to understand – sometimes even charging loyal customers higher rates than first-timers on the same route.

    Continue reading
  • NASA installs a new and improved algorithm to better track near-Earth asteroids
    Nearly 20 year-old software used to protect humanity gets an upgrade

    NASA has upgraded its near-Earth asteroid monitoring algorithm to model hazardous space rocks more accurately after nearly two decades, it announced on Tuesday.

    The new system, dubbed Sentry-II, is more powerful than its predecessor, Sentry. Astronomers working at the space agency's Center for Near Earth Object Studies can now automatically calculate thermal influences that nudge an asteroid’s orbit, potentially sending it hurtling towards our home planet.

    The so-called Yarkovsky effect describes the subtle and gradual change of motion when asteroids are heated by the Sun’s light. When asteroids spin, one side of its surface exposed to the star gets heated. As it continues to rotate, the hot region enters shade and cools down. Infrared energy is radiated outwards; the photons carry momentum and impart a tiny thrust on the asteroid. Over long periods of time, these small kicks can change their paths and knock them out of their original orbit.

    Continue reading

Biting the hand that feeds IT © 1998–2022