Throwing money at bug bounties won't beat zero-day dark markets

Study shows tools and bragging rights key to beating criminals

RSA 2015 The first academic study into the market for zero-day flaws has shown some surprising results, not least that throwing money at ever-larger bug bounty payouts might well be counterproductive.

The research – which was carried out by MIT principal research scientist Michael Siegel and Katie Moussouris, chief policy officer of bug bounty organizer HackerOne – traced the dynamics of the market for zero-day flaws by monitoring the activities both of crooks who collect vulnerabilities for attacks and researchers who report them to increase software defences.

Moussouris, who set up Microsoft's first big bounty program when she was at Microsoft in 2013, found that offering researchers money was a highly effective tactic – up to a point. But while bounties were a useful tool, she discovered that the people reporting bugs were motivated not only by money but also by the favorable publicity they could get to support their own businesses.

"Security researchers came to Microsoft of their own free will and voluntarily turned over bugs that could fetch six figures on the offensive market," Moussouris said during a talk discussing the research at the RSA Conference in San Francisco this week. "They were trained by the only incentive they had – which was 10-point Arial font in a bulletin."

By way of example, she cited the the bounty program Microsoft launched for the preview release of Internet Explorer 11. Microsoft paid out $28,000 in bug bounties but it also promoted a lot of security researchers. As a result, legitimate researchers gained a lot more business and were willing to hand over vulnerabilities that would have been highly valuable in pure cash terms if they had been sold to nefarious players.

The effect of the IE11 bug bounty program on so-called dark markets was remarkable, Moussouris said. Trading stopped dead while traders tried to work out what it would do for prices. After all, the more bugs were found by legitimate researchers, the fewer would be left to sell to criminal hackers.

But while the assumption that more researchers scanning your code for flaws reduces the amount of stuff on sale in dark markets seems logical enough, the research showed this was true only true up to a point. In practice, the effect was limited to early releases of software that have plenty of low-hanging fruit, rather than more established code that has already been hardened.

Another assumption is that increasing the value of bounties will result in more bugs being found and fixed. That's also true up to a point, but Moussouris said that in the long term such a tactic could have the opposite effect from the intended one.

"It was suggested that someone in the governments should buy up vulnerabilities at ten times the current market rate," she said. "How many vulnerability researchers would stay in their day job."

The working lifetime of the typical vulnerability researcher is already short enough. Barring a few outliers, most researchers typically only do the job for three or four years, the study found.

This is in part down to the software itself. In the past, new Microsoft operating systems have typically been released every three years or so, and now Redmond is increasing its release cadence. When each new OS comes out, researchers have to learn new skills or else move on to other areas of research.

Moussouris' study found, however, that if companies offer legitimate researchers new tools to carry on their research, as well as a certain financial stipend, then the researchers are both more effective at finding bugs and more likely to continue doing so.

"You cannot outbid the dark market," she said. "Instead, you need to create more interesting incentives." ®