Twitter uses HackerOne bounties to find biases in its image-cropping AI model

Claims it's the first algorithmic bias bounty competition


Twitter's saliency algorithm – otherwise known as its automated image cropping tool – has a problem with gender and race bias. The micro-blogging service is hoping to fix it by offering what it reckons is the industry's first algorithmic bias bounty competition.

The saliency algorithm employed by Twitter uses machine learning to crop images around the first spot eyes most frequently fall. In fall of 2020, some users complained the image cropping favoured light skin over dark, and women's legs and breasts over their faces.

Twitter promised to investigate, decided the machine learning code it employed was not really ready to have the keys to the castle, and gave the image-cropping controls back to humans. Employees published an academic paper describing the problem, and the company made its code public through GitHub.

Now the company has re-shared the saliency model, and its code, and asked for help in its improvement.

Twitter META Director Rumman Chowdhurty and Product Manager Jutta Williams wrote on the company blog:

We want to take this work a step further by inviting and incentivizing the community to help identify potential harms of this algorithm beyond what we identified ourselves.

The competition is available through HackerOne and is a part of DEF CON AI Village with five cash prizes ranging from US$500 to US$3500 up for grabs. The bounty program is open until August 6 and the winners will be announced on August 8 at a Twitter DEF CON workshop.

"Your mission is to demonstrate what potential harms such an algorithm may introduce," states Twitter's HackerOne post.

To score a bounty, participants must find harms that come from the process of cropping or displaying images and videos. Extra points will be awarded to participants who detail harms falling on marginalized communities. The harms can be either malicious or unintentional – although unintentional harms seem to gain more points, as do situations that affect more users or are more severely damaging to a person's well-being.

Users employing denial of service, model inversion and Black-box model extraction or copying attacks will be disqualified, as will those that lead to exploitable behaviours like remote code extraction.

It's an interesting problem for Twitter to be outsourcing, and if successful will have implications reaching into facial recognition technology and related AI problems. As one professional working in digital security puts it, it requires reverse-engineering human biases.

Twitter has, of course, outsourced assessments of its biases before – albeit in a very ad hoc fashion as millions of users daily damn or praise the tweets it serves. The company has also frequently been accused of bias against politically conservative voices. Opening itself to a formal assessment of algorithmic bias will probably do little to silence its critics. ®

Similar topics

Broader topics


Other stories you might like

  • Verizon: Ransomware sees biggest jump in five years
    We're only here for DBIRs

    The cybersecurity landscape continues to expand and evolve rapidly, fueled in large part by the cat-and-mouse game between miscreants trying to get into corporate IT environments and those hired by enterprises and security vendors to keep them out.

    Despite all that, Verizon's annual security breach report is again showing that there are constants in the field, including that ransomware continues to be a fast-growing threat and that the "human element" still plays a central role in most security breaches, whether it's through social engineering, bad decisions, or similar.

    According to the US carrier's 2022 Data Breach Investigations Report (DBIR) released this week [PDF], ransomware accounted for 25 percent of the observed security incidents that occurred between November 1, 2020, and October 31, 2021, and was present in 70 percent of all malware infections. Ransomware outbreaks increased 13 percent year-over-year, a larger increase than the previous five years combined.

    Continue reading
  • Slack-for-engineers Mattermost on open source and data sovereignty
    Control and access are becoming a hot button for orgs

    Interview "It's our data, it's our intellectual property. Being able to migrate it out those systems is near impossible... It was a real frustration for us."

    These were the words of communication and collaboration platform Mattermost's founder and CTO, Corey Hulen, speaking to The Register about open source, sovereignty and audio bridges.

    "Some of the history of Mattermost is exactly that problem," says Hulen of the issue of closed source software. "We were using proprietary tools – we were not a collaboration platform before, we were a games company before – [and] we were extremely frustrated because we couldn't get our intellectual property out of those systems..."

    Continue reading
  • UK government having hard time complying with its own IR35 tax rules
    This shouldn't come as much of a surprise if you've been reading the headlines at all

    Government departments are guilty of high levels of non-compliance with the UK's off-payroll tax regime, according to a report by MPs.

    Difficulties meeting the IR35 rules, which apply to many IT contractors, in central government reflect poor implementation by Her Majesty's Revenue & Customs (HMRC) and other government bodies, the Public Accounts Committee (PAC) said.

    "Central government is spending hundreds of millions of pounds to cover tax owed for individuals wrongly assessed as self-employed. Government departments and agencies owed, or expected to owe, HMRC £263 million in 2020–21 due to incorrect administration of the rules," the report said.

    Continue reading
  • Internet went offline in Pakistan as protestors marched for ousted prime minister
    Two hour outage 'consistent with an intentional disruption to service' said NetBlocks

    Internet interruption-watcher NetBlocks has reported internet outages across Pakistan on Wednesday, perhaps timed to coincide with large public protests over the ousting of Prime Minister Imran Khan.

    The watchdog organisation asserted that outages started after 5:00PM and lasted for about two hours. NetBlocks referred to them as “consistent with an intentional disruption to service.”

    Continue reading
  • Suspected phishing email crime boss cuffed in Nigeria
    Interpol, cops swoop with intel from cybersecurity bods

    Interpol and cops in Africa have arrested a Nigerian man suspected of running a multi-continent cybercrime ring that specialized in phishing emails targeting businesses.

    His alleged operation was responsible for so-called business email compromise (BEC), a mix of fraud and social engineering in which staff at targeted companies are hoodwinked into, for example, wiring funds to scammers or sending out sensitive information. This can be done by sending messages that impersonate executives or suppliers, with instructions on where to send payments or data, sometimes by breaking into an employee's work email account to do so.

    The 37-year-old's detention is part of a year-long, counter-BEC initiative code-named Operation Delilah that involved international law enforcement, and started with intelligence from cybersecurity companies Group-IB, Palo Alto Networks Unit 42, and Trend Micro.

    Continue reading

Biting the hand that feeds IT © 1998–2022