Communication around Heroku security incident dubbed 'train wreck'

Users claim lack of transparency following compromise of Github tokens


Efforts by Salesforce-owned cloud platform Heroku to manage a recent security incident are turning into a bit of a disaster, according to some users.

Heroku has run security incident notifications for 18 days and appears to have upset several of its customers due to a perceived lack of openness and communication.

github

So, what happened with GitHub, Heroku, and those raided private repos?

MORE CONTEXT

The most recent status update from just prior to midnight UTC on 3 May read: "A subset of Heroku customers will receive email notifications directly from Salesforce Incident Alerts (incidentalerts@msg.salesforce.com) regarding our continuous efforts to enhance security."

"We recommend that you reset your user account password," was the best advice the platform's support could give, said one Heroku user on Hacker News. Others harbored some healthy curiosity about what might lie behind the advice.

One customer said they'd invited the Salesforce incident handler to provide a "statement that confirms whether or not config variables and secrets were accessed, or that you're not sure."

According to the post, they received the reply: "We currently have no evidence that Heroku customers' secrets stored in config Var were accessed. If we find any evidence of unauthorized access to customer secrets, we will notify affected customers without undue delay."

Lack of clarity over whether "no evidence" simply meant Heroku did not know further alarmed users.

"Law of No Evidence: Any claim that there is 'no evidence' of something is evidence of bullshit," one user pointed out.

"This is turning into a complete train wreck and a case study on how not to communicate with your customers," another added.

The incident began when the Heroku's GitHub access tokens were compromised.

A statement on 15 April said: "We're actively investigating a report received on April 13, 2022 from GitHub that a subset of Heroku's GitHub private repositories, including some source code, were downloaded by a threat actor on April 9, 2022. We proactively notified our Heroku customers regarding this issue and will continue to provide updates to assist them as the investigation continues."

The news followed a 12 April statement from GitHub Security which said an investigation had found an attacker had abused stolen OAuth user tokens — an open standard for website or application access delegation — issued to Heroku and Travis-CI to download data from several organizations.

By April 27, GitHub said it was sending out its final notifications to impacted customers, and said the attackers used the stolen OAuth tokens issued to Heroku and Travis CI to list user organisations before choosing targets and cloning private repositories.

Its analysis of the attacker's pattern of behaviour suggested they were only listing organizations in order to identify accounts to target for listing and downloading private repositories, GitHub said.

You can read our analysis of the incident here.

The Register has asked Heroku's parent Salesforce to comment. ®

Broader topics


Other stories you might like

  • Verizon: Ransomware sees biggest jump in five years
    We're only here for DBIRs

    The cybersecurity landscape continues to expand and evolve rapidly, fueled in large part by the cat-and-mouse game between miscreants trying to get into corporate IT environments and those hired by enterprises and security vendors to keep them out.

    Despite all that, Verizon's annual security breach report is again showing that there are constants in the field, including that ransomware continues to be a fast-growing threat and that the "human element" still plays a central role in most security breaches, whether it's through social engineering, bad decisions, or similar.

    According to the US carrier's 2022 Data Breach Investigations Report (DBIR) released this week [PDF], ransomware accounted for 25 percent of the observed security incidents that occurred between November 1, 2020, and October 31, 2021, and was present in 70 percent of all malware infections. Ransomware outbreaks increased 13 percent year-over-year, a larger increase than the previous five years combined.

    Continue reading
  • Slack-for-engineers Mattermost on open source and data sovereignty
    Control and access are becoming a hot button for orgs

    Interview "It's our data, it's our intellectual property. Being able to migrate it out those systems is near impossible... It was a real frustration for us."

    These were the words of communication and collaboration platform Mattermost's founder and CTO, Corey Hulen, speaking to The Register about open source, sovereignty and audio bridges.

    "Some of the history of Mattermost is exactly that problem," says Hulen of the issue of closed source software. "We were using proprietary tools – we were not a collaboration platform before, we were a games company before – [and] we were extremely frustrated because we couldn't get our intellectual property out of those systems..."

    Continue reading
  • UK government having hard time complying with its own IR35 tax rules
    This shouldn't come as much of a surprise if you've been reading the headlines at all

    Government departments are guilty of high levels of non-compliance with the UK's off-payroll tax regime, according to a report by MPs.

    Difficulties meeting the IR35 rules, which apply to many IT contractors, in central government reflect poor implementation by Her Majesty's Revenue & Customs (HMRC) and other government bodies, the Public Accounts Committee (PAC) said.

    "Central government is spending hundreds of millions of pounds to cover tax owed for individuals wrongly assessed as self-employed. Government departments and agencies owed, or expected to owe, HMRC £263 million in 2020–21 due to incorrect administration of the rules," the report said.

    Continue reading
  • Internet went offline in Pakistan as protestors marched for ousted prime minister
    Two hour outage 'consistent with an intentional disruption to service' said NetBlocks

    Internet interruption-watcher NetBlocks has reported internet outages across Pakistan on Wednesday, perhaps timed to coincide with large public protests over the ousting of Prime Minister Imran Khan.

    The watchdog organisation asserted that outages started after 5:00PM and lasted for about two hours. NetBlocks referred to them as “consistent with an intentional disruption to service.”

    Continue reading
  • Suspected phishing email crime boss cuffed in Nigeria
    Interpol, cops swoop with intel from cybersecurity bods

    Interpol and cops in Africa have arrested a Nigerian man suspected of running a multi-continent cybercrime ring that specialized in phishing emails targeting businesses.

    His alleged operation was responsible for so-called business email compromise (BEC), a mix of fraud and social engineering in which staff at targeted companies are hoodwinked into, for example, wiring funds to scammers or sending out sensitive information. This can be done by sending messages that impersonate executives or suppliers, with instructions on where to send payments or data, sometimes by breaking into an employee's work email account to do so.

    The 37-year-old's detention is part of a year-long, counter-BEC initiative code-named Operation Delilah that involved international law enforcement, and started with intelligence from cybersecurity companies Group-IB, Palo Alto Networks Unit 42, and Trend Micro.

    Continue reading

Biting the hand that feeds IT © 1998–2022