Uber robo-ride's deadly crash: Self-driving car had emergency braking switched off by design

It's to avoid 'erratic vehicle behavior', says taxi app maker


One of Uber’s self-driving cars killed a pedestrian crossing the road at night partly because its emergency braking systems were turned off, according to the US government's National Transport Safety Board.

The watchdog's four-page preliminary report into the smash, released on Thursday, is a grave reminder that today's autonomous vehicles are helpless at crucial moments, and still require "attentive human operators."

In March, one of Uber’s computer-controlled Volvo XC90 vehicles, while in self-driving mode, hit a 49-year-old woman pushing her bike across a road in Tempe, Arizona. The crash killed her. The human driver was unhurt.

When video footage of the collision was released by police, initial questions were directed at the SUV's LiDAR systems. Why did its all-seeing sensors fail to detect her? The report, however, showed LiDAR did spot her about six seconds before the impact, when the vehicle was driving at 43 MPH. But nothing was done by the machine to stop. Below is a video capturing the moments before the fatal collision.

Youtube Video

Uber’s computer-vision software classified her as an unknown object or as another vehicle, before deciding she was actually a bicycle and couldn't predict the direction she was going to go.

uber_self_driving_crash

View of the self-driving system data playback at about 1.3 seconds before impact, when the system determined an emergency braking maneuver would be needed to mitigate a collision. Yellow bands are shown in meters ahead. Image credit: NTSB.

At about 1.3 seconds before the crash, Uber’s systems realized that the car needed to brake immediately to avoid smashing into her. But it didn’t bother because such emergency braking maneuvers are disabled when the car is in self-driving mode, to "reduce the potential for erratic vehicle behavior," according to Uber.

To make this clear: Volvo ships cars with collision avoidance and automatic emergency braking technology built in, but these safety features are disabled by Uber. And Uber's self-driving vehicle software doesn't do emergency braking of its own. So no automatic emergency braking.

Instead, according to Uber, via the transport board, “the vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator” when it thinks immediate braking is needed, which seems a rather large oversight to us.

uber

Uber says it's changed and is now ever-so ShinyHappy™

READ MORE

To put it more bluntly, this self-driving car cannot really self-drive at all. Humans still need to take over when necessary, and the braking system appears to be so unreliable and jumpy that it doesn't even alert the driver if there's a potential emergency. It's up to the person behind the wheel to be paying attention, and quickly slow the vehicle if something goes wrong. The car certainly isn't going to do anything about it.

Unfortunately, in this accident, the driver was not paying attention to the road, and was looking down moments before the collision. Although the driver managed to turn the steering wheel less than a second before the impact at 39 MPH, in an attempt to swerve out of the way, it was too late.

Test route

The accident is the first death by an autonomous vehicle on an American public road. The car was traveling on a well-known test route, was completing its second lap, and had been under the control of a computer for 19 minutes before the crash.

“The self-driving system relies on an underlying map that establishes speed limits and permissible lanes of travel,” the safety board's report stated.

"The system has two distinct control modes: computer control and manual control. The operator can engage computer control by first enabling, then engaging the system in a sequence similar to activating cruise control. The operator can transition from computer control to manual control by providing input to the steering wheel, brake pedal, accelerator pedal, a disengage button, or a disable button."

The watchdog said it would continue gathering evidence to piece together an accurate account of the car crash, with the hopes of issuing safety recommendations to prevent further crashes.

It is also interested in what the pedestrian was up to before the crash, and is calling any potential witnesses to contact them via email: witness at ntsb dot gov. Toxicology results showed she tested positive for methamphetamine and marijuana.

"Over the course of the last two months, we’ve worked closely with the NTSB,” an Uber spokesperson told The Register.

"As their investigation continues, we’ve initiated our own safety review of our self-driving vehicles program. We’ve also brought on former NTSB chair Christopher Hart to advise us on our overall safety culture, and we look forward to sharing more on the changes we’ll make in the coming weeks."

Uber has ceased testing its self-driving testing operation in Arizona following the death. But the San Francisco-based biz plans to resume testing again once it secures fresh permits and improves safety levels. ®

Broader topics

Narrower topics


Other stories you might like

  • Stolen university credentials up for sale by Russian crooks, FBI warns
    Forget dark-web souks, thousands of these are already being traded on public bazaars

    Russian crooks are selling network credentials and virtual private network access for a "multitude" of US universities and colleges on criminal marketplaces, according to the FBI.

    According to a warning issued on Thursday, these stolen credentials sell for thousands of dollars on both dark web and public internet forums, and could lead to subsequent cyberattacks against individual employees or the schools themselves.

    "The exposure of usernames and passwords can lead to brute force credential stuffing computer network attacks, whereby attackers attempt logins across various internet sites or exploit them for subsequent cyber attacks as criminal actors take advantage of users recycling the same credentials across multiple accounts, internet sites, and services," the Feds' alert [PDF] said.

    Continue reading
  • Big Tech loves talking up privacy – while trying to kill privacy legislation
    Study claims Amazon, Apple, Google, Meta, Microsoft work to derail data rules

    Amazon, Apple, Google, Meta, and Microsoft often support privacy in public statements, but behind the scenes they've been working through some common organizations to weaken or kill privacy legislation in US states.

    That's according to a report this week from news non-profit The Markup, which said the corporations hire lobbyists from the same few groups and law firms to defang or drown state privacy bills.

    The report examined 31 states when state legislatures were considering privacy legislation and identified 445 lobbyists and lobbying firms working on behalf of Amazon, Apple, Google, Meta, and Microsoft, along with industry groups like TechNet and the State Privacy and Security Coalition.

    Continue reading
  • SEC probes Musk for not properly disclosing Twitter stake
    Meanwhile, social network's board rejects resignation of one its directors

    America's financial watchdog is investigating whether Elon Musk adequately disclosed his purchase of Twitter shares last month, just as his bid to take over the social media company hangs in the balance. 

    A letter [PDF] from the SEC addressed to the tech billionaire said he "[did] not appear" to have filed the proper form detailing his 9.2 percent stake in Twitter "required 10 days from the date of acquisition," and asked him to provide more information. Musk's shares made him one of Twitter's largest shareholders. The letter is dated April 4, and was shared this week by the regulator.

    Musk quickly moved to try and buy the whole company outright in a deal initially worth over $44 billion. Musk sold a chunk of his shares in Tesla worth $8.4 billion and bagged another $7.14 billion from investors to help finance the $21 billion he promised to put forward for the deal. The remaining $25.5 billion bill was secured via debt financing by Morgan Stanley, Bank of America, Barclays, and others. But the takeover is not going smoothly.

    Continue reading

Biting the hand that feeds IT © 1998–2022