Rate of autonomous vehicle safety improvement slowing – research

Plus: Law changes in Germany mean robo taxis might be on their way there soon

Autonomous vehicles in the US are showing a declining rate of improvement in one key safety metric, according to research outfit IDTechEx. Despite this, the industry is making progress and may be nearing early commercialisation of technology such as robotic taxis.

IDTechEx has cast its eye over the 2021 autonomous testing and disengagement data from the California Department of Motor Vehicles (DMV), figures that can be used to gauge how safe autonomous vehicles (AVs) are.

Disengagement figures refer to the number of times an AV operator had to disengage the vehicle's autonomous driving function and take manual control while testing the vehicle.

Miles per disengagement. Source: IDTechEx

According to IDTechEx, the progress seen in the miles per disengagement metric has slowed between 2020 and 2021 for the biggest three firms developing so-called robotaxi vehicles. Prior to this, performance typically more than doubled year-on-year, but the most recent figures show an increase of just 1.5x.

Waymo is singled out for putting in by far the largest test mileage, at 2.3 million miles (3.8 million km) in 2021, a quadrupling its previous effort. However, Waymo's disengagement metrics have fallen from 29,900 miles (∼48,120km) per disengagement to just 8,000 miles (∼12,875km).

IDTechEx notes that Waymo has recently increased its testing within the San Francisco area, and speculates that the more crowded urban environment with more traffic, cyclists and pedestrians has proven a much more difficult challenge for its AV systems compared with the locations where the company had previously been testing.

Perhaps this is one reason why Waymo was recently reported to be suing California's Department of Motor Vehicles in an attempt to keep information about its AV crashes and other operational details a secret.

According to IDTechEx, this change sees Waymo drop to seventh place in the disengagement metrics, to be replaced in third place by China's DiDi, which has increased its performance from approximately 5,000 miles per disengagement in 2020 to about 41,000 miles per disengagement in 2021.

The second place goes to Cruise, which reportedly achieved 41,700 miles per disengagement, 46 per cent up on its 2020 results. Top of the leaderboard is AutoX with 50,100 miles per disengagement, which is apparently a 2.5 times improvement over their 2020 performance.

However, IDTechEx notes that AutoX covered a total testing distance of just 50,100 miles and reported just a single disengagement, so this achievement must be taken on that context. If they had experienced just one more disengagement, their performance would have been cut in half, whereas Waymo and Cruise racked up much more testing miles in total and so another disengagement would have less effect on their score.

IDTechEx analysis of California DMV, Disengagement Report 2019/2020. Source: IDTechEx

Despite this slowing rate of progress, as measured by miles per disengagement, IDTechEx claims that the outlook is optimistic for autonomous vehicles, and predicts that we shall see some early (and very limited) commercialisation of robotaxis this year.

While California and China are the key areas for AV development, the research firm also points to other places where trials are planned. One is Moovit, powered by Mobileye, expected to begin testing in Tel Aviv, Israel and Munich in Germany by the end of this year.

IDTechEx claims that Germany is now a growing hotspot for AV, with recent law changes paving the way for commercialisation of autonomous services on German roads. It predicts that Germany will be the second place after Japan to permit level 3 autonomous vehicles for public use, with an updated Mercedes S-Class set to be Germany's first level 3 vehicle later this year.

The research firm's report "Autonomous Cars, Robotaxis & Sensors 2022–2042" has more details here. ®