Ubuntu Linux 18.04 systemd security patch breaks DNS in Microsoft Azure

Snafu disrupts VMs as cloud giant offers workarounds

Microsoft Azure customers running Canonical's Ubuntu 18.04 (aka Bionic Beaver) in the cloud have seen their applications fail after a flawed security update to systemd broke DNS queries.

The situation is as odd as it sounds: if you're running Ubuntu 18.04 in an Azure virtual machine, and you installed the systemd 237-3ubuntu10.54 security update, you've probably found yourself unable to use DNS within the VM, which causes applications and other software relying on domain-name look-ups to stop working properly.

"Starting at approximately 06:00 UTC on 30 Aug 2022, a number of customers running Ubuntu 18.04 (bionic) VMs recently upgraded to systemd version 237-3ubuntu10.54 reported experiencing DNS errors when trying to access their resources," an update to the Microsoft Azure status page said on Tuesday.

While the issue is confined to this one version of Ubuntu Linux, it has nonetheless proved to be a showstopper for those affected.

An individual posting to an Ubuntu forum under the name Luciano Santos de Silva wrote, "Hey guys, nothing is working. My application has been out since this early morning. We have already tried to restart the nodes, restart the VM, but nothing has been working and we don't have any update from Microsoft. Four hours ago they said 'More information will be provided within 60 minutes, when we expect to know more about the root cause and mitigation workstreams.'"

Others report that the problematic update has affected Azure Kubernetes Service (AKS) clusters. And things do not appear to be going well based on the Azure dashboard, which presently is filled with warning icons of disruption across the world for multiple cloud services.

Azure dashboard ... DNS problems arising from systemd update in Ubuntu 18.04

The Ubuntu update trouble has been reflected on downdetector.com's Azure page, which currently shows elevated reports of problems.

According to Microsoft, the systemd snafu has also affected customers of Azure Government and Azure China. The Windows giant is recommending that Ubuntu 18.04 users disable automatic security updates for the time being. Gulp.

Microsoft Azure's most recent update occurred just after 1500 UTC. It anticipates its next update around 2100 UTC, "or as events warrant."

The systemd patch is a necessary one. It addresses CVE-2022-2526, a use-after-free() vulnerability that could be exploited to cause a crash or to run arbitrary code on a vulnerable box.

"This issue occurs due to the on_stream_io() function and dns_stream_complete() function in 'resolved-dns-stream.c' not incrementing the reference counting for the DnsStream object," explains RedHat's bug report. "Therefore, other functions and callbacks called can dereference the DNSStream object, causing the use-after-free when the reference is still used later."

But applying the patch breaks DNS on Ubuntu 18.04, or rather it did – the buggy update has been withdrawn for repairs. Other Ubuntu releases, including xenial, trusty, jammy and focal are not affected.

For those who have already applied the errant patch, there's a workaround that involves editing the resolved.conf file. Another workaround suggested by Microsoft Azure is simply "to reboot impacted VM instances so that they receive a fresh DHCP lease and new DNS resolver(s)." ®