Microsoft breaks geolocation, locking users out of Azure and M365
Customers banished to an IP address in Uzbekistan that Redmond’s cloud did not recognize
Microsoft has found a new and interesting way to break its cloud services: by messing up geolocation services and sending its users to Uzbekistan, which made it impossible for them to log in.
The Beast of Redmond let the world know about the mess with this oblique tweet regarding Microsoft 365:
We're investigating an issue where users with specific conditional access policies applied may be unable to access any Microsoft 365 service. We're reverting a recent change to mitigate impact. For more details please look for MO531859 in the admin center.— Microsoft 365 Status (@MSFT365Status) March 23, 2023
A kind Reg reader sent more detail: a Microsoft status notice that defined the problem as "a subset of users with geolocation-based conditional access policies experienced sign-in disruptions to Azure services."
The reader's info said the mess was caused by "a recent deployment applied to an infrastructure for regulating user geolocation had inadvertently provided incorrect IP location data."
This resulted in users who had an IP-based conditional access policy experiencing the sign-in issues mentioned above.
Our reader sent us an image of the error message seen by his clients in Darwin, Australia, which explained that Microsoft would not allow access to Teams because the traffic came from Toshkent, Uzbekistan.
Chileans also appear to have been sent to Uzbekistan if this Tweet is any guide:
the problem occurs because they are changing the IP to another country. pic.twitter.com/Fe5WhtWeTJ— Marcos (@marck_al) March 23, 2023
And so were United Statesians:
Not cool. Sending US-based traffic to Uzbekistan is unacceptable on many levels. pic.twitter.com/ZD6Wlmeaea— Brian Wilson (@firstname.lastname@example.org) (@brianwilson) March 23, 2023
Interestingly, the UK's National Health Service, a known Microsoft 365 user, listed the incident in its tech support feed. Hopefully the mess didn't impact services there.
- Microsoft admits Azure Resource Manager failed after code change
- Microsoft's Copilot AI to pervade the whole 365 suite
- WAN router IP address change blamed for global Microsoft 365 outage
- Microsoft Office 365 Cloud has a secret lining
Microsoft's Twitter thread about the incident suggests it was sorted out within about nine hours.
Users disputed that and did not appreciate Microsoft's handling of the situation.
Wow started 0300 GMT and now you tell us! I reported it on at 0830 GMT from the UK and was advised then that it was more than just ASPAC region. Come on chaps, I rely on these tweets as usually faster updates than the admin centre!— Arli (@ArliSunblade) March 23, 2023
Making matters worse, the IP addresses in question appear to be IPv4. Shame, Microsoft, get thee to IPv6!
Those of you with M365 admin accounts may be able to find a more detailed status report with the identifier "MO531859". If you read it, and it details the precise duration of the outage, please let us know in a comment so that when we next report an M365 outage we can accurately rename the service to reflect its actual uptime. ®