Microsoft hits Ctrl-Z after Teams trips over file sharing

Maybe don't push to production without properly testing first?

Microsoft Teams experienced a file-sharing outage overnight that disrupted collaboration for many users and forced the software biz to roll back a recent backend change.

The problem began during US business hours, with users reporting odd behavior. Attachments failed to upload, downloads stopped working, and collaboration for affected customers ground to a halt.

Microsoft swung into action and confirmed it was looking into the issue. A note, TM1055900, was posted in the Message Center as engineers investigated.

It took a few hours to identify the cause: readers will be shocked to learn that it was a modification to the backend which led to the outage. Reverting the change fixed it.

Microsoft has suffered a run of backend changes affecting functionality recently. One last week broke software licensing for Microsoft 365 Family subscribers and resulted in alarming and erroneous "Your subscription expired" messages being sent to some users. Then there was the Outlook fiasco caused by a dubious code change, followed by another Outlook fault later that month.

After the last Outlook fail, a Microsoft spokesperson told us: "We are working to enhance our detection of similar events and reduce the time needed to identify, mitigate, or prevent such impacts."

Maybe don't push stuff to production without thoroughly testing it first? File sharing is a key part of collaboration, and breaking it risks seriously disrupting the workflows of many users.

Not all Teams users were affected. Microsoft said the "impact was specific to users who were currently configured on public preview or part of the Technology Adoption Program (TAP)." So that's alright then.

There was also a sort-of workaround, which was doubtless of great comfort to affected customers: "Users were able to view/access previously uploaded files when accessing from the 'Shared' tab within all Teams clients."

Microsoft said: "We're reviewing our change management processes to identify why this issue was not detected prior to deployment."

Matthew Hodgson, CEO of the Element messaging service, told The Register: "The Microsoft outage once again highlights the risks of relying heavily on centralised cloud services."

He noted that going down the centralized route can create single points of failure. In this case, a back-end changed borked the files experience in Teams for many users. Hodgson suggested that companies could mitigate such risks by reducing reliance on any single provider through distributing data and workloads over multiple nodes. ®