AWS endures extended outage in Australia

Heavy clouds take out clouds

Sysadmins in Sydney had a horrible Sunday, while their CEOs bent their attention to wondering why their Foxtel Rugby sportscasts weren't working.

The Amazon Web Services AWS "power event" AP-SOUTHEAST-2 Region was almost certainly caused by a massive storm system that ran from Brisbane to the NSW South Coast, leading to a weekend of widespread flooding, blocked road and coastal erosion. A lot of usually dry places between Byron Bay and Sydney became navigable waterways.

Affected services included EC2, Elastic Load Balancing, ElastiCache, Redshift, Relational Database Service, Route 53 Private DNS, CloudFormation, CloudHSM, Database Migration Service, Elastic Beanstalk and Storage Gateway.

The outages reached six hours or more.

Twitter users noticed the troubles.

What is happening with AWS?

— Jonathan Ferguson (@jonoabroad) June 5, 2016

To Aus customers. Wild weather in Sydney causing power issues https://t.co/ep0Z9bHFPp apologies for any disruption.

— My Film Bag (@MyFilmBag) June 5, 2016

Linking the #aws Sydney power outage and the @mebank card outage. Can anyone confirm? Hmm?

— gkachel (@gkachel) June 5, 2016

AWS EC2 in Sydney was in such bad shape you couldn't even post a support ticket...

— geirmagnusson (@geirmagnusson) June 5, 2016

Now my #AWS instance which are still having issues won’t respond to stop/start

— Rachael (@halfwrittentale) June 5, 2016

About now we hope AWS finally starts thinking about running an AZ in Australia outside of Sydney

— Dawnstar Australis (@dawnstarau) June 5, 2016

There was a bit of “what's going on?” discussion on the Australian Network Operators Group's mailing list (AusNOG), which drew this thundering warning for media not to report list chatter that is posted in plain sight (The Register presumes is in response to a previous Vulture South story that drew from Ausnog posts).

So we won't tell you that some network CEOs were on the job on a Sunday dealing with the problem.

AWS still hadn't completely recovered at 7:00PM, Sydney time, on Sunday night, if Twitter posts were anything to go by.

Amazon Sydney EC2 still down, sorry folks #AWS

— FlexiTime (@FlexiTimeTeam) June 5, 2016

The water will drain away, eventually. ®