G-Cloud bigwig says soz for Cloud Store outage

Azure crash downed gov e-catalogue for 2 hours


The G-Cloud governor has apologised to public sector customers for the Cloud Store outage yesterday but pointed out that downtime is not confined to an off-premise delivery model.

The apps store is hosted on the Azure platform, which was down for eight hours, although the Cloud Store was only out of action for roughly a quarter of that time as Microsoft moved the platform to a different install.

Director of G-Cloud Chris Chant wrote on his blog that the current version of CloudStore was developed for free, while the government is being charged for hosting along with the accompanying minimum maintenance on a "pay as you go" basis.

He wrote: "The Azure instal we are using does not provide multi service deployment. This is because the CloudStore is not critical government system.

"As a result it is unlikely that we would ever pay the premium that would be needed to guarantee “five nines” up time for the site as it simply isn’t a wise way to spend scarce resources," he added.

Some customers working with the site emailed to get the data they required and while the outage was "disappointing" and work is being undertaken to "speed up the recovery process", Chant said there was "sufficient fallback" in the system.

"There are, of course, those who say 'this is the problem with cloud'. Well, those of us who have worked in a 'non cloud' IT environment for the last two or three decades have not been free of such outages I’m afraid, nor are we likely to be in the future."

However, he thanked Cloud Store builder Solidsoft "for helping us out of the problem" and gave "apologies to customers that were inconvenienced." ®


Other stories you might like

  • Cloudflare's outage was human error. There's a way to make tech divinely forgive
    Don't push me 'cos I'm close to the edge. And the edge is safer if you can take a step back

    Opinion Edge is terribly trendy. Move cloudy workloads as close to the user as possible, the thinking goes, and latency goes down, as do core network and data center pressures. It's true  – until the routing sleight-of-hand breaks that diverts user requests from the site they think they're getting to the copies in the edge server. 

    If that happens, everything goes dark – as it did last week at Cloudflare, edge lords of large chunks of web content. It deployed a Border Gateway Protocol policy update, which promptly took against a new fancy-pants matrix routing system designed to improve reliability. Yeah. They know. 

    It took some time to fix, too, because in the words of those in the know, engineers "walked over each other's changes" as fresh frantic patches overwrote slightly staler frantic patches, taking out the good they'd done. You'd have thought Cloudflare of all people would be able to handle concepts of dirty data and cache consistency, but hey. They know that too. 

    Continue reading
  • You need to RTFM, but feel free to use your brain too
    But I was only following the procedures!

    Who, Me? Monday is here, and with it a warning that steadfast determination to ignore instructions might not be such a silly thing after all. Welcome to Who, Me?

    Today's story comes from a reader Regomized as "Sam" and takes us back to his first proper IT job following his departure from the education system.

    Sam found himself on the mainframe operations team for a telecommunications company. The work was, initially, pretty manual stuff. The telco wasn't silly, and had its new recruits start by performing offline duties, such as gathering tapes and job tickets for batch runs, handling payslips, "basically anything involving a bit of leg work," he told us.

    Continue reading
  • Tropical island paradise ponders tax-free 'Digital Nomad Visa'
    Live and work in Bali, pay tax at home

    The government of Indonesia has once again raised the idea of creating a "digital nomad visa" that would allow foreign workers to live and work in the tropical paradise of Bali, tax free, for five years.

    The idea was raised before the COVID-19 pandemic, but understandably shelved as borders closed and the prospect of any digital nomads showing up dropped to zero.

    But in recent interviews Sandiaga Uno, Indonesia's minister for Tourism and the Creative Economy, said the visa was back on the drawing board.

    Continue reading

Biting the hand that feeds IT © 1998–2022