Google code cloud in six-hour blinkage

App Engine stalls

Google App Engine - the development and hosting cloud that serves up third-party apps and websites - was on the fritz for a good six hours this morning.

According to a public note from Google's App Engine team, the service began experiencing problems at around 6:30am Pacific. "We're currently seeing elevated Datastore latency and error-rates, as well as elevated serving error-rates," the note read. "All applications accessing the Datastore are affected."

By 8:45am, the App Engine team was in "unplanned maintenance mode," disabling all application deployments, Datastore writes, and memcache writes. Problems continued for nearly four hours. At 12:35, the team announced that all functions had returned to normal.

That would appear to be six hours of performance problems. Nonetheless, Google's official PR arm puts that figure closer to four and half hours. According to the App Engine "System Status Dashboard," the problem wasn't widespread until 8am.

"Today at 8 am PT datastore access for App Engine applications was affected due to a cluster-wide issue," they tell us. That was cluster-wide, not cluster-fuck.

"The team identified and fixed the underlying problem and service has now been restored. We apologize for the inconvenience and encourage anyone having technical difficulty to visit the System Status Dashboard or the Downtime Notify Group, which are both linked from the Google App Engine Community site."

According to Twitter, countless third-party sites were affected. And as TechCrunch reports, the cloud failure even hit those Googlers working on a version of Chrome for the Mac. ®