Blow me down with a feather, well, storage server software update gone awry: Nest vid streams go dark for 16 hours
Google so sorry after devices fly the coop in multi-hour outage
Google has apologized for 16 hours of downtime for countless Nest cameras that left angry owners unable to watch live video streams from their gizmos.
“We’re still doing some investigating, but at this point I can share that the issue was due to a scheduled storage server software update that didn’t go as intended,” explained Rishi Chandra, Google vice president overseeing Nest in a support note on Monday. “Whenever something like this occurs, we carefully look across our systems to figure out how to make sure it doesn’t happen again.”
Server failure and the complete collapse of an online service is not something most people would expect from Google, given its position as one of the world’s largest providers, if not the largest provider, of online services. “Please accept my apology,” Chandra wrote.
Users started complaining about the issue around midday Pacific Time on Monday. The issue affected all of Nest’s internet-connected streaming cameras – it sells indoor and outdoor cameras and a video doorbell – and even its latest Nest Hub Max smart display. Streams are usually accessible through a smartphone app or by logging into a Nest account through a browser, but instead they saw the Nest equivalent of the Spinning Wheel of Death.
It took a surprisingly long time for Google to fix the problem. It acknowledged the issue on its status page an hour after the first reports, but it took four hours to start restoring access. It then took until another four-and-a-half hours before related services such as notification alerts came back online. The issue wasn’t fully resolved until 0600 PT – roughly 16 hours after the issue first appeared.
The downtime was not appreciated by punters who pay up to $30 per month per camera to have access to 30 days of recordings.
“This outage is unacceptable and leaves my house and family at risk. All the expensive cameras I installed at my property are useless... and even worse zero recording history during this outage. I expect a credit on my Nest Aware account for this inconvenience,” complained one customer we chose at random from a long list of people complaining on Twitter.
Google has promised to provide more updates in the coming days. It is notable that it reported what looks like an identical issue back in December, raising questions over how it does critical and customer-facing updates and rollouts.
Ring in the changes: Mandatory two-factor authentication, login alerts, targeted ads opt-out after punters voice privacy gripesREAD MORE
The downtime is also one more black mark against Google since it started pulling Nest into its larger corporate family. The web titan bought the smart home gadget slinger for $3.2bn in 2014, and initially allowed the manufacturer to run autonomously, including using its own proprietary code and protocols.
But when Nest failed to put out any new products for several years, people started wondering what was happening. Then in April 2016, to a chorus of fury, Nest decided to brick its Revolv automation hub. That was the beginning of the end for Nest CEO Tony Fadell who left in June, just days before Nest finally released a new product - an outdoor surveillance camera with cloud-based storage on subscription.
With Fadell gone, the corporate overlords at Google started pulling Nest into its clutches. Nest’s decision not to include Google’s Brillo IoT standard in favor of its own Weave protocol was revised, and Nest's focus on Apple’s smartphones - because that’s what the majority of its customers used - was slowly bent in favor of Google’s Android devices.
And we're back
All was forgiven however when Nest put out its first new products for years: a camera with impressive software capabilities in May 2017 and then a whole new range of an outdoor camera, doorbell and security system in September 2017. The outfit was back on top, offering a more expansive range of products, albeit at a higher price than the growing competition, in particular from Ring - recently bought by Amazon.
But then came Google’s efforts to cram its own products into the Nest range. Google Assistant was added as a voice-activated controller - with very mixed results - and Google’s answer to Amazon’s Alexa, the Google Mini, was pushed as an extension to the Nest product range. It didn’t work well and ruined Nest’s hard-earned reputation for seamless integration.
Those painful integration efforts continued regardless. In 2018, Google announced that Nest was going to “join forces with Google’s hardware team,” i.e. be fully subsumed into the corporate monster. And the next year, all the names were changed: the Google Home Hub became the Nest Hub and all Nest products had the “Google” unceremoniously stuck on the front of them.
But when Google announced in May 2019 it was going to kill off its third-party integration program Works with Nest, there was such an explosion of anger that Google was forced to backtrack - at least for a while.
Come this way
Since then Google has been trying to force Nest users to move over to Google accounts - something that large numbers of customers are stubbornly refusing to do, based on the fair assumption that everything they do from that point will be added to the Google advertising-database.
As Google tries to suck Nest’s tightly focused ecosystem into its larger world of online services, users keep reporting problems - of which this week’s server upgrade is just the latest example.
The truth is that a lot of people don’t want to invite Google into their home. It’s great for searching the web and email, but everyone knows that the internet giant exists to sell your data to advertisers and people don’t want their homes and daily lives to be turned into dollar signs.
Nest products were different. They were standalone, worked seamlessly and efficiently and you paid for them, so there was no profit motive to sell whatever information could be gleaned from their use. But Google just can’t help itself: everything is saleable and now it just needs to pull those Nest products into its systems to access all that delicious data.
The fact that users were unable to see their camera live streams for half-a-day may have been an irritant, but it’s not clear that most of them recognize what it really represents: Google eyeballs on your life. You’d think the Silicon Valley kingpin would have been more careful about the transition given the sensitivities, but it wasn’t. Because, after all, if the shit hits the fan, it only takes 20 minutes to write an apology. ®
PS: Internet-controlled pet feeder biz Petnet went offline for a week this month, leaving animals to go hungry, it was reported.