123 Bork? Six-day DNS record-edit outage at domain name flinger 123 Reg enrages users
It's not DNS. It can't be DNS. It's actually a TITSUP*
Customers of the UK's self-professed #1 provider of domain names, GoDaddy-owned 123 Reg, have had a frustrating few days after finding DNS records disappearing from their dashboards.
The issue is a nasty one as the vanishing of the records prevents users from assigning domain names to IP addresses or making edits. This is quite unfortunate when one considers that registering domain names is what 123 Reg is all about.
Things began to go off the rails as long ago as last Thursday, 5 November, when the company admitted that some customers "may experience issues with DNS management." Not to worry though, the hardworking 123 Reg team was on the case and a fix would be deployed ASAP.
Except, alas, it wasn't soon enough. The problem continued until the afternoon of 6 November, when there was some good news: the cause of the DNS record problem had been identified. However, rolling out the fix would need a bit of weekend working, according to the company's status page, so the new ETA for resolution was set to 9 November. After all, nobody needs to tinker with DNS records over the weekend, right?
A Register reader got in touch to share his own experience of attempting a customer's migration to Microsoft 365 which, sensibly, was planned over Saturday and Sunday. His tale of woe included repeated attempts to coax a response out of the company and a lost weekend while he fruitlessly refreshed his control panel in search of the needed DNS records.
Another told us the inability to edit zones was holding up deployments for a number of clients, and even the 123 Reg support team appeared unable to make some of the more urgent changes manually. "Outbound domain transfers do seem to be working however," he told us.
It took until after 17:00 UTC on Monday for an update to be released by the 123 Reg team, but the fix only partially resolved things. Some users found themselves still unable to make updates to their DNS records as of Tuesday, days after problems were first reported.
The company told those still wondering just where in blue blazes the functionality they were paying for had gone that the next deployment was set for Tuesday and would resolve the issue once and for all.
Hello Chris and thank you for reaching you. Our team deployed a fix yesterday which resolved the problem for some users. The next deployment is set for today, which should fix it for everyone else. Please keep an eye on our status page for more info: https://t.co/7BvXqz8TtH.— 123 Reg Help (@123regHelp) November 10, 2020
At time of publication, however, 123 Reg's status page continued to admit the company was "aware" of problems, but affected customers hoping for an ETA after a nearly a week of borkery weren't given one.
The Register has contacted 123 Reg to find out what befell its systems and the company has now sent us a statement:
"Following a recent domain platform update, some customers were prevented from viewing or editing their DNS records. We have identified and corrected this for the majority of customers affected. We encourage any customers who are still experiencing errors to contact us at www.123-reg.co.uk/contact for assistance. We understand the importance to our customers of the management of DNS records and we are sorry for any inconvenience caused."
We'll leave the last word to one of the company's frustrated customers:
Paul remained sanguine when informed that 123 Reg was actually owned by GoDaddy. ®
Ah that's a bit annoying but at least Godaddy has working DNS!— Paul Beebe (@paulbeebe) November 10, 2020
* Tottering Infrastructure Thwarts Service User Plans