On Call Welcome back to On Call, The Register's weekly dive into the world of those who live in dread of the surprise pager or midnight phone jangle.
Today's tale comes from a reader we'll call "Jordan", who was working at a systems integrator a few short years ago. He and his chum (who was put into the Reg anonymiser and spat out as "Tim") were happy at the outfit, which "hosted apps for a number of clients which we sold under a SaaS arrangement".
"It was," Jordan told us, "a great gig for us as the company was small with nice people but, well, not very switched on."
Both Jordan and Tim had lengthy commutes to the office, with long hours resulting in one or two overnight stays. "We did put a lot of hours in but we also played hard too," said Jordan, as he and Tim "pushed the Milton Keynes nightlife to the limits".
Our hero returns home £500 richer thanks to senior dev's appalling security hygieneREAD MORE
Those roundabouts can be quite the adventure for the unprepared.
Jordan, it can be safely assumed, enjoyed a beer or two. On the night in question, "the guys were doing an overnight change control for a UK bank". Jordan had managed to avoid any involvement due to, as Tim put it, "having shoulders like a Coke bottle".
We suspect that means they sloped a tad rather than being filled with fizzy caffeinated delight.
At 3am the inevitable happened, and the phone rang. Jordan, who had imbibed some alcoholic beverages, was
passed out "snoozing on my settee" when he was woken by a call from the Project Manager telling him: "Tim and the guys are still at it, that they hit a snag around midnight and have been struggling with it ever since."
And would he mind, even though this wasn't his change, hopping on a WebEx to help out?
Jordan complied, pausing only to, perhaps unwisely, grab a beer before firing up the laptop.
Tim, who Jordan described as "a lovely bloke but if something can be said in 5 words or 50, he'll go for 50", explained the problem. A Tomcat server had been upgraded, but since the reboot was throwing errors.
Even without the beer, Jordan would not describe himself as a boffin about such matters, explaining: "I can do technical, but [expletive deleted] usually gets worse before it gets better when I do." Hence he mainly stuck to the talking.
However, with the error log visible, Jordan did still possess just enough in the wits department to do what any techie worth their salt would: "I googled the first error message I saw in the log file."
While Tim continued "talking-whinging", Jordan clicked the first link and attempted to focus on the web page.
"Given, I'm probably still quite a bit drunk, tired and desensitised to Tim's technical babble I read... sort of... this article and without any thought I just about mumble semi-coherently, 'have you checked the host file?'"
Everything went a bit quiet. The server's host file was opened on the WebEx and Jordan read the next paragraph in the article. With the power of beer overriding his fear of ridicule, Jordan asked the next question: "Is that IP address right?"
Years later, Jordan still remembers Tim's heartfelt sigh "because he knows he will never live this moment down".
"No," said Tim, "that hostname should just resolve to localhost, not a completely different IP."
Cue a swift host file edit, restart and "It's ALIVE!"
Jordan was lauded for his "keen technical insight" when he travelled to the office the following week, which had saved the company from the embarrassment of having to roll back the change for that UK bank.
Naturally, he never admitted "how I stumbled upon someone else's answer whilst googling half-cut".
Ever solved a problem thanks to a visit from the beer fairy? Or been called upon to fire up a remote desktop from the sanctity of the pub? Send an email to On Call and tell us all about it. ®