A toast to being in the right place at the right time
Techie went to investigate mysterious network dropouts, when the solution popped up
who, me? Once again, dear reader, it is Monday, and we brace for the start of a new working week. To ease the pain, The Reg brings you Who, Me? in which readers tell their tales of derring-do – or at least derring-try.
This week meet a reader we'll Regomize as "Nellie", who long ago worked for a Primary Care Trust – part of the UK's National Health Service (NHS). Now, for non-UK folks, it's worth noting that a lot of the buildings the NHS uses were built long before technology was a big part of healthcare. Heck, some of them were built before hand-washing was a big part of healthcare. But we digress.
The point is that these buildings did not have dedicated spaces for servers and the like, so the techs who installed them made do as best they could. In the case of the building in which this week's adventure is set, that meant installing a half-height rack under a bench in the break room.
When Nellie started as team leader, the various IT bods were invited to compile a list of their primary gripes – the things most deserving of a fix.
In one health center the number one gripe was mysterious WAN dropouts. Every so often – not regularly – the network would simply disappear, and then reappear a few minutes later. No-one knew why.
Naturally, there was redundancy built in, but it was in the form of a very slow VPN. This simply was not fit for purpose.
Staff were sent out to the site, diagnostics run, monitors monitored, packet sniffers sniffed, and nothing found. A mystery.
- Fed up with slammed servers, IT replaced iTunes backups with a cow of a file
- Seriously, boss? You want that stupid password? OK, you get that stupid password
- Rigorous dev courageously lied about exec's NSFW printouts – and survived long enough to quit with dignity
- An important system on project [REDACTED] was all [REDACTED] up
One morning, on the way to work, Nellie decided to pay a personal visit to the building just in case. Since the kit in question was in the break room, a cuppa was brewed and a laptop opened to begin testing.
Just then, the phone rang. It was one of the remote engineers: "Link's gone down."
Nellie confirmed locally that, indeed, it had.
Then another unexpected sound: the popping of a pop-up toaster. Nellie watched as a staff member collected their toasty treat, unplugged the toaster, and plugged in another cable.
"Link's back," said the remote engineer.
"And I can see why," said Nellie, having witnessed an operator fault no diagnostic software could ever have detected.
It transpired that when the local cable provider had installed the fiber terminator in the rack, there hadn't been a spare power point available – so they just plugged it in above the counter. For years the medical center had been getting by on a barely functional network every time someone made toast.
All for want of a double adapter.
Have you ever encountered similar breakfast-related downtime? Has someone else's cut corner cut into your day? Tell us all about it in an email to Who, Me? and we'll maybe brighten someone's Monday.