On Call Corporal Cockup meets Major Outage in this week's episode of On Call as a reader's helpful walkthrough takes down the telephony server.
Today's recollection by reader "Simon" (for that is not his name) takes us back more than a decade to the early part of this century, where he was putting in time doing first line support for a public sector firm.
As was so often the case, a user called in with a problem. They reckoned it was an odd one – something to do with the network – and naturally wanted Simon to sort it out. The user in question was not located in the same office as Simon (and was actually a 20 minute drive away) so he fired up Netop to take a look at what was going on.
For those not familiar with Netop, the Nottingham-based company was founded back in 1981 and claims a userbase numbering in the millions. Acquired by Impero earlier this month, its products include live chat tools and remote control software. It was the latter that Simon loaded to diagnose the problem.
"I tried to connect remotely onto the staff member's PC," he told us, "but I couldn't connect. I dug deeper and I couldn't even ping it."
Wondering if perhaps there was an issue with the PC's network configuration – perhaps a borked default gateway or iffy DNS – Simon hopped onto another bit of hardware at the site to see if he could ping the machine on the local network. He selected a server, one of the telephony units (tasked with voicemail rather than calls), and had a go at getting a response back from the stricken PC.
- Can't get that printer to work? It's not you. It's that sodding cablin.... oh beautiful job with that cabling, boss
- George Clooney of IT: Dribbling disaster and damp disk warnings scare the life out of innocent user
- To have one floppy failure is unlucky. To have 20 implies evil magic or a very silly user
"So," he told us, "in my youthful naivety, I talked them through how to disable/re-enable their NIC."
It seemed a reasonable thing to do. Talk a user through the actions required while pottering through the same dialogs on your own screen.
"The problem," said Simon, with breath-taking understatement, "was that I was connected to the telephony server at the time whilst going through the steps…"
As he chattered away to the user, he went through the steps to kill the NIC: "At the point I hit to right click > disable ... I realised I had just disconnected the telephony server from the network!"
Voicemail was no more, although thankfully the phones were still working. After the bowel-emptying realisation had subsided, Simon thought quickly. What to do?
"I told the staff member I would travel down to fix it immediately," he told us. An in-person visit from IT support? The worker must have thought all their Christmases had come at once. Best not try to use voicemail though, eh?
Simon rushed to the site and barrelled into the server room. The telephony box was swiftly reconnected before he sauntered over to the stricken user to deal with the original problem. "Turned out their Windows Firewall was enabled," he told us. "The location of the AD object was the cause of the original issue too."
A simple fix for a mighty brain, able to deal with user gripes as well as remotely disconnect servers by accident.
The user, ignorant of Simon's real motive for the on-site visit (deborking what he had borked previously), was delighted and said: "I have never had someone travel out so quick to fix this kind of problem before!"
Simon, striding back to his car, called back: "It's all part of the service!" There is no record regarding if the customer spotted his lengthening nose or if there were any external signs of his internal cringing.
Ever done something really silly but somehow spun things so you were the hero of the hour? Or idly talked a user through a process only to find doing the same on your own terminal had… unexpected consequences? Of course you have, and you should share your story with an email to On Call. ®