Nothing's working, and I've checked everything, so it must be YOUR fault

Yes, but are you sure? Really sure?


On Call The customer is always right. Except when they're not. Here we have a story from the On Call archives concerning connectors, telephones, and a user blessed with a little too much confidence.

Today's tale takes us back to the 1990s and comes from a former employee of a now-defunct telecom equipment vendor. We'll call him Felix, for that is not his name.

"I got to do a lot of on-site troubleshooting for the specific subsystem of the telecom kit that I had expertise in," Felix explained. One of the UK's telephone operators of the era was going through a period of modernisation and replacing decade-old kit with something decidedly more modern.

The modern devices were considerably faster and less power hungry than the elderly hardware. However, the rollout also necessitated some serious reconfiguration of the older gear. "A particular blade in the old kit," Felix told us, "could be made to operate in either an active or passive state depending on the orientation of a plug-in connector block.

"To change the state all you had to do was pull the blade, change the connector block orientation and plug the blade back in."

The hardware was pretty fundamental when it came to keeping the UK PSTN network going and a failure could mean that some end users might find themselves with dead landline phones. This being the era before the dominance of the mobile phone, "not having landline service was a VeryBadThing™," said Felix.

The call came in at 4:30pm on Friday, because they always do. It came directly from the tech lead of the operator's modernisation programme. Felix recalled the very much one-way conversation.

"It's all gone TITSUP!" Felix recalled him screaming down the phone, "Location X is failing to sync with the rest of the network, I've checked all the hardware and the config and it should work. But it's not!"

All we know about Location X was that it was a very, very important customer and it being down was costing the telephone operator an unthinkable sum per hour in terms of lost revenue as well as reputation.

Felix adopted the diligent approach familiar to all those who take such calls: "Has anyone changed anything? Are you absolutely sure the config is correct?"

"Of course it is!" bellowed the caller, "and we need someone down here to fix it ASAP!"

"Of course they did," sighed Felix.

Being possessed of the shortest of straws, it fell to Felix to source a company car and all the equipment required to diagnose what had befallen the gear before undertaking the 250-mile (402km) trip to site.

"By the time I was ready to start work it must've been close to midnight," he said.

But at least he was there.

He checked the clock signals coming out of the kit and confirmed that, yes, it was definitely not synching with the rest of the network. Next, he looked at the hardware. Out came the blade and…

"Lo and behold the connector block was in the 'passive' orientation."

"If ever I've seen the face of someone who wanted the ground to swallow him up," said Felix, "Mr Tech Lead was him."

Wordlessly, Felix moved the connector block to the correct orientation and replaced the blade. Everything sprang into life. There had been no need for all the expensive test equipment nor, to be honest, his presence. He hadn't even had time to book a hotel in the panic to get to site.

"So," he surmised, "basically I travelled 250 miles on a Friday evening, with a car full of tens of thousands of pounds worth of test gear, under threat of 'the world is ending' only to unplug a plug-in connector, turn it through 180 degrees and plug it back in again...

"Back then I could claim call-out and overtime, though, so at least it didn't seem quite so bad."

As for who had done the deed, Felix reckoned it was more likely a local tech being helpful. It wasn't until the upgrade programme had got under way and other blades had been changed out that the problem reared its head.

Still, we've all had that call from the customer who was convinced everything was correct until a quick eyeball confirmed it most definitely wasn't. Did your connector cockup involve a blue flash or a red face? Tell all with an email to On Call. ®

Similar topics


Other stories you might like

  • Saved by the Bill: What if... Microsoft had killed Windows 95?

    Now this looks like a job for me, 'cos we need a little, controversy... 'Cos it feels so NT, without me

    Veteran Microsoft vice president, Brad Silverberg, has paid tribute to former Microsoft boss Bill Gates for saving Windows 95 from the clutches of the Redmond Axe-swinger.

    Silverberg posted his comment in a Twitter exchange started by Fast co-founder Allison Barr Allen regarding somebody who'd changed your life. Silverberg responded "Bill Gates" and, in response to a question from senior cybersecurity professional and director at Microsoft, Ashanka Iddya, explained Gates' role in Windows 95's survival.

    Continue reading
  • UK government opens consultation on medic-style register for Brit infosec pros

    Are you competent? Ethical? Welcome to UKCSC's new list

    Frustrated at lack of activity from the "standard setting" UK Cyber Security Council, the government wants to pass new laws making it into the statutory regulator of the UK infosec trade.

    Government plans, quietly announced in a consultation document issued last week, include a formal register of infosec practitioners – meaning security specialists could be struck off or barred from working if they don't meet "competence and ethical requirements."

    The proposed setup sounds very similar to the General Medical Council and its register of doctors allowed to practice medicine in the UK.

    Continue reading
  • Microsoft's do-it-all IDE Visual Studio 2022 came out late last year. How good is it really?

    Top request from devs? A Linux version

    Review Visual Studio goes back a long way. Microsoft always had its own programming languages and tools, beginning with Microsoft Basic in 1975 and Microsoft C 1.0 in 1983.

    The Visual Studio idea came from two main sources. In the early days, Windows applications were coded and compiled using MS-DOS, and there was a MS-DOS IDE called Programmer's Workbench (PWB, first released 1989). The company also came up Visual Basic (VB, first released 1991), which unlike Microsoft C++ had a Windows IDE. Perhaps inspired by VB, Microsoft delivered Visual C++ 1.0 in 1993, replacing the little-used PWB. Visual Studio itself was introduced in 1997, though it was more of a bundle of different Windows development tools initially. The first Visual Studio to integrate C++ and Visual Basic (in .NET guise) development into the same IDE was Visual Studio .NET in 2002, 20 years ago, and this perhaps is the true ancestor of today's IDE.

    A big change in VS 2022, released November, is that it is the first version where the IDE itself runs as a 64-bit process. The advantage is that it has access to more than 4GB memory in the devenv process, this being the shell of the IDE, though of course it is still possible to compile 32-bit applications. The main benefit is for large solutions comprising hundreds of projects. Although a substantial change, it is transparent to developers and from what we can tell, has been a beneficial change.

    Continue reading

Biting the hand that feeds IT © 1998–2022