Could you just pop into the network room and check- hello? The Away Team. They're... gone

Number One, send in another Away Team

176 Reg comments Got Tips?

On Call Friday is upon us once again, and as the week disappears into the rear-view mirror we have another tale from those princes and princesses of the pager in our regular On Call column.

Today's helpdesk ticket comes from "Christopher", a member of that hardy tribe that like to refer to themselves as "On-call engineers".

Christopher was sunning himself on the West Coast of the United States when his pager went off. He told us: "The network for an entire office building in our company's North Carolina campus had gone down".

North Carolina is very much on the other side of the US so not somewhere Christopher could easily get to. No matter, even in those backward times (a pager? Really?) remote access could be used to sort out most things, right?


Planes, fails and automobiles: Overseas callout saved by gentle thrust of server CD tray


Alas, no. Christopher's local IT Operations Center opened a conference bridge, and despite multiple support personnel flinging in their two cents, the team was stuck. "I was unable to diagnose the problem remotely: both the in-band and out-of-band networks to that building were down."

This was a job for a fleshy meatbag rather than a virtual engineer.

Naturally, the problem had occurred at the weekend, so there weren't many people on site to poke at the physical hardware. However, there was some good news: "There was a network change in progress in another building on the North Carolina campus. We paged out to the team. The site Network Manager called in. He and his two engineers working the change would go over and check out the down building."

The Away Team were dispatched (it's unclear if they were wearing red shirts) and thumbs were twiddled while the gang made their way to the afflicted area. Christopher was not left waiting long.

"They called in a few minutes later from the hardline phone in the main telecom room for the down building: 'All power appears to be off in the room. I see a red light on the PDU [Power Distribution Unit]. We're trying to figure it out.'"

While the on-site team prodded the balky equipment, "the rest of us on the bridge brainstormed ways to assist".

We'll leave it to Christopher to describe what happened next.

"A few minutes later we heard a loud BANG on the bridge. Then 'GET OUT OF HERE NOW' from the manager.

"Then nothing.

"We called out to them. No answer."

Near-panic ensued. The entire away team had been lost, possibly vapourised (Star Trek-style) by a suddenly sentient piece of telecoms hardware.

Situated on the other side of the country, the team pondered what to do. Call 911? How do you even call 911 for North Carolina from the West Coast?

While the gang tried to get more bodies on-site to find, er, what might be more bodies, Christopher explained: "We're also paging out to the manager and engineers trying to get them to rejoin the bridge.

"Minutes pass. Is the team OK?"

After what seemed like a lifetime (or just enough time for Captain Picard to deliver a sorrowful soliloquy following the sudden passing of an all-too-expendable crew member, way down the list of credits), the team finally dialled in.

"A power system for the room had 'exploded' (although not a big explosion like in the movies – more a loud bang and some smoke). The team was fine."

The power system less so.

Before the network could be restored, something would need to be done about the power.

That would need another Away Team.

Ever teleported a team into peril or heard something go boom on a conference call? Rinse the memory with an email to On Call. ®


Biting the hand that feeds IT © 1998–2020