This article is more than 1 year old

'The server broke and so did my back on the flight to fix it'

Epic tale of idiot sysadmin and the trek to clean up the mess left behind

I awoke in more pain that I had ever felt in my life

Next came a 90-minute drive to get to the hotel nearest the site. JT finally made it to his destination at 3:00 AM, crashed out and woke up “four hours later in more pain than I had ever felt in my life.” Walking to the bathroom was only just possible, and that in a stooped shuffle.

“After a short shower I was finally standing and decided to head into the office,” JT explains. That plan was derailed by a request to visit a new plant because the CEO would be there.

“I drove up to the new plant and pulled into the parking lot nearly running over the CEO. After an uncomfortable discussion with him about why I was some place other than where he expected me to be I was required to walk along with the rest of the group on a tour of the new plant, stepping over wire, steel framing, and conduit while avoiding tripping over the occasional cursing labourer. I was almost weeping by the time I got back to my rental car.”

JT eventually made it to the office, having decided that the last thing he needed was a doctor – he was probably better off not knowing what was up with his back until he fixed the server – so he scoffed “a huge handful of ibuprofen and acetaminophen” before tackling the mess Hubswitch had left behind.

“It only took five minutes to figure out what had happened,” JT recalls. “When Hubswitch had come in the previous morning the Apple server was powered off. He hit the power switch to get it booted.

"The system came up and said it couldn't find a boot disk, but hey, there's a hard drive here would you like to format it? It was an idiotic decision by Apple. If a server boot drive became unreadable the first thing the computer would do was offer to format it. There were two options on the screen. The first was 'No'. The second with the bold default lines around it was 'Okay'. Hubswitch had hit the Enter key without reading the message.”

Hubswitch had therefore made the boot drive the main data drive, then formatted it a few times.

“Everything was gone,” JT says. “None of my file recovery tools worked. The drive had been obliterated by ineptitude.”

It took JT a few hours to rebuild the server, during which time a five-month-old backup turned up. JT tried doing most of this without standing up, but “occasionally had to painfully limp across the building to solve a problem. Every step was pure agony.”

A couple of days later JT felt well enough to fly home, where his wife had kindly placed a mattress on the floor. And there JT lay for three days.

The eventual diagnosis was three herniated disks.

“To this day,” JT tells us, “every time I feel even the slightest twinge in my legs ... I think of Hubswitch – and I want to hunt him down and kill him.”

Do you have a story to rival JT's? If so, do let me know and we'll slot you into a future On-Call. We're also keen to hear from those who've moved overseas for our eXpat Files features. Again, ping me. ®

More about

More about

More about

TIP US OFF

Send us news


Other stories you might like