Pop quiz: The network team didn't make your change. The server is in a locked room. What do you do?
Not all heroes wear capes. Some are Unix admins.
Who, Me? Welcome to another entry in The Register's Who, Me? archives. Today, a reader goes full Hollywood to save the day (and fix some IP addressing).
Our story comes from Dave and takes us back to the Australia of the 1990s. It was the era of Paul Keating and John Howard and, significantly, a time of advancement in telecommunications technology.
Riding that wave was our reader, "Dave" (no, not his real name) who was working in software and infrastructure for a government agency. His team had developed an imaging system ("back when that was hard," he said modestly) that could display trademark registrations on the new-fangled Windows desktops that were popping up all over the place.
"The application worked a treat," explained Dave, "and saved everybody heaps of time in their day job, and successfully displaced some dedicated SUN workstations that were used to display the coveted trademarks database images."
"Instead of queuing, examiners could simply use their desktop PCs to do their day job. Unfortunately, it was a victim of its own popularity."
It was indeed. The problem was that the servers were in one office and the examiners in another, about 12km away. Connecting the two was a 2Mbps link, which was frankly not up to the job as more and more users piled on.
Ever keen to keep the users happy, Dave had a brainwave. Why not simply put the servers in the same building as where the bulk of the work was done? A simple job – lug the physical hardware to the new location, set up the IP addressing for its new home and – Hey presto! Speedy images and happy users.
Change requests were approved and, at 9pm on Sunday, Dave and his team yanked out the server. An hour later it was racked in its new location, communicating its contentment via the medium of flashing lights and spinning fans.
Except none of the workstations in the building could see it. It was connected. Lights were on. But the all-important images were not accessible.
"After a couple of hours of fault-finding, we discovered that the networks team had not yet implemented the IP address change that was needed to allow the server to reside at the new location on the network," explained Dave.
It was now 1am. In seven hours' time, 600 users were due to turn up and expect things to be working. What to do? An urgent call was placed to the networks team, but nobody answered: "There was no such thing as 24x7 support back then," said Dave.
So again, what to do?
Dave, who had probably watched way too many heist movies, had an idea. Sure, the team could simply revert the change. Or he could become Dave The Action Hero. The securely locked network equipment room was on the other side of the partition wall. The door wasn't an option, but he found that if he slid a ceiling tile out of the way he could clamber through the ceiling cavity and over the partition...
"The landing on the floor hurt a bit," he said, "but within a couple of minutes I had rebooted the FreeBSD server that ran DNS into single user mode, made the necessary modifications to the DNS files and restarted the server."
A swift jab of the big green "Exit" button and Dave was out. It was now 2am. Six hours until the users were due and everything was up and running.
He slept in that morning and arrived in the office at lunchtime "to be greeted by some *very* angry network team members.
"It turns out that the correct solution did not involve jumping the partition."
So the users were happy, one set of Dave's management was happy. And one set was livid. Dave shrugged, the application was now working beautifully. And he had learned an important lesson.
- Planning for power cuts? That's strictly for the birds
- Hacking the computer with wirewraps and soldering irons: Just fix the issues as they come up, right?
- A time when cabling was not so much 'structured' than 'survival of the fittest'
- I would drive 100 miles and I would drive 100 more just to be the man that drove 200 miles to... hit the enter key
"Now I do security consulting work," he told us, "It is amazing how the adventures of my youth helps me make sure that my clients are secure –something something Floor-to-Ceiling mesh securing data centres..."
For some reason we can't shake the image of Bruce Willis clambering through a duct in Die Hard from our minds after reading about Dave's adventures. Although the line "You asked for miracles? I give you FreeBSD" doesn't have quite the same ring to it.
Tell us about the time you had to channel your inner Tom Cruise in the name of DNS via an email to Who, Me? ®