When we asked how you crashed the system we wanted an explanation not a demonstration
One dumb downtime-inducing panic is survivable. The second is a career-killer
Who, Me? Once again, gentle readerfolk, it is time for the Reg's weekly confessional – Who, Me? in which we recount readers' technical anti-triumphs. This one follows on from recent tales of big red buttons, and goes one better.
This week meet someone we'll Regomize as "Hal", who recounted for us a tale from the epoch of the mainframe, when glass cathedrals full of sacred boxes stood protected and air-conditioned. In this faraway time, none but the appropriately credentialed could pass through the sliding doors and gaze upon the IBM-emblazoned bezels of these mighty boxes.
Well, the appropriately credentialed and their interns. More on that in a sec.
One night, Hal tells us, a disaster happened. The mainframes had, all at once, shut down. The terminals to which they spoke had shut down. The peripherals and various doo-dads with which they doo-did had all shut down. The entire business, effectively, had ground to a halt.
Support crews were awoken and called to the site to reboot the system and diagnose the fault. Senior technical staff were alerted that a meeting would be held at 07:00 AM to figure out what had gone so very very wrong.
At that morning meeting, the support crew determined that the system had not crashed at all. In fact the machines had not encountered any fault and had behaved exactly as they were designed to in the event of an emergency shutdown.
A what's that did you say? Emergency shutdown? What emergency shutdown?
It transpired that an intern, who should never have been alone in the glass cathedral in the first place, had spotted an "ambiguous message" on one of the consoles and panicked. Fearing an emergency, he fled the room and slapped the big red button on his way out.
- Server installer fails to spot STOP button – because he wasn't an archaeologist
- Programming error created billion-dollar mistake that made the coder ... a hero?
- How not to test a new system: push a button and wait to see what happens
- Job 1: Get the boss on the network. Job 2: Figure out why Job 1 broke the network for everyone else
This was, of course, not the protocol. Clearly, something had gone wrong in the training process to put an intern in such a position that they hit the big red button instead of grabbing a senior member of staff to explain the "ambiguous message" on the terminal, right?
So the intern was in big trouble, as was whoever was supposed to be in the room with said intern. But senior people needed to know exactly how this happened so it could't happen again.
They asked the intern to walk them through what had happened, what had he been doing, and all of the steps he had taken.
Which he did – in great and demonstrative detail for every step.
Even the one when he slapped the big red button.
As the mainframes once again shut down, taking with them all the work the support crew had done through the night getting everything up and running again, the supervising tech "lost all sense of decency" as he lifted the intern bodily and placed him in a nearby garbage receptacle.
Needless to say this incident became the abrupt end to an internship, though thankfully he suffered no serious injuries. Indeed he was taken on as a programmer – possibly, Hal suspects, to head off any legal action stemming from the whole tossed-in-the-trash thing.
Have you ever gone a bit too far in investigating a tech fault? Fixed something only to break something else? Don't keep these stories to yourself – the green thing to do is to recycle them with an email to Who, Me? We promise not to throw them in the trash.
Editor's Note: Who, Me? will resume on January 9.