C'mon SPARCky, it's just an admin utility update. What could possibly go wrong?

Now, where was I? [tappity tap] ... BALLS!

Who, Me? Hey hey hey, it's Monday! The new week is but a caffeinated beverage away. Come join us in celebrating another Register reader's flirtation with career-ending disaster with a morning dose of Who, Me?

It was the late 1980s, and our contributor, fresh out of university, had inexplicably landed a job as system administrator in a large academic institution.

"Scott", as we're going to call him, "was pretty much in charge of day-to-day tasks on several Sun servers, including," he modestly added, "one of the first SPARC-based systems in UK."

Scalable Processor Architecture (SPARC) was the brainchild of Sun and Fujitsu and a relatively successful RISC system. It was, even by the standards of Sun, exotic hardware back in the late 1980s.

On the day in question, Scott had been upgrading some in-house administration utilities. "This necessitated logging in as root, running some scripts and then tidying up using the rm command."

He was a cautious fellow, and had created a "scratch" directory with his normal user ID in which to perform the work. Once the upgrade was done, he would simply clear it down. Sensible.

Scott had just finished the compilation of the source and installed the updates when the phone rang with some pesky user needing a printer problem resolving. Scott had to su to root to fix the issue before running a rm -r * in his scratch directory to tidy up after the earlier upgrade.

And tidy up it did, just not quite how Scott had imagined.

"I forgot to exit my root shell so I ran this command from the root directory as the root user."

Let the person who has never forgotten where they were before running something horrifically destructive cast the first stone...

"After mere moments, my spider-sense started tingling and I stared in disbelief and growing horror at my VT220 terminal as the realisation of what I had done hit me and I frantically hit Control-C."

Luckily, Scott was distracted from the bowel-loosening terror within seconds as the phone began to ring off the hook. Hordes of angry academics and students bombarded him with the same question – just what in blue blazes was happening?

Naturally, he took the only avenue available to him. "I called my local Sun Microsystems support line to log a fault with the disk drive." AKA: he told a little lie in order to buy himself some thinking time.

The news that the borkage was a "hardware fault" calmed things down somewhat and Scott was able to start dealing with the smoking wreckage of the file system after managing to get the server booted from a QIC-150 cartridge.

As it turned out, backups had been taken. It took a few sweaty hours to restore everything, and then a few more hours before Scott realised that he also had to reinstall the boot block. But it was done and, impressively, done in time to permit an on-time exit (we imagine to a nearby hostelry to calm shattered nerves).

But there was still the question of the lie "hardware failure". Not a problem: "I had deliberately disconnected the Ethernet cable because I did not want the server to appear functional until the Sun hardware engineer turned up."

The following morning, with the server still, er, "down", Scott got a call from the inbound Sun engineer asking him to "put the kettle on". Knowing that his ruse would not survive an encounter with an engineer presented with a fully functional server, "I recounted my tale of woe to him apologising profusely for wasting his time."

The piggy bank of favours was cracked open: Scott and the engineer had a pretty good working relationship stemming from Scott's willingness to overlook an occasional miss of the contracted call-out times because, hey, it was academia and "it was rare that a problem could not wait an extra few hours and allow the engineer to get home at a reasonable time and attend to my issue the following day."

Stir in the fact that changing a 1Gb drive back in the day was a two or three-person job, and the engineer was more than happy to play his part in the ruse.

Network attached, "hardware fault" rectified, and Scott was surely the hero of the hour rather than the villain of the piece.

He had another brush with the back-up tapes in the 1990s, but that is another story.

And that is why "I should never have root access..."

Ever felt the stomach-dropping lurch of an unexpected directory delete? This hack has certainly watched in horror as a folder of production databases disappeared into a Netware wasteland. How about you? Send an email with your confession to Who, Me?. ®

Similar topics

Broader topics

Other stories you might like

  • September 16, 1992, was not a good day to be overly enthusiastic about your job
    If I get in early and work hard, everyone will notice, right?

    Who, Me? "The early bird trashes the business" is a saying that we've just made up, but could easily apply to the Register reader behind a currency calamity in today's episode of Who, Me?

    Our hero, Regomized as "Mike", was working as a "data entry operative" for a tourism company in 1992. The company ran bus tours to the then brand-new EuroDisney, parent company of Disneyland Paris (now the most visited theme park in Europe), which had opened earlier that year.

    Mike was an eager beaver, his youthful naivete having convinced him that if he worked extra hard, came in extra early, and kept the in-tray clear, then his efforts would be both noticed and rewarded with promotion and a bump in pay.

    Continue reading
  • An international incident or just some finger trouble at the console?
    All routers are equal, but some are more equal than others

    Who, Me? Welcome to an edition of Who, Me? where some configuration confusion left an entire nation cast adrift.

    Today's story is set in the early 2000s and comes from a reader Regomized as "Mikael" who was gainfully employed at a European ISP. The company had customers in multiple countries and Mikael's team was responsible for the international backbone.

    "Us senior network engineers were widely regarded as consummate professionals," he told us, before adding, "at least amongst ourselves."

    Continue reading
  • A discounting disaster averted at the expense of one's own employment
    I know what this process needs: Microsoft Access!

    Who, Me? A tale of discounts and process improvement via the magic of Excel, Access and a fair bit of electronic duct tape we imagine. Welcome to Who, Me?

    "James" is the Regomized reader of record today, and continues the theme of running the risk of doing a job just that little bit too well with an ancedote from the end of the last century involving his first job out of university, at a certain telecommunications giant.

    The job involved a process of calculating the discount received by big customers (the ones with multiple branches). "For the life of me I can't remember what the main DB was called," he told us, "but it was the old style green writing on a black screen that took forever to download the necessary data."

    Continue reading
  • In IT, no good deed ever goes unpunished
    When being helpful can mean being shown the door

    Who, Me? Going above and beyond in IT can sometimes lead to also going directly out of the door, as one Register reader found when discovering that sometimes efficiencies can be less than rewarding.

    A reader Regomised as "Will" told of us his days working at a now-defunct company that produced large telephone switches. In those days whenever a major software revision occurred, customers were expected to send in their configurations and Will's group would merge them into the latest and greatest. A new load would then be returned to the customers.

    It was not a fun process, not least because of constant hardware and software failures during the merge process. "When I first started, there was a constant grumble about how unreliable the machine used for the merging was," Will told us.

    Continue reading

Biting the hand that feeds IT © 1998–2022