Dumpster diving to revive a crashing NetWare server? It was acceptable in the '90s

Broken pencil + ball of stolen Blu Tack = IT joy

Who, Me? Today is Monday the 58th of March 2020. As a service to stop the days blurring into one, take a moment to enjoy a tale of NetWare, Blu Tack and a broken pencil in today's Who, Me? reader confession.

This time we return to the 1990s, and to a region of Australia that our protagonist, a person we'll call "Howard", described as "a three-horse town where everyone knew who's dating everyone else's sister".

Top marks for the use of the word "dating" there, Howard.

Howard was plugging through the end of his education and working part time at a local computer store. His job mainly entailed popping out to deal with customers bigger than those that might be merely toiling along with word processing on a Windows 3.11 box.

His confession starts at what he told us was "a decent-sized financial institution" having trouble with NetWare 3.11. The system in question had to log (for legal reasons) card transactions that had been sent to an AS/400 for batch processing. While IBM's metal kept chugging along, at some point between 2 and 6am the NetWare server would lock solid.

While not privy to what this logging app was doing, "given NetWare 3.11 was co-operative multitasking and the custom logging app was a NLM [NetWare Loadable Module] and ran in Ring0, it was most likely the culprit crashing the whole show... but it was out of scope."

NLMs turned up in NetWare 3.x and were the bane of many an administrator's life. The handy binaries could do all manner of things, such as implementing hardware drivers, but bad behaviour could take down a server or make it unresponsive. Things improved in version 4 and 5, but the smell of instability lingered.

Wayne's World product placement visual gag (Wayne eats Pizza Hut pizza, displays branding, while talking about product placement)

Getting a pizza the action, AS/400 style


The NLM in question here was off the table so Howard had to come up with another solution.

"The usual procedure," he recalled, "was that one of the locals would arrive for work the next morning and they'd eyeball the server."

No flickering of the hard disk LED after a few minutes meant the reset button needed a prod and the night's transactions manually reconciled. Fortunately, the fact the crashes always occurred in the small hours meant that there weren't too many but...

"Management," he remembered,"would have an early morning meeting containing a minor blame-storm."

This gradually escalated, as the frequency of the incidents increased, into a full blown "rage-fest," according to Howard, which was "shoved through the phone line at us."

An immediate workaround was demanded for the reboots with a fix for the cause needed at some undefined point in the future.

"It already felt like we were circling the drain," sighed Howard.

However, there was this newfangled thing called "Linux" doing the rounds with some flaky NetWare Link Services Protocol (NLSP) support. It wasn't complete, but would allow a type of "Who's on the LAN" broadcast. Parse the response, and it would do for an "Are you alive?"...

"The penny dropped with a resounding clang."

At this point, Howard transformed from being merely a computer shop part-timer with a penguin fetish into a hero for all time.

A battered old 486 box was pulled from storage and had Linux installed. A bit of Perl was used to check if the NetWare server was still alive; if there was no response within a set period then a reset was needed.

And the reset?

"I found a broken CD-ROM mechanism along with its Mitsumi 8bit ISA card in a dustbin, and with a generous amount of stolen Blu Tack sourced from a few dozen or so cubicles, a broken pencil and some tape, this was all added to the front of the non-reading CD-ROM drive as a 'finger'."

You can guess what the next step was.

"When the server would not respond, the Perl script (running as root) would eject /dev/cdrom, send an email to double-check for dropped transaction log entries, and go to sleep for 20 minutes till the server came back up."

The final flourish was the positioning of the old 486 in front of the NetWare tower server. With a pair of Yellow Pages, and some copies of a long-defunct print publication for fine-tuning of height, the CD-ROM tray and its "finger" were perfectly positioned in front of the reset button.

Better still, Howard ensured there was not enough space for the CD-ROM tray to fully open, so having jabbed the reset button it would retract: "Ready to strike again."

"GENIUS!" he told us, modestly.

Despite a slap on the back and the promise of a free beer on Friday, Howard soon moved on to other things and thought no more about his hack until 1999 when a friend told him about "the most dodgy thing [he'd] *EVER* seen in this accounting and finance firm that had a crappy old 486 rebooting a server tower via a pencil on the front of the CD-ROM tray."

Howard's hack had spent half a decade toiling away.

"I laughed," he said, "and drove on quietly."

Can you top Blu Tack, a pencil and broken CD-ROM? Share your tales of blatant bodgery from days long gone (or all too recent). Discretion (and a dodgy pseudonym) assured via an email to Who, Me? ®

Similar topics

Broader topics

Other stories you might like

  • An international incident or just some finger trouble at the console?
    All routers are equal, but some are more equal than others

    Who, Me? Welcome to an edition of Who, Me? where some configuration confusion left an entire nation cast adrift.

    Today's story is set in the early 2000s and comes from a reader Regomized as "Mikael" who was gainfully employed at a European ISP. The company had customers in multiple countries and Mikael's team was responsible for the international backbone.

    "Us senior network engineers were widely regarded as consummate professionals," he told us, before adding, "at least amongst ourselves."

    Continue reading
  • A discounting disaster averted at the expense of one's own employment
    I know what this process needs: Microsoft Access!

    Who, Me? A tale of discounts and process improvement via the magic of Excel, Access and a fair bit of electronic duct tape we imagine. Welcome to Who, Me?

    "James" is the Regomized reader of record today, and continues the theme of running the risk of doing a job just that little bit too well with an ancedote from the end of the last century involving his first job out of university, at a certain telecommunications giant.

    The job involved a process of calculating the discount received by big customers (the ones with multiple branches). "For the life of me I can't remember what the main DB was called," he told us, "but it was the old style green writing on a black screen that took forever to download the necessary data."

    Continue reading
  • In IT, no good deed ever goes unpunished
    When being helpful can mean being shown the door

    Who, Me? Going above and beyond in IT can sometimes lead to also going directly out of the door, as one Register reader found when discovering that sometimes efficiencies can be less than rewarding.

    A reader Regomised as "Will" told of us his days working at a now-defunct company that produced large telephone switches. In those days whenever a major software revision occurred, customers were expected to send in their configurations and Will's group would merge them into the latest and greatest. A new load would then be returned to the customers.

    It was not a fun process, not least because of constant hardware and software failures during the merge process. "When I first started, there was a constant grumble about how unreliable the machine used for the merging was," Will told us.

    Continue reading
  • An early crack at network management with an unfortunate logfile
    It's a backronym, right?

    Who, Me? Come with us on a journey back to the glory days of Visual Basic 6, misplaced enthusiasm and an unfortunate naming incident. Welcome to Who, Me?

    Today's tale comes from a reader Regomised as "Stephen", who was working in the IT department of a Royal Air Force base. "My duties were many," he told us, "from running daily backups of an ancient engineering system using (I kid you not) reel-to-reel tapes to swapping out misbehaving printers."

    This being the early 2000s, his boss loaded up our hero with more tasks. He could change printers and tapes, so Visual Basic (and its bedfellow, Access) should present no problem.

    Continue reading

Biting the hand that feeds IT © 1998–2022