Sysadmin sank IBM mainframe by going one VM too deep

Tried to blame it on a bug, but logs don't lie


Who, me? Welcome to another glimpse inside the dark-curtained (in)box that is "Who, me?" – El Reg's confessional column in which readers seek penance for sins of the past.

This week, meet "George", who many years ago was one of two sysadmins in the office products division of a three-letter computer company. And was bored. Bored, bored, bored!

Which was when he got curious about the virtualization capabilities of the IBM mainframe he and a few dozen other developers toiled on every day.

"Each developer had their own virtual machine running IBM's mainframe OS, CMS," he said. "Normally, CMS saw everything. But you could issue commands to the VM hypervisor by prefixing your input with a special character, the default being a hashtag #.

"For instance, to shut down VM – and thus the whole machine – the command was '#cp shutdown'." George, as a sysadmin, could do this to all instances of VM. The rest of the crew could only ditch their own stuff.

So there's George in his office, twiddling his thumbs and mulling over the concept of VM, when "the idea occurred to me that VM created virtual machines".

"So it might be possible to boot VM 'in a virtual machine'."

Moving his thought experiment into the real world, George gave it a whirl...

"It actually worked," he said. "I set the second level command character to ';' and proceeded to create a third level virtual machine inside the second level."

But that raised another question for our curious George... "Can I boot VM in the third level machine?"

The answer? Yes. "That worked and it had a command character of '%'," he said, adding: "Which begs the question..."

hazmat suit wearing person

Tech team trapped in data centre as hypoxic gas flooded in. Again

READ MORE

At this point cunning readers can probably see where this is going, because George eventually tired of nested virtualization and shut down his VMs.

"But, of course, my fingers automatically typed '#cp shutdown'; rather than use the commands I'd assigned to the intermediate machines," said George, the panic still palpable all these years later and over the medium of email.

"So, VM did what it was commanded to do and shut down the REAL machine, killing off all 72 developers at once."

Faced with a rather sticky situation, George did what all good sinners do, and looked to shift the blame. In this case, he said, it was an "unknown bug" – but as is often the case, there's always someone watching.

"The other developer could see my shutdown command in the log files," confessed George. "He was not happy with me that day."

Have you ever virtually killed your team's productivity? Has curiosity nearly killed your cat? Absolve yourself by writing to the priest of Who, me? ®

Similar topics


Other stories you might like

  • Despite global uncertainty, $500m hit doesn't rattle Nvidia execs
    CEO acknowledges impact of war, pandemic but says fundamentals ‘are really good’

    Nvidia is expecting a $500 million hit to its global datacenter and consumer business in the second quarter due to COVID lockdowns in China and Russia's invasion of Ukraine. Despite those and other macroeconomic concerns, executives are still optimistic about future prospects.

    "The full impact and duration of the war in Ukraine and COVID lockdowns in China is difficult to predict. However, the impact of our technology and our market opportunities remain unchanged," said Jensen Huang, Nvidia's CEO and co-founder, during the company's first-quarter earnings call.

    Those two statements might sound a little contradictory, including to some investors, particularly following the stock selloff yesterday after concerns over Russia and China prompted Nvidia to issue lower-than-expected guidance for second-quarter revenue.

    Continue reading
  • Another AI supercomputer from HPE: Champollion lands in France
    That's the second in a week following similar system in Munich also aimed at researchers

    HPE is lifting the lid on a new AI supercomputer – the second this week – aimed at building and training larger machine learning models to underpin research.

    Based at HPE's Center of Excellence in Grenoble, France, the new supercomputer is to be named Champollion after the French scholar who made advances in deciphering Egyptian hieroglyphs in the 19th century. It was built in partnership with Nvidia using AMD-based Apollo computer nodes fitted with Nvidia's A100 GPUs.

    Champollion brings together HPC and purpose-built AI technologies to train machine learning models at scale and unlock results faster, HPE said. HPE already provides HPC and AI resources from its Grenoble facilities for customers, and the broader research community to access, and said it plans to provide access to Champollion for scientists and engineers globally to accelerate testing of their AI models and research.

    Continue reading
  • Workday nearly doubles losses as waves of deals pushed back
    Figures disappoint analysts as SaaSy HR and finance application vendor navigates economic uncertainty

    HR and finance application vendor Workday's CEO, Aneel Bhusri, confirmed deal wins expected for the three-month period ending April 30 were being pushed back until later in 2022.

    The SaaS company boss was speaking as Workday recorded an operating loss of $72.8 million in its first quarter [PDF] of fiscal '23, nearly double the $38.3 million loss recorded for the same period a year earlier. Workday also saw revenue increase to $1.43 billion in the period, up 22 percent year-on-year.

    However, the company increased its revenue guidance for the full financial year. It said revenues would be between $5.537 billion and $5.557 billion, an increase of 22 percent on earlier estimates.

    Continue reading

Biting the hand that feeds IT © 1998–2022