Watt's the worst thing you can do to a datacenter? Failing to RTFM, electrically
A subtle change to a vital piece of equipment almost derailed a major project
Who, me? Welcome once again gentle reader to another Monday morning, and with it an instalment of Who, Me? in which Reg readers cushion your entry to the working week with tales of things going not quite right.
This week meet "Nikolai" who spent some time in the mid-1980s working on some fancy bespoke communications solutions. On the project in question Nikolai was employing two very expensive microprocessor emulators – because you need one at each end to demonstrate a comms solution, of course.
Underlining the rarity and expense of these emulators, Nikolai tells us there were only three of them in the country at the time, and he had two thirds of the supply. He doesn't spell out what the other one was doing – probably not comms, we're guessing.
The project had not gone well and had been delayed several times by the time this story starts, when Nikolai's client delivered an ultimatum: show us a demo or the deal would be cancelled.
The day before the scheduled demo, one of the emulators began behaving strangely – "not obeying its own instruction set at all," Nikolai lamented.
- Sure, give the new kid and his MCSE power over the AS/400. What could possibly go wrong?
- Polishing off a printer with a flourish revealed not to be best practice
- Lesson 1: Keep your mind on the ... why aren't the servers making any noise?
- The price of freedom turned out to be an afternoon of tech panic
The search began to discover the cause of the problem, and a potential culprit was found: "the 5V rail on the emulator was checked and found to be only 2.5V" according to Nikolai. What could cause such a strange anomaly? Nikolai vaguely recalled a colleague fiddling with a power supply earlier in the day, and investigated.
Sure enough, the attached power supply was set to 2.5. Why? A prank maybe? Not a very funny one. Nikolai cranked the power supply back up to 5.
At which point the magic blue smoke escaped from the emulator. One of the two emulators Nikolai had. Of the three in the country.
It turned out that the setting on the power supply was not at fault at all. A broken wire in one of the cables that connected two boards within the emulator was the culprit.
So why had Nikolai's colleague fiddled with the power supply? Well – you'll laugh – it turned out he'd switched the display on the power supply from volts to amps, and Nikolai hadn't noticed. The power supply had been correctly outputting 5V at 2.5 amps, or 12.5W. By doubling the amps, Nikolai had sent 25W to the emulator, blowing its little mind.
Thankfully, there was that third emulator in the country, which was very hastily obtained. Nikolai did what he could with one emulator overnight and, when the spare arrived, all worked correctly. And Nikolai learned a lesson about reading the units on a power supply.
If you've ever jeopardized a major project with a moment's inattention, we want to hear from you. Send an email to Who Me? detailing your misadventure, and we'll make that moment last forever.