BOFH: When the Sun rises in the West and sets in the East, only then will the UPS cease to supply uninterrupted voltage
Until that time, if it ain't broke … oh god, someone call the ambulance
Episode 8 "Eeeeeeasy does it …" I say to the PFY, "almost there …"
"What're you guys doing?" Richard asks, blundering into the Server Room without permission.
That's Richard — not Rich, not Richey, and definitely not Dick — a new consultant, engaged to make the company more "agile".
Dick is 26.
The year Richard was born I set a company record for how fast a VAX 11/780 would roll down the road from our building. I could've got a world record if it hadn't been for that cyclist.
"What do you mean, Dick?" the PFY asks.
"Richard," Richard says.
"But what are you doing?"
"We're replacing the batteries on this UPS," the PFY replies.
"Because the old batteries are faulty."
"Wouldn't it be better to buy a new one?"
"Yes and no, Dick," I say.
"Richard. And why yes and no?"
"We could get a new one in a couple of days, but it would not be quicker to shut down all the gear connected to the old UPS, and then try to power it all back up again later."
"So you're putting new batteries into an old UPS instead of using the server room UPS?" he gasps, spotting a low-hanging efficiency.
"Another good question, Dick …"
"Sure. See, this UPS was built more than 12 years ago when you were starting high school and is a 3-Phase, 3000VA, 'small form factor', rack mount UPS — with hot-swap lead acid batteries. No one makes these any more — or at least makes them this good."
"I still don't understand why you can't shut everythi—"
"Everything downstream of this is ancient kit has been running LITERALLY for years and CANNOT be powered off. We're talking hard drives with stiction, gear with weird voltage cooling fans that have to be hand spun to boot without a BIOS fault, a BMS which NO-ONE understands and a bloody fax machine which has not received ONE SINGLE FAX FOR MORE THAN FIVE YEARS BUT WHICH THE CEO HAS INSISTED MUST BE KEPT RUNNING UNTIL WE REPLACE THE CORPORATE STATIONERY."
- BOFH: Despite the extremely hazardous staircase, our IT insurance agreement is at an all-time low. Can't think why
- BOFH: I'm so pleased to be on the call, Boss. No, of course this isn't a recording
- BOFH: But we think the UK tax authorities would be VERY interested in how we used COVID support packages
- BOFH: Postman BOFH's Special Delivery Service
"We don't even have the phone line any more," the PFY adds.
"But I still don't see why it can't be on the main UPS?"
"A) Because of that whole shutdown debacle, but also B) the server room UPS has modern safety features like RCDs and floating earth detection. Ancient pieces of kit, on the other hand, have what you might call a laissez-faire attitude towards safety and will still be delivering uninterrupted voltage long after the ambulance has left and the smell of burnt flesh is finally vented from the room. Clipping the earth leads off our kit and slapping them into an out-of-the-way cabinet that's mounted on a thick rubber blanket was just a belt-and-braces thing."
"So you're just keeping all the old stuff running?"
"Why don't you just replace all the old kit?"
"The fear of the unknown," I reply. "That BMS box — no one has a clue about it. Don't know what it does, don't know how it does it. Hell, we don't even know where those 100-core cables out the back of it go. All we know is if it's powered down, the building starts to smell of damp basement and a big fan on the roof starts spinning."
"And the sewer stops emptying properly," the PFY adds.
"But surely you could replace it?"
"That's a good suggestion, and one a few people have made in the past — though for most of those people the thought of swimming around in 15,000 litres of human waste is enough to convince them to leave well enough alone. Other people, well, they have to sit the practical exam …"
"Surely you should find out what it does?"
"Again, great point Dick, but I don't think that any living person knows what it all does. Except for the fax machine — we know that does nothing."
"So you just going to keep it all running?"
"Indeedy," I say, as the PFY and I slide the new battery pack into the machine. "3 … 2 … 1 … >push< >click< >beepbeep!< … And we're done for another couple of years!"
"Don't you see this as an opportunity to upgrade?" Richard asks.
"Ah, youth!" I say. "So short on practical experience. I probably have eight to ten more battery changes before I retire, so why rock the boat? Today-Simon says that all problems will be Tomorrow-Simon's problem, and Tomorrow-Simon knows how to fake a workplace accident to get a month off to recuperate."
"But we could fix things!" he gasps, pointing to a corner of the server room. "All those manuals over there, surely one of them documents this stuff."
"Oh, we've not been over to that part of the room for years!" I blurt.
"But you could find out how to replace all that stuff and free up some time," he gasps.
"But I like the battery place. They know us there."
"They have homemade chocolate hobnobs," the PFY adds.
"You're … keeping all this running for biscuits … every TWO YEARS?"
"We can get it down to about 11 months if we let the batteries get really low before putting the power back on," I say.
"Look, we'll check these manuals out, if we can find some documentation we could—" >KZERT!<
"Oh, did we not tell him it was THAT cabinet?" I ask.
"I'll call the ambulance," the PFY says.
"And I'll put the extractor fan on." ®