On-Call Welcome to yet another instalment of On-Call, The Register's week-ending column in which we share readers' stories of extreme sysadminnery performed under extreme duress.
This week, meet “Lee” who told us he used to work for a British government department that operates lots of offices overseas, usually in embassies.
Lee was On-Call for plenty of them and could be flown out on little notice to sort out technology messes.
One such trip involved “four of us sitting around a cage full of monkeys, feeding them fruit” because one of the simians had swallowed a USB drive that contained classified files.
Another trip involved a junior minister, a fondleslab damaged by a camel, and a sprint to the Middle East to retrieve data before a high-level meeting.
Lee's longest story concerned what he described as a “UCS” – that's an Unfortunate Civil Servant - who, while posted to a developing African nation decided it would be fine to take his laptop to a local and totally-not-cleared-for-working-on-government-devices tech support provider.
The local examined the UCS' laptop and declared it needed an upgrade from Windows XP to Windows 7. At which point the UCS handed over the USB dongle that is necessary to get government computers to do anything.
“Handing over the dongle was tantamount to a major breach of the Official Secrets Act,” Lee told us. The UCS thought he could get away with it by staying in the same room as the laptop and the local specialist.
So he popped the USB dongle into the laptop and booted it.
The local specialist then produced a home-burned Windows 7 Home Edition disk and tried to install it. Which didn't work, so he opened the laptop and replaced the shiny new SSD with an old 2.5” 40Gb he had at hand.
Windows 7 now installs and the UCS found himself in possession of “an up to date version of Windows 7, albeit with a pirated serial number, but with no office suite and no access at all to the (heavily) encrypted office network.” The local “specialist” next produced a disk full of drivers, a pirate Microsoft Office disk, installed the lot and presented it to the UCS.
Who then connected it to the embassy network.
“Slight exaggeration, but about two seconds later an alert has gone off in Whitehall,” Lee told us. Next: “a swift exchange of internal 'WTF just happened' emails.”
“By the time my department has been prodded into action our man in Africa was being screamed at by various civil servants, who in turn are being yelled at by their superiors and, presumably, right up the chain to where the politicians take it in turn to yell at people.”
Three hours later, Lee and two colleagues were told they were on the next plane to Africa. On arrival the next day they restored the original hard disk, upgraded all of the secure USB dongles, ran a four-hour scan of the embassy network (it was clean) and then tried to figure out why the UCS had been so silly.
“The UCS had accidentally hit WiFi button on the side of the laptop and turned it off,” Lee told us, adding that the UCS “received an official scold, which used to be a one year advisory not to promote on their career folder.”
Lee left this job four years ago, but says he still has frequent flyer points a-plenty even this far down the track!
Have you ever jetted in to fix a messier mess than those Lee un-messed? If so, write to On-Call and we might tell your tale in this slot next Friday. ®