Workload written by student made millions, ran on unsupported hardware, with zero maintenance
Nobody minded for 20 years or so, until another student took action
On Call Many a Friday arrives with a feeling that the previous four days of toil occupied more than 96 hours, which is why The Register always marks the day with a new instalment of On Call, our reader-contributed tales of fun times delivering tech support.
This week, meet a reader we will Regomize as "Rik" who shared the story of how, during his undergraduate computing studies, he scored a student placement in the IT support team of a large UK company that traded 24/7 on various financial markets.
This was the sort of environment in which wobbly tech could cost millions in a moment. So Rik was a little surprised when, on top of the usual student chores of fetching coffee and doing drudgework, he was also asked to look at "the platform hosting a real-time graph used to inform the shift traders." The graph wasn't critical, but sometimes revealed market intelligence that proved extremely profitable and impactful.
And that platform had problems.
"I was told that the hardware/infrastructure team had concerns about supportability and there had been no changes for over two decades," Rik recalled.
His next stop was therefore to visit the infrastructure and hardware team, which informed him the graphing code worked across a single Sun SPARCstation 2 and a PC.
Folks from the engineering team were able to find the relevant Java and C code, and – after examining it and chatting to colleagues – Rik learned its purpose was to monitor a single real-world variable and graph it. He discovered that a previous student on placement wrote a driver for a General Purpose Interface Bus (GPIB) for the sensor that sucked in the variable, and a Java http servlet to read it and share it.
That code was a proof of concept that ran on the student's desktop. Nonetheless, traders had seen this work, profited from it, and begun to rely on it.
"The idea was it would be retired or productized at some point," Rik learned.
- Cheapest, oldest, slowest part fixed very modern Mac
- Police ignored the laws of datacenter climate control
- Techie labelled 'disgusting filth merchant' by disgusting hypocrite
- Techie wasn't being paid, until he taught HR a lesson
Which didn't happen. But the hardware it ran on was moved into a proper datacenter.
Because this was not an official workload, it wasn't powered from the datacenter's main power source – its power cables snaked under the floor and emerged next to a regular wall socket.
The same sockets used by cleaners when they vacuumed the facility.
Nobody knew why the app failed at the same time every week. They just lived with it until Rik made inquiries about the situation.
As his inquiries progressed, he was asked if code for the app existed, and if it was worth persisting with, or could be replaced with an off-the-shelf system.
Rik surveyed the market, and found solutions that could do the job – but were massively overengineered and overpriced.
He ended up modernizing and re-platforming the code – which wasn't vastly difficult because C and Java hadn't changed that much in 20 years. Adding USB support to replace the GPIB was a challenge, as was readying it to run in a VM with proper failover.
But Rik was able to do the job, and the app was finally productized!
And it only took two students, working twenty years apart, to finish the job!
Have you ever been asked to fix unofficial apps, written one yourself, or delivered mission-critical services while still a student? If so, click here to send On Call an email and we'll consider your story for a future instalment.
Don't be shy – we always need more yarns to consider. And remember: you'll always be anonymous. No story too silly, but we do try to avoid smut. ®