Oh no, you're thinking, yet another cookie pop-up. Well, sorry, it's the law. We measure how many people read us, and ensure you see relevant ads, by storing cookies on your device. If you're cool with that, hit “Accept all Cookies”. For more info and to customize your settings, hit “Customize Settings”.

Review and manage your consent

Here's an overview of our use of cookies, similar technologies and how to manage them. You can also change your choices at any time, by hitting the “Your Consent Options” link on the site's footer.

Manage Cookie Preferences
  • These cookies are strictly necessary so that you can navigate the site as normal and use all features. Without these cookies we cannot provide you with the service that you expect.

  • These cookies are used to make advertising messages more relevant to you. They perform functions like preventing the same ad from continuously reappearing, ensuring that ads are properly displayed for advertisers, and in some cases selecting advertisements that are based on your interests.

  • These cookies collect information in aggregate form to help us understand how our websites are being used. They allow us to count visits and traffic sources so that we can measure and improve the performance of our sites. If people say no to these cookies, we do not know how many people have visited and we cannot monitor performance.

See also our Cookie policy and Privacy policy.

This article is more than 1 year old

Dev's telnet tinkering lands him on out-of-hour conference call with CEO, CTO, MD

'Please stop... you're doing something to the radios'

Who, me? Welcome all, to the merry world of Who, Me?, our weekly trip down memory lane for techies who want to get something off their chest.

This week, we hear from "Gavin", who was working at an ISP that had a few thousand point-to-multipoint radio links and had been asked to write a script to backup their configs.

"They already had a bare config backup going every night, but the bosses wanted to get some extra diagnostics out as well, which involved running a telnet command on the units," he said.

"They were rough 'n' ready Russian-made things with their own firmware, so the commands were pretty straightforward and the same across base stations and CPEs."

Gavin wrote the script in Perl – "the telnet module saw good use in other projects just fine" – and set about testing it on a few hundred devices at a time.

Man types something into Mac while sipping a glass of lemon water. Not a brilliant idea. Photo by SHutterstock

College PRIMOS prankster wreaks havoc with sysadmin manuals

READ MORE

It looked fine for a handful, Gavin said, so he started forking the process so it was quicker and would finish through the night.

"Then I started noticing my script kept hanging, seemingly randomly."

Gavin said he would kill the process, make a change to his telnet login or commands, and then try again.

"This seemed to progress it every time to a larger batch of devices."

Gavin kept doing this, out of hours, for about 20 minutes, until his personal phone rang.

"I picked up to realise I was in a conference with the MD (who was fairly technical, especially around these radios) the operations director, the CTO and the CEO – essentially all top brass," he recalled.

"Rather politely (compared to their usual behaviour during priority one incidents) they asked whether I was 'doing something' with the radios."

Gavin explained that they'd assigned him this project, to which he was told to "please stop for now" as it was "causing problems with some of the units".

After facing the walk of shame the next morning, "under the glaring eyes of colleagues and poor installation technicians about to be dispatched or returning from jobs", Gavin found the source of the problem.

"Because the firmware on a lot of the devices wasn't updated for years, their telnet client had a number of quirks, such as it wouldn't ask for a password or would only ask for a password," he said.

"In some of those cases, my script would happily keep sending the password/user in a loop, even if it didn't see the right prompt, effectively flooding the device."

And, if the extra traffic wasn't enough, the CPU would crash and then they had to dispatch the technicians – to four different cities in the UK from just one base. During the night. To dozens of sites.

"I had a stern talking to by the bosses but received no punishment," Gavin said.

He added that he remained an employee for years, as "adaptability to shifty infrastructure and business knowledge was what kept us going as a business. Or maybe because I was the only dev at the time…"

Gavin says his story became part of his company's lore, told at dinners and parties. Has anything you've ever done ended up outlasting your time at a firm?

Tell Who, Me? about it by emailing us here. ®

 

Similar topics

Similar topics

Similar topics

TIP US OFF

Send us news


Other stories you might like