Bug that wiped customer data saved the day – and a contract

Ignorance really was the way to achieve bliss

Who, Me? Welcome to another working week, and another installment of "Who, Me?" – a weekly reader-contributed column that unearths your errors and reveals how you rebounded afterwards.

This week, meet a reader we'll Regomize as "Caleb" who told us of the time he worked for an internet service provider that had problems with one of its biggest clients.

"They constantly complained about their network being slow and their internet access being even slower," Caleb told The Register.

"As this was one of our largest clients at the time, my boss grabbed me and we visited the customer."

The first thing Caleb and his boss saw was a collection of 3Com routers the client used to connect to the ISP and their other offices over IPX.

"I looked at the config and noticed the customer did not have a default route set," Caleb admitted. He wasn't sure if that was the problem, so he made some changes he thought might be useful.

The router Caleb worked on then rebooted, which he expected. But when it restarted, its previous configuration was gone. This was unexpected, but Caleb later discovered it was a known bug with the routers, perhaps due to dodgy NVRAM.

Caleb and his boss decided to restore the IP connection and figured the IPX side of the network was the client's problem.

Once they finished the job, Caleb and his boss noticed that the client's internet access had become blazing fast.

"We showed them proof that their own IPX networking setup was the reason for their slow internet. They were very happy and remained a customer for several years."

Looking back on the job, Caleb thinks it was his mistake that saved the day.

"If that router hadn't eaten its own configuration, we would have lost them as a customer," he wrote.

Have your mistakes turned into miracles? If so, click here to send your story to Who, Me? We'd love the chance to tell your tale on a future Monday. ®