Lies, damn lies, and KPIs: Let's not fix the formula until we have someone else to blame
When 95 + (5 * RAND()) is all your spreadsheet needs
Who, Me? Monday has arrived once again and with it the sweet, sweet music of a reader's darkest IT misdeeds in The Register's weekly Who, Me? feature.
Today's story comes from "Alban", who in the mid-Noughties worked at a mobile telephone company "in a country more known for its musical than technological prowess".
"At the time," explained Alban, "the telco was partly owned by (or a partner of) Vodafone and was required to report to England monthly some specific key performance indicators (KPIs) about their network."
This was information such as cell traffic, availability and so on. "It's my understanding," said Alban, "that Vodafone was quite specific about how to calculate those" – a fact that will become significant as the sinning goes on.
"Some of these numbers were considered very relevant and discussed by the bigwigs in their monthly powwow to define problem areas and set actions.
"They took this very seriously and even reviewed in the following month whether those actions were successful."
Hey, I wrote this neat little program for you guys called the IMAC User Notification ToolREAD MORE
Ah yes, execs do love their metrics.
One of Alban's tasks was to write the scripts that produced some of the numbers in which the chief of operations was most interested. One figure measured availability and was rendered as a percentage: "Everyone was happy when it was above 95 percentage, below that threshold people got chastised by intensive management scrutiny."
Alban's scripts had been ticking over happily for a couple of years and it wasn't until he was tracking down a bug in the report where a new KPI didn't seem to match the old one that he made an awful discovery.
Somewhere in his formula Alban had mixed up a + and a -, which led to "a really bad effect of numerical cancellation and the value was basically just a random number".
Which "by pure chance and some additional errors happened to be in most cases around 95 per cent".
The result was that the bigwigs had spent the last year working on what Alban delicately described as "numbers totally unrelated to reality".
It got worse: "So whatever problem hotspot they identified in the network was just random luck, and the value next month was just random again. So all the successful managerial problem solving was in reality nothing but re-rolling the dice next month."
And worse still: "Also, the real numbers weren't as rosy as the random ones; they had a bigger tendency to be between 90 and 95 per cent."
Just to plop the metaphorical cherry on top of the fiasco: "Some managers – including the chief of operations – had part of their bonus tied to this indicator."
Pretending that nothing had happened was not an option, since the newly created indicator didn't match up with Alban's magical random KPI generator. So he bravely confessed to the chief.
Doubtless steeped in the management bullcrap speak of the era, the ops chief had a 30,000-feet view of things and could see a problem that Alban had not spotted. "Specifically that management had just proudly reported back to Vodafone how huge a success the last quality initiative was."
What to do?
Naturally, the chief had a solution.
He decided to hold back Alban's fix for the KPI until "the mobile access network guys had a big upgrade of their systems and blamed the less rosy new numbers on them and their new software".
"As you can expect," Alban said sadly, "this didn't improve my trust in managers."
Ever protected your bonus by pointing a finger at the innocent, or stared agape at the antics of management when faced with a distressing new reality? Email your confession to the occasionally sympathetic vultures at Who, Me? ®