Exclusive Hard-pressed outsourcing titan Capita wrestled with almost sixty separate outages at its data centres in the space of just half a year, The Register can exclusively reveal.
An internal log of the problems dating between June and December 2017, seen by us, detailed the extent of the downtime, with as many as 44 incidents flagged as critical or "P1".
The outage data follows recent comments from Capita's latest CEO, Jonathan Lewis, who said top brass had historically underinvested in the business as they pursued acquisitions to drive the top line growth.
Amid a second profit warning in Capita's history, Lewis branded the UK-headquartered company as "too complex", said it was "driven by a short-term focus", and "lacks operational discipline and financial flexibility".
Issues with issue-logging
The severity of the problems ranges from multiple clients being unable to access network or telephone services, to glitches that only partially affect a client's customer base.
One insider told us: "From my experience of working within the Capita IT infrastructure, I can concur [with] Jonathan Lewis's observations about underinvestment and unnecessary complexity."
Another noted that without a root cause analysis behind each ticket "it would be risky to state that lack of investment is the problem for each case".
However, our second insider said a number of the incidents appeared to indicate a lack of skilled resources, a possible lack of training or that underskilled staff may have been asked to pick up too large a workload.
He also noted – with some irony – that in the outages list, the most commonly occurring item was a reliably unreliable old version of BMC Remedy currently in use by Capita, the ticketing portal intended to deal with incidents.
"There was piece of work spun up to refresh the BMC platform but as money became tight in Q1/16 this was terminated. If they haven't refreshed it since then it looks like it is grinding to a halt on a regular basis, which is just what you want from the primary workflow tool for client tickets and internal techie activities."
Capita is looking to migrate its core ERP system out of the Capita-managed data centres into a public cloud offering. Sources claimed this was in part due to some of the technical errors Capita experienced in its own data centres.
Moving data centres
A Capita spokeswoman said: "Capita has a cloud-first strategy and it would be incorrect to attribute migration to anything other than our strategy."
She added: "60 [outages] is not a high number considering the size of Capita's IT estate and compared to other organisations of this size."
Capita has hoovered up a number of public sector contracts in recent years, having recently won an £80m deal with Transport for London to provide internet services across 1,000 sites. A previous project with TfL to upgrade the IT system that runs London's congestion charge was badly delayed.
The Register asked Capita what assurances it could give that millions of commuters would not be affected by this volume of data centre incidents. The company declined to answer this or other questions we posed.
Capita has had issues with a number of its contracts lately: in January, Prudential tore up its £722m deal with the outsourcer five years ahead of time; and a number of glitches have been exposed in Capita's recruitment system for the Ministry of Defence.
The company was also subject to a probe by the National Audit Office over its £700m GP support system.
Some £1bn was wiped off Capita's market capitalisation following a suspension of dividends and a £700m rights issues earlier this month.
Immediately afterwards, Barnet Council, dubbed "easyCouncil" because it has more than half a billion pounds' worth of contracts with Capita, put contingency plans in place to examine how it would handle the fallout should Capita fail. ®