Since the publication of the Agile Manifesto, there’s been a steady acceptance that Agile is the way to go when it comes to software development. The old waterfall method was seen as something rather quaint and old-fashioned, the equivalent of hanging onto your vinyl LPs when the rest of the world was downloading onto their iPods.
And yet, just as vinyl is making a comeback so we see that waterfall is still clinging on tenaciously. In fact, it’s not just clinging on but positively ruling the roost. According to Gartner's IT Key Metrics Data, waterfall methods were employed on 56 per cent of development projects in 2015.
So, given that it’s generally accepted that agile projects offer much scope for effective development, why are organisations still clinging to the old methods?
Old school is cool
It’s something that worries David Norton, research director at Gartner's Application Governance and Strategy Group. He says there are two strands: “There is the organisational drift back to Waterfall where a company has moved to agile practices and not changed the culture.
“Then there’s change at the project and programme level. This is far more common, we get about two or three calls a week where that’s happening or in danger of happening.”
The hold-up with agile is not due to technical issues – agile deployment remains the best option for most software projects – rather cultural obstacles need to be overcome.
Middle management wall
Translation: middle management, which is like permafrost – something that’s hard to penetrate. “If they don’t change, then we say that the agile antibodies aren’t strong.”
It’s a resistance others have noticed. A senior analyst at a FTSE100 insurance company (who wishes to remain anonymous) explained these “cultural” problems in terms of an organization still running mainframes and handling batch processing.
“You get these guys in their 50s, with no social skills and they’re the ones who maintain the systems. The company can’t get rid of them as the kids haven’t got the right skills,” he told me.
Typically what happens, our analyst explained, is that management will decide to “go agile”.
“They’ll drop in some new age guru who will use agile to develop a web-based system at the front end. At some point, however, they have to speak to the backend and connect to the CICS, DB2 and COBOL applications and that’s where it all breaks down,” he concludes.
He cites as one example a project that was implemented using an agile approach but after a few months, it was clear it was falling behind and eventually the team started using an old methodology.
It’s a situation Norton’s seen in other companies. He says that such situations have to be handled carefully. “If I see guys in their 50s who have been doing COBOL since Moses, I know there’s going to be pushback.”
He cites one client in the US where two of the company’s brightest COBOL programmers both resigned on the same day, the day after they’d been told the company was going agile.
“That considerably slowed the company down – there was enough knowledge to keep the lights on but it did mean that it slowed down the release of new products – as they were struggling to keep the lights on.”
In such situations, it’s important that companies move gradually. “The attitude is that they think ‘I’m not having someone breathe down by neck, telling me what to do.’ In that instance, you can’t go fully into agile mode, it’s literally baby-steps so you can slowly move them.”
Church of Agile
One of the problems, however, is that there’s a tendency for there to be an almost evangelical belief in agile as a methodology and that this sets up conflict.
Mike Adam, CEO of referral marketing agency Digital Animal, has seen the way that agile has become all-pervasive. “There’s a belief in agile as the way that things should be done, it’s like it’s the Church of Agile,” he says.
In March, he wrote a blog setting out the limitations of agile and attracted much criticism for going against received wisdom. Adam sees agile as a dragon in need of slaying. According to Adam, the age of Agile will soon be over - Winter is coming.
He sees that agile has become accepted because it’s fashionable: “Agile is a hipster methodology – it’s when methodology becomes a religion that you get problems.”
Adam stresses he absolutely agrees that agile development is the way forward but - and this is the big but - he points out that the methodology isn’t really that important. What is important, he says, is that there’s a strong leader – one with vision. He draws parallels with the showrunner for TV series, someone who provides an over-riding world view.
Gartner’s Norton agrees that personnel are key to this – and not necessarily technology leaders. “I’ve seen one company where they were doing agile and have reverted; what’s happened is that it lost a couple of people from the finance side and compliance side, the new people came in and didn’t understand what agile was about,” he says.
It’s not only management that can hinder agile rollout. According to Wayne Harris, a product owner at Chroma Sports, agile doesn’t sit very easily with the Prince 2 project management methodology, for example. “Trying to run an agile project with Prince 2 project management is a nightmare,” he says.
There are other methodologies out there, too, that agile must contend with: PMP and CAPM to name just two.
But back on Prince. Harris sees reporting as a major issue and the way that agile meets head-on with this methodology. “The biggest issue is being able to measure what’s happening - Agile isn’t easy to report.”
He knows many companies are aware this is an issue. “I went to a seminar to listen to an agile evangelist talk about this. There is a way that Prince can work with Agile, said the evangelist: either the Prince methodology can change or project managers can become a rare breed and can become scrum-masters,” says Harris.
Difficulties with Prince aside, there’s little doubt that agile is going to be the way forward. More and more organisations are adopting it as a way to run projects and, as some of the old guard retires, there will be less resistance from established programmers.
Despite his run-ins with Prince, Harris is still very involved with agile development, revelling in the freedom it gives him. Although, he says, that he’s never worked in a wholly agile environment. David Norton endorses this view, a hybrid approach is becoming popular, he says.
This growing move, however, should not be the go-ahead for organisations to simply crowbar in agile methodology. It’s an approach that requires firm leadership, a clear vision and total buy-in from all parties.
Gartner’s Norton describes one company that has adopted exactly the right approach to ensure senior management buy-in. “JP Morgan Australia senior managers have installed Kanban boards for themselves. One of them told me ‘as a business guy, I needed to show by example'.”
There is no reason why new development projects need revert to waterfall but the business culture has to alter first: without this, companies will be stuck in the old ways. ®