Agile Manifesto co-author blasts failure rates report, talks up 'reimagining' project
Jon Kern is looking for Agile exemplars, not the 'Agile Industrial Complex'
Interview The Agile Manifesto was published almost a quarter of a century ago. Yet as the years have rolled by, its lofty ideals have run headlong into the brick wall of management desire for process and reporting.
Jon Kern, one of the authors of the Agile Manifesto, calls this the "Agile Industrial Complex."
The proliferation of processes and frameworks that have sprung up in the wake of the Manifesto are topics close to his heart. "It feels like we stepped back in time," he tells The Reg. "As if the manifesto never existed, and we're back to heavy processes."
Why are we back with these giant diagrams or giant processes? Well, because it gives comfort to those middle managers who really don't know what's going on as much as they might think they do...
The elephant in the room, and one of the reasons Kern is speaking to us, is a recent report on Agile failure rates published last month.
At the time, we speculated that it could be seen as a thinly veiled plug for Impact Engineering. The report has since come under fire over its methodology and conclusions.
Kern doesn't mince words: "I thought the research was trash. It didn't even remotely, shall I say, mirror what I would consider to be Agile."
He gives examples: "Silly things, like the project requirements weren't clear before the software development process began ... that's such an old trope, you know?
"[The suggestion] that the Manifesto says: 'Yeah, forego clear requirements' – why would you want to do that? That's just silly…"
Kern is blunt about why some individuals gravitate toward processes that might seem at odds with what the Manifesto is all about.
He asks: "Why are we back with these giant diagrams or giant processes? Well, because it gives comfort to those middle managers who really don't know what's going on as much as they might think they do."
Kern gives an example: "I can latch on to this, I can see roles, and it's nice. You know, the diagrams look great. And it's almost like Waterfall; it gives me that false sense of security.
"And I'll just pretend to ignore reality."
There is a certain purity to the original Agile Manifesto that has been eroded over the years as processes, frameworks, and tools have been developed around the concept. Kern compares the situation to a comedy sketch in which a patient in an operating theater is lost among all the exciting, shiny machinery.
He admits to dropping out of the Agile scene around a decade ago when getting certified in one framework or another was all the rage. "So I just kind of extracted myself. I stopped going to the conferences and all that crap because it was just BS, in my opinion."
"I'm just gonna go off and work with teams that have fun, doing Agile stuff and building cool products."
He is back now, though. "I'm kind of saying, 'All right, I'm gonna give it one more shot to try to get more people to understand'."
Since 2023, Kern and others have been, in his words, "reimagining Agile." The goal is to track down exemplars: "Places where Agile is flourishing, and shine a light on those so people can see what good looks like."
"And from a research point of view, my colleagues and I specifically are interviewing such exemplars to try to figure out what did they do to feather the nest? Because it's not about doing Kanban or Scrum … there are some misconceptions about the difference between doing some Agile framework and being Agile.
All that said, it is hard to escape the feeling that well-meaning frameworks and best practices have partially obscured the Agile Manifesto's spirit over the years. There is an entire generation of engineers who have grown up with Agile who won't remember some of the heavyweight processes popular in the previous generation that gave rise to the Manifesto.
"So I feel I owe it to this next generation to at least make a crack at helping them see.
Kern is focusing on exemplars to provide what he calls a "beacon of hope."
"And then describe in one or two pages [that] this is what it's supposed to be like: 'Here're things that I should be looking for. And here's how it was going on here. And being able to find things that might look like a match for where I'm working and try them out.'
- Fragile Agile development model is a symptom, not a source, of project failure
- Report: 83% of UK software engineers suffer burnout, COVID-19 made it worse
- 'Business folk often don't understand what developers do...' Twilio boss on the chasm that holds companies back
- Software shortcuts: Pay down your tech debt. It's time to fix a price
"So that's really my motivation. We need it now more than ever to help … the more that people understand what's kind of behind the Agile mindset and that it's not this stupid crap that you see being spread around, the greater the chance that we can help people face the ever-increasing complexity that's going on today. So it's really about helping to get behind what it takes to be Agile so that folks can grow their own capacity and in turn their team's and in turn their organization's."
Looking ahead, Kern describes the Manifesto as timeless: "It got at the gist of that social thing we call software development and, really, it's not hard to stretch it to any kind of product development."
He adds: "The challenge of Agile is it being more of a mindset. And I think it's the hardest way to be because you need to be constantly vigilant. [It's] more pragmatic than dogmatic, except we treat it more like a dogmatic process... So I think it's that dichotomy that makes things suck." ®
Updated at 1157 UTC on July 22, 2024, to add
A spokesperson at Engprax sent us a statement following the publication of the above interview with Kern: “The fieldwork collecting the data was conducted independently and the research agency was given no indication of any preference of results.
"A 2021 review found the methodology used in this research to be consistent with those conducted using alternative methodologies when applied to software engineer populations."
The spokesperson also pointed to Dr Bertrand Meyer's book Agile! as "notably critical of Agile's lack of upfront design."
“We are passionate about being challenged and we welcome further research, particularly that of a greater evidential value and that which challenges our work," the Engprax rep added. "In the past we have made updates to our public work in light of new contradictory information when the grounds for such updates would not have been publicly known hadn't we chosen to share them.
"However, we are concerned that personal harassment of those publishing research critical of Agile may lead to a silencing effect to researchers and those critical of Agile - that which is no doubt desired by extremists in the debate.
"It speaks volumes that it takes a company whose bread-and-butter work includes standing up to powerful institutions and providing pro-bono work for whistleblowers facing retaliation to be courageous enough to dare criticize Agile."