Scaling agile software development

Divide and Conquer


Reader Poll The question we asked on Monday - whether agile development could scale - certainly put the cat among the pigeons.

There were undoubtedly those for whom the answer was a resounding ‘no!’ – not least because for many, Agile is predicated on the basis of small teams. Big teams = big problems, we were told, and Agile is not going to help resolve them. Here’s some example feedback.

“Keep It Simple S****” – When the projects becomes too big, no matter the methodology, it will fail.
The sheer amount of “work stuff” that mires a large project will bog a team of 500 – but if you’re controlling a workforce of that size you’re certainly not thinking in an Agile way.
You can call it agile even when it’s a mammoth floundering in a tar pit, but that doesn’t make it so.

A major bone of contention seemed to be adoption of Agile as a clever term, rather than a distinct way of doing things. “Sometimes managers hear great buzzwords but don’t get beyond the first paragraph,” said one comment. “Agile is something that should have been kept secret from the marketing depts,” said another.

From the wealth of comments you provided, it's clear that Agile is more than just marketing: “It’s not that hard to scale, providing you know what you are doing. I’ve done it again and again with multiple enterprise customers,” said a self-confessed Agile coach/consultant.

But are they right – or like the Joe Satrianis of this world (there, I’ve mentioned him again), is it just that when such people are involved, everything seems to work?

It seems the key to answering this is how agile projects are structured. The fundamental principle is be ‘divide and conquer’ – that is, if running a project with several hundred people is always going to be difficult, far better then to consider it as a series of smaller projects. “If the task can be broken down into small enough pieces then it could still use agile methods,” we are told. And indeed, “Agile can actually make it easier to work with bigger teams, because it helps you break things down into rational chunks”.

Complexity will grow inevitably as the project grows. Returning again to the comments, there is some consensus in how large-scale projects can be run as a number of smaller-scale Agile projects, but with two caveats: that the role of management is to deal with the complexity and communications, and that the challenge becomes how to bring together and integrate the different pieces – which may not all have been developed in an Agile way. We saw a number of pointers in this latter vein.

You don’t have to attempt to scale the agile approach, as you are best served in some areas (such as the foundation APIs and critical services) to maintain a solid reliable and quality tested platform, but in other areas notably those which are closest to the business and the users.
I have not seen it successfully scale beyond 10 without starting to take more hybrid approaches (e.g. use Agile approaches for prototypes of key functionality and to mitigate largest risks. Use more traditional Prescriptive approaches for work to be outsourced to external development resources, ensure full test coverage, etc.)
Instead of having a 500 people project, create 10 projects of 20 people and a tree of say 1+4 integration projects each with 2 or 3 people each.

With all of this in mind, we’d be interested to understand better your own experiences. What’s your involvement been in Agile projects large or small, and how have you got on? We’ll collate your views and play them back to you at the end of the week.

Reader Poll

Q1. Can you tell us a little about yourself and your involvement in Agile projects?

I am currently working on one or more Agile projects
I have recent direct experience of working on Agile projects
I have been observing Agile projects and have gained indirect experience
Other (please state)

Q2. In your honest opinion, do you really believe Agile can scale beyond:

5 people
10 people
50 people
100 people
500 people

Q3. What would you say were the key success factors of larger scale Agile? (1-5 where 5 is most critical)

1 2 3 4 5
A clear, coherent programme management overview across the sub-projects involved
Maintaining up to date configuration management of the project artefacts (code, documents, models etc)
Continuously integrating multiple parts of the application into a single deliverable
Packaging and delivering application elements (against a baseline)
Keeping a clear separation between development, test and live systems
Managing communications within and between project teams
Other (please state below)

Q4. What would you say are the things that cause the most problems to larger-scale Agile? (1-5 where 5 is most problematic)

1 2 3 4 5
Keeping the project running over distributed sites
Deciding what should be in the platform layer and what shouldn’t
Ensuring a sufficient level of quality of sub-project deliverables prior to their integration
New requirements having undue impact on work already underway
Difficulties in keeping to pre-ordained timescales and timeboxes
Other (please state below)

Q5. What are the most important facilities to have in place? (1-5 where 5 is very important)

1 2 3 4 5
Tools to enable management of project artefacts within sub-projects
Tools to enable management of project artefacts across the entire project or programme
Tools to support re-use and/or sharing of artefacts between teams
Tools to enable collaboration and communication between and within teams
Tools to support quality and integrity testing prior to integration
Tools to support performance and scalability testing across the integrated project.
Other (please state below)

Q6. What have we missed – and have you any war stories or success anecdotes you wish to share?

Q7. What are the challenges of implementing and using continuous integration? (Please tick all that apply)

Build Performance
Communicating build status to the group
Configuring tools to support CI workflow

Similar topics


Other stories you might like

  • Stolen university credentials up for sale by Russian crooks, FBI warns
    Forget dark-web souks, thousands of these are already being traded on public bazaars

    Russian crooks are selling network credentials and virtual private network access for a "multitude" of US universities and colleges on criminal marketplaces, according to the FBI.

    According to a warning issued on Thursday, these stolen credentials sell for thousands of dollars on both dark web and public internet forums, and could lead to subsequent cyberattacks against individual employees or the schools themselves.

    "The exposure of usernames and passwords can lead to brute force credential stuffing computer network attacks, whereby attackers attempt logins across various internet sites or exploit them for subsequent cyber attacks as criminal actors take advantage of users recycling the same credentials across multiple accounts, internet sites, and services," the Feds' alert [PDF] said.

    Continue reading
  • Big Tech loves talking up privacy – while trying to kill privacy legislation
    Study claims Amazon, Apple, Google, Meta, Microsoft work to derail data rules

    Amazon, Apple, Google, Meta, and Microsoft often support privacy in public statements, but behind the scenes they've been working through some common organizations to weaken or kill privacy legislation in US states.

    That's according to a report this week from news non-profit The Markup, which said the corporations hire lobbyists from the same few groups and law firms to defang or drown state privacy bills.

    The report examined 31 states when state legislatures were considering privacy legislation and identified 445 lobbyists and lobbying firms working on behalf of Amazon, Apple, Google, Meta, and Microsoft, along with industry groups like TechNet and the State Privacy and Security Coalition.

    Continue reading
  • SEC probes Musk for not properly disclosing Twitter stake
    Meanwhile, social network's board rejects resignation of one its directors

    America's financial watchdog is investigating whether Elon Musk adequately disclosed his purchase of Twitter shares last month, just as his bid to take over the social media company hangs in the balance. 

    A letter [PDF] from the SEC addressed to the tech billionaire said he "[did] not appear" to have filed the proper form detailing his 9.2 percent stake in Twitter "required 10 days from the date of acquisition," and asked him to provide more information. Musk's shares made him one of Twitter's largest shareholders. The letter is dated April 4, and was shared this week by the regulator.

    Musk quickly moved to try and buy the whole company outright in a deal initially worth over $44 billion. Musk sold a chunk of his shares in Tesla worth $8.4 billion and bagged another $7.14 billion from investors to help finance the $21 billion he promised to put forward for the deal. The remaining $25.5 billion bill was secured via debt financing by Morgan Stanley, Bank of America, Barclays, and others. But the takeover is not going smoothly.

    Continue reading

Biting the hand that feeds IT © 1998–2022