DevOps still 'rarely done well at scale' concludes report after a decade of research

Puppet Field CTO Nigel Kersten speaks to the Reg about 10 years of studying DevOps adoption

Interview On the 10th anniversary of its first survey, the multi-vendor State of Devops report concludes that DevOps is still "rarely done well at scale," and that "almost everyone is using the cloud, but most people are using it poorly."

The report is opinionated and there is no vendor pitch, perhaps because it is sponsored by a variety of DevOps companies including CircleCI, Puppet, bmc, New Relic, Snyk and Splunk. The first survey was in 2011 so this the eleventh report but a decade after the first.

"We don't want to produce something lightweight, there's a lot of crappy surveys out there, that I don't think are particularly robust in their methodology or insightful in their findings," Puppet's Field CTO Nigel Kersten, who has been involved with the State of DevOps Report from the beginning, told us.

The survey was completed by 2,657 individual DevOps professionals across the world, with around 75 per cent from the USA, Canada or Europe, and covering a wide range of organisation sizes.

The general approach is to divide organizations into those that are highly evolved, mid-level performers, or low performers. "Highly evolved" means frequent deployments, very short lead time for application changes ("less than an hour"), quick response to failures ("less than an hour"), and a change failure rate of less than 5 per cent. "Low" is the other extreme, with a lead time for changes of "between a week and six months." According to the report, "the vast majority of organizations are stuck in the middle."

This division informs the rest of the analysis, which compares the responses of high, mid, and low performers on questions like "Members of my team have clear roles, plans and goals for their work." 89 per cent of high performers agreed with this, compared to just 46 per cent of low performers.

Have some standards!

Another striking example: 57 per cent of high performers felt their IT systems conformed with all five of the US NIST (National Institute of Standards and Technology) essential characteristics of cloud computing, these being "on-demand self-service, broad network access, resource pooling, rapid elasticity or expansion, and measured service," whereas just 5 per cent of low performers could tick these boxes.

"Almost everyone is using the cloud, but most people are using it poorly," the report says. Asked to expand on why, Kersten told us that "most people are not taking advantage of those things, and essentially lifted and shifted their existing processes and practices into the cloud."

Has the way businesses do DevOps improved in 10 years? "It's definitely improved," said Kersten; yet reading the report one gets the sense that progress is slow. Over the past four years, the percentage of high performers has gone up from 10% to 18%, the survey said, and low performers down from 11% per cent to 4 per cent ; yet most remain in the middle group (78 per cent or 79 per cent).

Kersten distinguishes between "start-ups and tech companies" which "are not really talking about DevOps, they're just doing it," and "the enterprise, the mass market of IT" where "DevOps has become really successful at the team level, but because of the interactions between adjacent teams and departments, and the weight of process built up sometimes over centuries, the interaction between teams becomes critical, and that's where folk are really failing. It's not really a DevOps problem."

It is a familiar story for those who have studied software development and delivery: the methodology counts for less than the interaction between people. "Well-run companies set clear goals, we know which team does what, people communicate, they have autonomy. They tend to do really well at DevOps."

The report reads more like a paper on what it takes to do DevOps (or perhaps to run a business) well, than a typical survey, which is mostly a good thing.

Another point that comes out of the report is the importance of what it calls self-service internal platforms. "What we mean is not just centralizing IT," said Kersten, "but moving away from the anti-pattern of lots of small cross-functional teams all doing their own thing … developers shouldn't each be reinventing the wheel on how do I provision a virtual machine or create a testing environment of a CI/CD pipeline."

Having internal platforms that do this is a good thing he said – provided they really are self-service. "Where we see people fail is where they still use what we call ticket ops, where to do this thing I need to describe what I need and then someone will give me almost what I asked for."

A team for all functions

A cross-functional team is one that spans the whole application lifecycle from code to deployment, as opposed to a more specialist team that might only be concerned with database administration, for example. Are cross-functional teams a good thing?

"It depends," Kersten said. "There are underlying strata of technology that are better off centralized, particularly if you've got regulatory burdens, but that doesn't mean you shouldn't have cross-functional teams … too far in either direction is definitely terrible. The biggest problem we see is if there isn't a culture of sharing practices amongst each other."

One thing to avoid, said Kersten, is a DevOps team. "I think we've broken the term DevOps team inside organisations," he told us. "I think it has passed beyond useful … calling your folk DevOps engineers or cloud engineers, these sorts of imprecise titles are not particularly useful, and DevOps is particularly broken."

Automation is critical to "fast flow" teams

Automation is critical to "fast flow" teams

What if an organization reads the report and realises that it is not good at public cloud and not effective at DevOps, what should it do? "First optimize for the team," said Kersten.

"How much time you're spending in manual work doing toil, and focus on automating those things away. Then start optimizing processes across teams, and this is where middle management have to step up and acknowledge that this is their role … let's try and optimize the whole system rather than just each individual team doing the best they can. We don't operate as islands inside these large organisations."

That said, Kersten highlights one area which he described as low hanging fruit. "We've seen that legacy environments are a drag upon everyone's evolutionary journey. Managerial focus on actually optimizing your legacy environments and not just thinking of them as a sunk cost that you've got to keep paying the bill for, is one of the most impactful things you can do."

A section in the report on "overcoming the burden of legacy" has more detail on this, saying "invest in your legacy environments so they are no longer a constant inhibitor of progress."

The full report can be found here

Similar topics

Broader topics

Other stories you might like

  • Google sours on legacy G Suite freeloaders, demands fee or flee

    Free incarnation of online app package, which became Workplace, is going away

    Google has served eviction notices to its legacy G Suite squatters: the free service will no longer be available in four months and existing users can either pay for a Google Workspace subscription or export their data and take their not particularly valuable businesses elsewhere.

    "If you have the G Suite legacy free edition, you need to upgrade to a paid Google Workspace subscription to keep your services," the company said in a recently revised support document. "The G Suite legacy free edition will no longer be available starting May 1, 2022."

    Continue reading
  • SpaceX Starlink sat streaks now present in nearly a fifth of all astronomical images snapped by Caltech telescope

    Annoying, maybe – but totally ruining this science, maybe not

    SpaceX’s Starlink satellites appear in about a fifth of all images snapped by the Zwicky Transient Facility (ZTF), a camera attached to the Samuel Oschin Telescope in California, which is used by astronomers to study supernovae, gamma ray bursts, asteroids, and suchlike.

    A study led by Przemek Mróz, a former postdoctoral scholar at the California Institute of Technology (Caltech) and now a researcher at the University of Warsaw in Poland, analysed the current and future effects of Starlink satellites on the ZTF. The telescope and camera are housed at the Palomar Observatory, which is operated by Caltech.

    The team of astronomers found 5,301 streaks leftover from the moving satellites in images taken by the instrument between November 2019 and September 2021, according to their paper on the subject, published in the Astrophysical Journal Letters this week.

    Continue reading
  • AI tool finds hundreds of genes related to human motor neuron disease

    Breakthrough could lead to development of drugs to target illness

    A machine-learning algorithm has helped scientists find 690 human genes associated with a higher risk of developing motor neuron disease, according to research published in Cell this week.

    Neuronal cells in the central nervous system and brain break down and die in people with motor neuron disease, like amyotrophic lateral sclerosis (ALS) more commonly known as Lou Gehrig's disease, named after the baseball player who developed it. They lose control over their bodies, and as the disease progresses patients become completely paralyzed. There is currently no verified cure for ALS.

    Motor neuron disease typically affects people in old age and its causes are unknown. Johnathan Cooper-Knock, a clinical lecturer at the University of Sheffield in England and leader of Project MinE, an ambitious effort to perform whole genome sequencing of ALS, believes that understanding how genes affect cellular function could help scientists develop new drugs to treat the disease.

    Continue reading

Biting the hand that feeds IT © 1998–2022