A report from DORA, that's the Devops Research and Assessment sponsored by Google and other DevOps vendors, says 26 per cent of surveyed technology workers consider themselves "elite performers."
DORA was founded in 2015 by DevOps specialists Nicole Forsgren, Jez Humble, and Gene Kim, and in late 2018 was absorbed by Google Cloud. Each year the gang, now led by Google's Dustin Smith, publishes an Accelerate State of DevOps report, co-sponsored by nine other DevOps outfits.
The research is based on responses from "1,200 working professionals," we're told, with over half in organizations of 500 or more employees. The majority of respondents work in development, software engineering, DevOps, site reliability engineering, or management. Two out of five participants are said to have at least 16 years of IT experience.
DORA categorizes these folks into four categories: elite, high, medium, and low. To be elite, one must deploy multiple times per day in an on-demand fashion, require less than one hour lead time for changes, need less than one hour recovery time in the event of an outage or defect, and achieve a 0-15 per cent failure rate on changes.
DORA has now added a fifth metric, reliability, defined as the degree to which one "can keep promises and assertions about the software they operate." This is harder to measure, but nevertheless the research on which the report is based asked tech workers to self-assess their reliability. There was a correlation between reliability and the other performance metrics.
Self-assessment
According to the report, 26 per cent of those polled put themselves into the elite category, compared to 20 per cent in 2019, and seven per cent in 2018. Are higher performing techies more likely to respond to the survey? That seems likely, and self-assessment is also a flawed approach; but nevertheless it is an encouraging trend, presuming agreement that these metrics and survey methodology are reasonable.
Much of the report reiterates conventional DevOps wisdom. NIST's characteristics of cloud computing [PDF] are found to be important. "What really matters is how teams implement their cloud services, not just that they are using cloud technologies," the researchers said, including things like on-demand self service for cloud resources.
- Report: 83% of UK software engineers suffer burnout, COVID-19 made it worse
- DevOps still 'rarely done well at scale' concludes report after a decade of research
- JavaScript, GitHub, AWS crowned winners in massive survey of 32,000 developers
- Chaos is good for you, says first 'state of chaos engineering' report
New this year was a look at documentation, often not the favourite occupation of developers and engineers. "We found that about 25 per cent of respondents have good quality documentation, and the impact of this documentation work is clear: teams with higher quality documentation are 2.4 times more likely to see better software delivery and operational (SDO) performance," the researchers said.
What makes good documentation? Use cases, guidelines for updating existing documentation, and clearly defined ownership of documentation, they suggest, as well as including documentation as part of the software development process.
DORA's interactive chart showing how patterns, practices, and outcomes are related ... Click to see it in action
The DORA team also identified certain patterns and practices as beneficial, including loosely coupled architecture, continuous integration, merging work into a shared trunk frequently (as opposed to having long-lived branches), the use of open-source technologies, observability practices, management of database changes, and deployment automation – though note that this is a report sponsored by DevOps vendors.
How can techies and their bosses avoid burnout? It is all about having the right internal culture, the researchers said. "Teams with a generative team culture, composed of people who felt included and like they belonged on their team, were half as likely to experience burnout during the pandemic."
What about the worst-performing tech workers? The good news is that they have declined in number, from 15 per cent in 2018, to 12 per cent in 2019, and seven per cent in 2021 (or possibly they no longer bother to complete surveys.) Most respondents though fell into the high (40 per cent) or medium (28 per cent) categories.
So for those who want to improve, this is another bit of research that explains the principles; but translating that into the nitty-gritty of actual business and wrangling legacy technology and technical debt will remain challenging. ®
