IT's Holy Grail, but is DevOps a Poisoned Chalice for sysadmins?

A journey without end...

DevOps is 2016’s tech holy grail – unified development and operations, both working to deliver what the business needs, quickly, reliably, and adaptably. Done well, DevOps transforms the way organisations work; it helps break down barriers between tech teams, and between technology and the rest of the business.

Good DevOps is the antidote to increasing segmentation and specialisation within companies. With the promised benefits, is it any wonder that senior managers are pushing for it in organisations spanning all sizes and industries?

The reality of DevOps implementation is different. DevOps changes the way organisations run, and the way the people in them think. It redefines what’s possible and desirable. But it comes at a price. Change is hard. And some people have further to go, more changes to go through, and more to lose, than others.

The changes DevOps require in staff throughout the organisation are hard for some people to wrap their heads around. Within development, it can be difficult to think about infrastructure, security, and stability. In the business, it can be difficult to get used to the idea of working side-by-side with technical staff. For operations, the stakes are higher.

DevOps and IT Operations

There is a team within all organisations that have learned to define their value based on up-time, stability, and security; a team that have been repeatedly told that their value is in how well the systems run; a team which is invisible, until things go wrong, at which point people start shouting; the IT operations team. DevOps is a godsend for the operations team. It’s a new way for the organisation to think, where the concerns that have been given to operations, and which they have been fighting to protect for years, are shared with the rest of the company, and where out of hours support is performed by the people who wrote the system. This is a great relief. It is also terrifying.

For so long, operations teams have defined their worth by these metrics, and now they’re being taken away. What’s left? What will operations teams do, now? When developers do on-call, fight fires, and build-in stability, scalability, and security, what’s left for operations? When infrastructure is defined in code, who needs operations? Is it time to update the CV and move on, or is there a role for operations once developers have taken everything over?

People Resist Change for Valid Reasons

From where a traditional system administrator sits, it’s sometimes impossible to see what the world will look like when concerns, security, uptime, and support are shared responsibilities. This is a different experience than developers or testers moving to DevOps. Developers are being made responsible for more of the system, which enhances job security. No developer looks at DevOps and sees job loss. The end result may be collaboration, but the path that individuals in different departments take are very different. System Administrators, however, work to implement DevOps with the perception that they may be unemployed once they finish.

Having been through a number of DevOps implementations, we can say that these concerns come up regularly. DevOps is a whole new way of doing things, and requires learning how to think differently, how to use different tools, and how to work closely with people that have previously had nothing to do with operations. There’s a lot to learn, and fear makes learning much harder. If the goal is effective learning, fear must be eliminated (or at least significantly reduced).

Fear, Uncertainty, and Doubt

Much of the resistance to DevOps comes from fear and uncertainty around the outcomes, and doubt about the personal benefits. The first step in reducing this fear actually starts before implementing DevOps, or talking about introducing it. The first step is communication.

Before starting the transformation, it’s important to understand what people would spend their time doing if they didn’t spend hours firefighting, preparing for releases, telling people no, or tracking down bugs in production systems. Different people in the team will have different answers, but it’s important that those answers are clarified, and dug into, until staff see how moving past their current responsibilities lets them contribute directly to business goals, and frees up time for learning new things, giving them a compelling reason to assist in any change effort.

Once new DevOps practices start being introduced, it’s important to link individual’s motivations and the impact of implementing new ways of working together. In a well-considered implementation, business drivers, individual motivations, and new methods of behaviour will align, allowing people to work, for their own reasons, toward the same goals. This is not easy.

“If you aren’t making mistakes, you aren’t taking enough risks.” Debbie Millman

Operations staff, who have spent their careers learning how IT systems work, how they fail, how to identify problems, and how to build scalable, stable, secure systems, are invaluable to people attempting to do it for the first time. They can shorten the length of time it takes for developers to understand the infrastructure by orders of magnitude.

Whether infrastructure is in code or manually installed, people who understand those systems, the impacts of bugs, and the impact of updates, are needed. And for a long time, after DevOps starts, operations will be the people with that knowledge. While they’re busy collaborating with developers and business staff to share that knowledge, they’ll also be learning from them. They’ll be picking up knowledge they’ve never had access to, before – why the business wants particular things done, how the code is going to accomplish it, and the value of the infrastructure that will enable it. Operations staff have a lot to contribute to DevOps, in the short- and long-term. They just need to see it.

Fine Print

Even the most effective transformation is going to come up against problems or obstacles. Things are going to fail. Mistakes are going to be made. More than anything, it must be clearly understood that the migration from the old way of working to the new is an evolutionary process, where there will be setbacks, and lessons learned, before the final goal is reached. And even the concept of ‘final goal’ is fraught, as it implies that DevOps is a journey with an end, with a set of artifacts that indicate DevOps success. The truth is more nebulous.

The acceptability of failure can be a difficult concept for people to accept, even when it makes perfect sense, especially if they work in operations. Which is worse, 5 hours of downtime 1/yr, or 10 minutes of downtime 1/month? In many companies, 10 minutes of downtime would be cause to reduce the number of deployments, to reduce the risk of further outages, even if it increased the likelihood of a longer outage. The truth is that more frequent outages, that last less time, are a sign of better resilience, and better systems, than rarer, more severe outages. During the learning process, mistakes will happen that increase the frequency of outages, and this is OK. Over the long run, the reduction in total downtime will more than make up for the increase in outage frequency.

It’s the Journey That Matters

DevOps is the path. It is the way of thinking, of approaching problems, of seeing problems in a unified way, that creates better results for internal staff and customers. As such, there is no final, right implementation. There are no DevOps artifacts. There are no DevOps tools. Each artifact, each tool, can be used to enable or inhibit DevOps. Each company will apply DevOps ideals in ways that suit their culture, and their customers. They’ll then reflect on what they’ve learned about themselves and their customers, and they’ll iterate. Again. And again. And again. DevOps is the journey.

Noah is Head of Transformation at OpenCredo, working with clients to improve their effectiveness. He focuses on holistically understanding the organisation, its goals, and its current challenges, and working with the people in the organisation to adapt to a world where changing requirements, technologies, and skills is the norm. This blog originally appeared on Open Credo's website.

Want to learn more about DevOps, Continuous Delivery, and Agile? Head to our Continuous Lifecycle Conference from May 3-5. Full details here.