Victory! The smell of skunkworks in your office in the morning

Adios, the Canny Guru solving Sphinx-level COBOL riddles

While it’s easy to start up a few, flashy new DevOps teams, releasing to production each week and flaunting the ball-and-chain of enterprise governance, scaling that change to your organisation will always be challenging, if not crushingly impossible.

When it comes to scaling the skunk-works, I’m reminded of a conversation with a struggling enterprise architect. I often use the company’s mobile app and it’s updated frequently, integrated well with iOS, and provides an overall very pleasant experience. Such results are normally unexpected from this type of aged, highly regulated, lumbering enterprise. As this enterprise architect was masterfully telling me why their organisation was doomed, I piped in: “But the mobile app is pretty good - excellent even!”

“Oh. Well. I mean sure. But that’s the mobile team,” the EA said in an almost: “You kids today and your: ‘I’ve-got-it-all-figured-out attitude - these olds’!” tone, “They’re different.”

My first thought was: “Er, well, maybe you should go figure out what they’re doing right.” More broadly, this situation pointed to the too-common anti-pattern of letting successful skunk-works teams live in isolation too long. There are two approaches I’ve seen for airing out the skunks and spreading change wider.

The smiling knife roll

Changing an organization from within is extremely difficult. Most staff were hired to do a specific job and if they’ve achieved seniority, they’ve mastered their daily tasks and figured out how to max out their performance reviews. Few people are excited to change such a comfortable statis. And in larger organisations outsourcing contracts often act like concrete poured atop bleached coral.

In these circumstances, brute force and fear is often the fastest way to scale up the team of skunks. This starts with a tops-down mandate from executives who are deathly afraid of being “disrupted” by the dog-under-desk startups. Once vendors and consultants with their slides of startup logo-doom circle through, these executives get that “sense of urgency.” Focused, annual freaking out is common, but what’s key is that the executives actually round up budget and corporate attention to spend on this change.

More than likely, the next move will be to hire an outside maverick who carries a well-cared-for roll of knives who immediately engages in bureaucratic knife fighting. Internal champions can work as well and in some more staid, closed cultures may be all that’s possible.

Either way, what’s key is slicing away the organisation mould and focusing on building up new staff and teams who are amenable to change. Otherwise, as so many tales of cowed mavericks show, nothing happens and you all too soon hear the soft clanking of those knives being rolled up and shown to the door.

As the new leader builds up their posse, they must also start the hard job of internally marketing how fantastic this new DevOps stuff is by highlighting the success of relevant projects. A well planned series of small projects can build momentum and build up the internal folk-lore of success. Coupled with that, internal conferences with staff from the new teams are often used to convince the coral encrusted that following new methods might just be a good idea, and even make their lives better.

Hearing tales of success and, it’s hoped, an easier work-life from peers is often the only way to convince staff. After all, management has always been going on about “change,” and look where we are now - changing again!

There are uncountable micro-level tactics to discover and fix. Management must discover and iteratively try to fix these issues rather than relying on teams to heal-thy-self. Cajoling senior developers to pair program is a good example. Often, senior developers are threatened by the prospect of sitting at a desk with “junior” developers. Pairing seems to threaten their status as The Canny Guru who’s grey-beard status is often relied on, for example, to solve Sphinx-level COBOL riddles.

But here’s the incentive our knife-roll cuts with: in theory, by nature of being a senior developer, The Canny Guru actually enjoys writing code. At home, they’ve likely Python’ed up some sort of Raspberry Pi contraption to mist their iguana when the terrarium hygrometer comes up foul. Instead of being in meetings all day to dispense sage-insights, when the senior developer pairs, they find that they’re back to writing code most of the day. They return to their old joy. And, if attending meetings and putting together well spaced out diagrams is their new love, well, it’s clearly time for them to be, er, “promoted” to management.

Organisations are immutable

There’s a handful of case studies from financial companies and government agencies following the principal I like to call “organisations are immutable.” For those not up on their nerd talk, this means you can’t change an organisation once it is set up. The sunny side of this, is that you can still create new ones.

In this method of introducing a new approach to software, like DevOps, a brand new group is created that operates under the new bureaucratic norms, slowing adding new projects to the new group. High-level management blesses this new organisation to sweep its arm across a messy governance table of half drank ticket queues and droning CABs, starting with a tabula rasa.

The existing - now “legacy” - organisation continues to operate as needed, but slowly, projects and people are moved to the new organisation. Service dependencies from new to old are mediated and hidden behind APIs and facades, trying to decouple the two into a sort of reverse-quarantine: the old organisation is blocked off from infecting the new group.

The new organisation follows all the new-fangled notions, from the pedestrian practice of free lunches and massages, to paired programming, to fully automated build pipelines… all enabling the magic of small batches that result in better software.

The magic of this method is that it avoids having to unfreeze the glacier, namely, the people who don’t want to work in a new way. Instead of doing the hard work of changing the old organisation, management slowly moves over willing people, reassembling them into new teams and reporting structures. The old organisation, though perhaps de-peopled, is left to follow its waterfall wont.

Publicly, companies have said that something around 30 per cent of people won’t “make it” to the new organisation. When I talk to executives in less than public forums - for some reason, always in poorly lit places like bars and parking garages - they say the number can end up being close to 70 per cent.

The “digital services” agencies created by various governments in recent years are some of the most documented examples here. Several large corporations have applied this pattern as well, often calling the new organisations “Labs.” The entirety of the old glacier is far from melted, but the rate of liquefaction is having actual, real business effect. Who among us, after all, can resist ordering pizza through a watch?

Success is the best deodorant

Either approach is still difficult and takes time. Based on what I’ve seen the first year will yield anything from 10 to, perhaps, 50 applications managed in the new fashion. It could be more if you’re blessed with people and software that is easily massaged into your new process. In the second year you can expect to up that rate much more.

Changing how a large, 50-plus-year-old organisation with thousands of applications is much harder than failing to success with all those hats-on-cats applications you see from the the ramen-fed, high-hemmed skinny jeans set. Once you build up a streak solid wins, though, introducing DevOps is easier to ripple through the organisation... so long as management actually does their job of, well, managing. ®