Kubecon Europe Two of the early developers of Kubernetes, Brendan Burns and Tim Hockin, took part in (separate) "Ask me anything" events at virtual Kubecon Europe, offering insights into the past and future of the world's favourite container orchestrator.
Kubernetes "all began in the fall of 2013", according to co-founder Brendan Burns, then at Google but now a corporate VP at Microsoft, where he is responsible for Kubernetes on Azure, Linux on Azure, and more.
We present [Kubernetes] as a monolith because it solves a class of problems all at once, but each piece is really distinct
At Google, Burns, along with with two colleagues (Joe Bida and Craig McLuckie), was inspired by the potential of Docker tooling and container standardisation and set out to create a "minimally viable orchestrator".
Tim Hockin was one of a few software engineers who joined the team in early 2014. Google was not new to containers since it was already running a "unified container-management system... we internally call Borg", as described here, but although Kubernetes was in some ways an evolution of Borg, a key difference was that Kubernetes was always intended for external developers.
As part of that aim, Burns described how they also "spent a significant chunk" of their time "convincing executive leadership that open-sourcing this project was a good idea".
Kubernetes v1.0 was released in 2015, at which time the Cloud Native Computing Foundation was also formed in partnership with the Linux Foundation, with Kubernetes donated as the seed project.
Microsoft's Brendan Burns, responsible for Azure Kubernetes Service, wants to see "Visual Basic for the cloud"
Kubernetes is now huge, perceived as the future of computing infrastructure in succession to virtual machines (VMs), just as VMs took over from bare metal as the most common unit of compute deployments, especially in cloud environments.
After five years, how does Hockin (who worked on Borg before Kubernetes) feel about it now? "Both unreal and boring at the same time," he told Kubecon Europe. "GKE is a huge and growing business, and I helped do that, but also the core problems are the same as last year."
What problems? "Networks and storage are always hard to integrate. More customers have more than one cluster, so multi-cluster is hot hot hot," he said. "Scale too."
Hockin is interested in the idea of treating entire clusters as things that you can blow away and replace on demand. “Blue-green clusters is a thing I would not have predicted … upgrade by replacing, clusters as cattle,” he said “Still very early, but a lot of excitement.”
How does Kubernetes differ from Borg? "K8s and Borg are very different in the details, but ideologically very similar," said Hockin. "Borg doesn't have a networking model in the same way K8s does. Borg apps are WRITTEN to run in Borg, so it can be more prescriptive. Borg apps are very homogenous – same libs, same RPC system, same authentication," he said, whereas Kubernetes was designed to work with existing open-source systems.
"But they both focus on automation, ephemerality, dynamic management, and absolving users of the need to care about details that (mostly) don't matter to them," said Hockin.
Will Google migrate from Borg to Kubernetes? "We run some cloud services on k8s already, but Borg has 14+ years of features that are custom built for search, ads, Gmail, and K8s DOES NOT want those custom features," he said.
What do people misunderstand about Kubernetes? "We present it as a monolith because it solves a class of problems all at once, but each piece is really distinct," said Hockin. "The API part of K8s is pretty thin – almost all of the actuation is async to the API and totally replaceable."
Is Kubernetes too complex? "It is complicated because it is over 100 different APIs working in concert. There are a lot of moving pieces and subsystems. But I don't think it is any more complex than any other OS-like thing. I struggle with the idea of simplicity and inherent complexity. Can we make it simpler? Sure – at what cost? Which features are you willing to cut? The truth is that EVERY feature is being used by someone."
What would he change with hindsight? "I would have focused on disconnecting the components more, less 'monolithic release'," said Hockin. "I would have argued harder for nested namespaces. I would have made the walls at the edge of a cluster less impenetrable. I would have focused more on API machinery as an IDL [Interface Definition Language] – more power in the core of the API."
Does he still "talk with Brendan after he went over to the enemy?" a participant asked. "Microsoft is not 'the enemy' and I try not to position contributors in opposition," said Hockin firmly.
Burns himself came on to be grilled the following day. Like Hockin, he reflected on the growth of Kubernetes. Did he think at the start it would be so big? "We saw the empty space in the tech landscape and believed that something would come along and fill it," he said, "but there were no guarantees it would be Kubernetes. At the time, there were quite a few alternatives."
'Directionally, we're headed to virtual nodes'
Burns spoke about the adoption of virtual nodes in Azure Kubernetes Service. Virtual nodes are a means of using on-demand resources in a cluster, an alternative to deploying VM nodes or using VM-based auto-scaling.
"We see a number of people working with virtual nodes, but the support for virtual nodes in Kubernetes in general is still experimental, so it's not the predominant use case yet," he said.
"It tends to be focused on specific workloads that are hard to do in Kubernetes e.g. each container in virtual nodes is in its own hypervisor, so it's a great solution for easily running untrusted code alongside your application. I think what we are seeing is that it's clear most people don't really care about the underlying machines. They want them gone."
Are virtual nodes the precursor to "a serverless experience with per-pod resource utilisation pricing?" asked another participant. "Directionally, we're headed to virtual nodes, but we need to do more work in the Kubernetes community," said Burns. "I think we are headed there, I mean with node auto-repair coming to AKS and autoscaling already present, we're practically there at some level."
What is the biggest mistake developers make? "The biggest problem people make for themselves is exposing their internal objects as the external API," said Burns, "because suddenly you're locked, you don't want to break your clients, but you need/want to iterate your internal representation. It is very hard to 'undo' that.”
Asked about the future, Burns spoke of his wish to see a "broadening the market of cloud-native developers. I still want Visual Basic for the cloud. We're getting closer, but we're not there yet."
Visual Basic was revolutionary in its day, making Windows development easier and unleashing a flood of custom applications... as well as tons of spaghetti code and unmaintainable projects, resulting in its successor Visual Basic for Applications earning top spot in StackOverflow's "Most dreaded language" list. Burns, we presume, was talking about VB's ease of development, and not the baggage that came with it. ®