How simple, stateful storage and Kubernetes streamlines developer productivity

By iterating on standards, HPE CSI Driver and storage approach smooths application dev lifecycles

Sponsored Feature Containers have become the wellspring of application developer productivity gains, especially in the cloud-native world. However, owners of stateful applications now have the same capabilities at their fingertips, all without the storage-based limitations of years gone by.

It's worth taking a look back at the evolution of Kubernetes and storage if only to show how far companies like Hewlett Packard Enterprise (HPE) have advanced. Plus, it gives us a sense of the progress made toward allowing users to instantly provision storage and clone production-level datasets while steering performance and capacity during runtime for peak reliability that yields better application response times, especially during periods of high congestion.

No more storage bottlenecks

For application developers and release engineers, storage was long considered a bottleneck which hindered successful Kubernetes deployments. While this might have been the case in the early days when various plugins offered little or no data management or dynamic provisioning capabilities, times have changed.

It used to be challenging to incorporate vendor-specific storage drivers into Kubernetes. And while there were plenty of efforts to that effect, they did not deliver the broad functionality that standardization initiatives generally provide. This is one reason why Google and others worked to develop the Container Storage Interface (CSI) specification, which allows third parties to write a driver within their own domain that could present standard constructs onto Kubernetes.

CSI provides a thin shim between the container orchestrator and the backend storage systems that is unique for each vendor. Now that it is mature and widely recognized by developers, customers can pop to the CSI documentation and see all the drivers for various storage interfaces. While this is certainly helpful, it can still be a bit of a jumble to sift through, however.

HPE has laid its engineering and storage systems expertise on top of this vendor-neutral effort to allow users to tap into a container storage provider (CSP – a REST API specification that defines how provisioning, mounting and deallocation workflows are invoked from a host client) which is able to handle all the data management operations on a range of storage infrastructure.

This shift means that Kubernetes deployments are no longer restricted to the more ephemeral world of stateless applications. Stateful application developers can also capture the full promise of container-based development with better data management, higher efficiency, secure multi-tenancy and far faster development times, even for traditional applications going through modernization.

Pushing productivity

Most applications that deploy on Kubernetes and run in containers are derived from stateful applications of one sort or another (databases, for instance). As storage flexibility has increased, this means the next challenge is to integrate production datasets into test workflows, especially as higher productivity leads to faster software releases, which drives higher quality deployments.

All of this also means testing needs to improve with the addition of production-like data in the dev/test and staging environment.

That is where the functionality of the HPE CSI Driver for Kubernetes comes in. Stateful application owners can integrate that level of data early on in test cycles using instant zero-copy clones of production instances in Kubernetes and other functions inherent in their enterprise storage. With the ability to clone and attach that data to basically any type of container infrastructure where the driver is available, the bar for application development in containers has been raised.

Although vendor-neutral, the HPE CSI Driver has support for HPE's entire storage portfolio, which is especially useful as large enterprises go on their application modernization journeys.

The HPE CSI Driver's power is in allowing the ability to dynamically provision persistent volumes which can be set by system and storage admins. The focus of HPE's driver is to provide a consistent experience across private cloud environments.

Control runtime storage performance/capacity

HPE CSI Driver for Kubernetes - when attached to primary storage like the all-NVMe HPE Alletra cloud-native data infrastructure (powered by HPE GreenLake), HPE Primera or HPE Nimble storage systems - means users can instantly provision storage and clone production datasets while controlling volume performance and capacity during runtime to minimize operational disruption and downtime. Users can also deploy multiple Kubernetes clusters on the same infrastructure with top-line storage infrastructure backing production-level data for full developer productivity.

When storage is kept distinct from the Kubernetes cluster, existing clusters can be wound down and redeployed without changing the data, which remains shielded and can be made active again as soon as a new cluster is provisioned. In short, the development environment can be deployed and recovered quickly, without too much concern over the safety of the data—a major advantage for test-to-production projects.

Combining these external storage lines delivers an ideal environment for customers who want to scale compute, network, and storage individually. In addition to snapshots and clones, users can also get sophisticated multitenancy so you can run production, staging, and test/dev on the same storage by compartmentalizing all the storage resources to support a particular workload, and then you can scale that back or grow as the need for capacity or performance changes.

Existing HPE customers won't need to know the details of the various HPE storage systems, and they can replicate data between the Alletra 9000 and Primera and between the Alletra 6000 and Nimble systems, for even better scaling and flexibility of performance.

HPE hyperconverged solutions such as HPE Alletra dHCI (disaggregated hyperconverged infrastructure) or HPE SimpliVity data virtualization platform can give customers the flexibility of using storage provided by the infrastructure layer CSI drivers, with the option of using HPE CSI Driver for Kubernetes for richer data management for stateful applications.

Easy storage provisioning for Kubernetes clusters

But perhaps the most important aspect of the HPE CSI Driver is that storage managers can delegate control and command and conversely, application developers writing containerized microservices applications don't know – and don't have to care – that enterprise-grade storage with features such as zero copy snapshots are making their stateful storage more scalable and resilient. They can do it in a self-service manner, just like they could provision virtual machines and stateful storage on server virtualization hypervisors in the days before containerization.

All of which leaves the application developers to do what they do best, and the storage managers to make sure the scale is there to support applications as they move from test/dev through qualification and into production. All on enterprise-class, scalable storage.

"Containers are designed for portability, so any organization using them must create an architecture that allows for data to be wherever it needs to be, whenever it needs to be," says Michael Mattsson, Technical Marketing Engineer at HPE.

"Containerization, and Kubernetes in particular, helps businesses move away from imperative operational models that require multiple teams to perform tedious tasks on a regular basis. It allows them to adopt an agile declarative model where there's a clean separation of concerns, along with a high degree of automation and abstractions that make sense for running a high-performance technology company," he adds.

Developers already have enough to think about in getting stateful applications into production as quickly as possible – the HPE CSI Driver might at least mean that having sufficient underlying storage resources ready and available isn't one of them.

Sponsored by HPE.