The SmartNIC revolution fell flat, but AI might change that

The idea of handing off networking chores to DPUs persists even if it hasn't caught on beyond hyperscalers

Analysis In 2013, Amazon Web Services announced a new C3 instance type and made vague references to what it described as "enhanced networking" enabled by an Intel Virtual Function interface.

The cloud colossus later admitted that "enhanced networking" was made possible by using souped-up network cards that included enough processing power to run workloads such as firewalls and load balancers while also handling the usual packet-pushing.

This arrangement worked for AWS because moving firewalls to a network card freed some server capacity that the cloud giant could rent. It also increased the speed of network traffic and offered some security improvements through an isolated environment to run network functions for boxes that could handle workloads for multiple tenants.

Other hyperscalers also created similar cards – which came to be called SmartNICs or "data processing units" (DPUs). Mellanox liked the idea too, and in 2017 turned it into a DPU product called "BlueField" that it pitched as ideal to speed up data as it moved across all-flash storage area networks.

By 2019, VMware had taken notice of SmartNICs and began adapting its flagship hypervisor to run on the devices so they could host network functions. Nvidia noticed BlueField too, and in 2019 bought Mellanox.

In 2021, Intel joined the party with its own hardware, although it called them "infrastructure processing units" instead of SmartNICs or DPUs. AMD followed a year later by acquiring DPU maker Pensando.

Also in 2022, VMware released a product called the vSphere Distributed Services Engine that could manage SmartNICs and a distributed firewall that ran on them.

At that point, SmartNICs therefore had the backing of VMware, Intel, AMD, and Nvidia, all of which have plenty of datacenter customers and pushed DPUs as ready for deployment to take customers' networks into a bright new future.

And then not much happened.

VMware told The Register that the Distributed Services Engine didn't catch on with customers. Analyst firm Crehan recently published data suggesting that service providers which rent server capacity remain the dominant customers for DPUs, and it "has not yet seen broad penetration into markets beyond this."

Crehan also noted "new innovations, use cases, and deployment models such as DPU-enabled Ethernet switches and CPU replacement, which should expand the customer base."

Cisco is a leading proponent of DPUs in switches, having made them the centerpiece of its "Hypershield" security product and N9300 smart switches. That's validation for DPUs being useful but won't drive volume sales.

The booming field of AI might, though. Analyst firm Gartner recently published a reference architecture for AI on the edge and running on Kubernetes. Both guides suggest use of DPUs – as does Nvidia in its reference architecture for AI clouds.

And just a few days ago, another major player – Red Hat – got behind DPUs. IBM's open source shop quietly announced a technology preview for the "OpenShift DPU operator" in OpenShift 4.19.

Red Hat told us it thinks DPUs can run virtual switches, load balancers, or firewalls.

Ju Lim, a senior manager for OpenShift Product Management and a Red Hat Distinguished Engineer, told The Register DPUs could also be useful for "optimizing database and analytics workloads through direct NVMe management."

Lim thinks the cards have a role in AI, too. "An AI-powered recommendation engine could run inference workloads on the DPU while dedicating host resources to continuous model retraining, maximizing both performance and resource efficiency," he suggested.

The AI revolution may therefore finally bring about the DPU revolution.

Which is a very 2025 outcome. Dammit. ®