Future is bright for NVMe-over-Fabrics with TCP and Ethernet, say Solarflare, Lightbits

We'd be more confident in the tech if others were touting this

Lightbits Labs and Solarflare are promoting the idea of NVMe-over-Fabrics (NVMe-oF) using TCP-over-bog-standard Ethernet, instead of RDMA over Converged Ethernet (RoCE) via data centre-class Ethernet, iWARP, InfiniBand or Fibre Channel.

Solarflare claims NVMe-over-TCP using Ethernet provides IT departments with the same high bandwidth, low latency performance as iWARP and RoCE, and it'll work with 90 per cent of the 100 million Ethernet ports shipped into their data centers each year.

The biz's XtremeScale NICs support a universal kernel bypass suite that lets applications tap directly into the tech and make the most of it, accelerating NFV and storage server traffic. Solarflare claims this NVMe-over-TCP kernel extension provides its network adapters with the ability to approach the latencies of NVMe local flash storage over conventional POSIX-compliant Ethernet TCP networks – ie: no RDMA required.

A Solarflare video describes its NVMe TCP approach:

Youtube Video

It says that “the kernel-based overhead stack in ROCE is a very heavy IO stack and introduces additional latency into every storage transaction.” Solarflare’s NVME-oF/TCP gets rid of this and simplifies things by making the target appear as “normal storage” and runs on ordinary Layer-2 Ethernet switches.

But it only runs on Solarflare NICs, and that is a limitation.

Israeli startup Lightbits Labs also has the same idea. It says it is working with Facebook, Intel, and other industry players on defining and implementing NVMe-over-TCP as part of an NVMexpress.org working group.

Lightbits' designs ... Click to make this graphic bigger

Lightbits claims its pre-standard NVMe-over-TCP offering provides high performance and delivers consistently low latency. Random reads average at 120μs while random writes run at an average of 46μs, we're told.

NVMe-over-Fabric using TCP promises direct-access PCIe flash drive speeds across Ethernet with no special hardware, excepting Solarflare NICs, making it less expensive to acquire and operate. But only two companies are known to be working in this area, suggesting that other NVMe-over-Fabric suppliers are either blind or biased, or that there are issues we don’t yet know about. ®