Off-the-shelf servers spar with million-dollar storage arrays

Cheap-as-chips Nexenta squares up to EMC and NetApp over performance


VMworld 2011 Cheap and cheerful storage from Nexenta matched EMC and NetApp's multi-million-dollar systems in VMworld 2011's Hands On Lab, and took up some of the slack from its rivals when difficulties arose on the first day. EMC and NetApp have cooly brushed off Nexenta's claims it outperformed them.

The storage infrastructure for the Hands On Lab (HOL) at the show in Las Vegas was provided by the trio of vendors after they were approached by VMware, with Nexenta getting the call in February.

VMware wanted Nexenta to provide the storage component for a SuperRack concept, a commodity white box-based alternative to VCE's Vblock and the NetApp/Cisco Flexpod. Arista provided switching technology for the SuperRack.

Nexenta said it was responsible for up to 60 per cent of the Hands On Lab storage load at one time, despite only having a fifth of the work allocated to it, because of problems, it is speculated, with one of the other systems.

The lab ran on vSphere v5 in the cloud, a geo-distributed public cloud with data centres in Amsterdam (Colt), Miami (Terremark) and Las Vegas (Switch SuperNap). Each component lab was deployed using VMware's Cloud Director v1.5 (not the release generally available), and on NFS. We understand there were 480 lab seats, with dual-monitor workstations using VMware View desktops, and 24,000 lab seat hours of operation.

There were some 27 individual labs in the overall HOL, and around 5,000 people used the HOL in total during its 50 hours of operation.

Storage speeds and feeds

EMC provided two VNX 7500 arrays configured with three file blades plus a standby blade. The arrays came "with SSD, FAST Cache, FAST VP, and loads of 10GbE". The bulk of the EMC load ran on NFS, and VNX VSAs ran in many of the labs.

We understand one VNX 7500 was an all-flash configuration with 96 SSDs. The other had more than 140 15K RPM hard drives and an SSD storage tier. The estimated street list price for these two VNX configurations was $2.6 million.

NetApp supplied, we understand, two FAS3270 arrays, each with four Flash Cache cards and 144 15K RPM hard drives. The estimated list price for the pair was $1.4 million.

Nexenta provided four instances of its ZFS-based NexentaStor, known as SuperRack, running on SuperMicro Westmere hardware: essentially, generic commercial off-the-shelf (COTS) X86 machines. A pair of high-availability servers were used per instance. Data was stored on around 160 7,200 RPM 2TB Seagate SAS hard drives, with STEC 8GB Zeus RAM and Zeus IOPS (200GB) SSDs, which were used for read and write caching. It says its systems for the show cost $325,000 at list prices.

Storage performance

EMC said that:

  • there were 131.115 terabytes of NFS traffic in the labs;
  • there were 9.73728 billion NFS operations;
  • the average I/O size was 14KB;
  • internal average VNX NFS read latency was 1.484ms; and
  • internal average VNX NFS write latency was 2.867ms.

A total of 13,415 lab sessions were run, creating and destroying 148,138 virtual machines over four days. A VM was created every 1.215 seconds and there were 10 to 25 VMs per lab seat session, with each VM up to 26GB in size.

Nexenta claims 7.9 billion storage operations were performed by the four NexentaStor systems. They ran four out of eight HOL vertical app areas for the duration of the show. The peak load was 154,000 4KB NFS operations at sub-1ms latency and the peak bandwidth on a single Nexenta controller was 1,300MB/sec. The average latencies were:

  • Read: 2.194ms
  • Write: 2.133ms

Load sharing

Nexenta's CEO, Evan Powell, blogged:

There were instances during the show when NexentaStor took on a far greater workload than had originally been scheduled – for reasons we can only speculate. This meant that we were at times running over 50 per cent of the verticals whereas we originally were expected to carry approximately 20 per cent of the load. NexentaStor handled this without breaking a sweat.

I don’t want to publicly speculate about why we took on the additional workload – it’s something the organisers and other vendors may be able to share, although they may be reticent to do so.

EMC's Chad Sakac, a vExpert VP in the VMware Technology Alliance, said:

These HOL situations are always frantic, always push everyone (both the humans and the technology) to the limit, including VMware and the partners who support them. Every year, most of the folks - VMware and supporting partners - have a 'bad moment'. Frankly, it’s more about how one copes, in my opinion.

This year, it was relatively uneventful compared with 2009 and 2010. Yes, there were some periods where storage latency was bad. Yes, it placed a lot of stress on all the storage platforms. In particular, read latency in one case was pretty hardcore.

A NetApp source said: "I checked with our guys who were at the VM HOL. Zero failures. In fact, feedback was that we blew the rest away."

There was a DRAM failure on a Nexenta server but the high-availability failover moved the processing load to the paired server and users didn't notice the blip.

Outside the Nexenta environment, a misconfigured VM caused some performance problems as well, with that being fixed by upgrading its memory from 4GB to 8. On top of that a network latency issue, perhaps the same read latency issue Sakac referred to, affected the first day of the lab sessions.

The HOL will be a feature of VMworld in Denmark, to be held between 18 and 20 October, and the EMC, NetApp and Nexenta storage systems will be present there.

COTS work is one moral of the story

Flash played a huge role in all the HOL storage systems. In Sakac's view:

From a storage reporting standpoint ... SSD is the biggest game changer across the industry – and its commoditisation (in full swing) will change a lot. This will impact everything: server-side storage models (SSDs, PCIe-based flash, and virtual storage models), all SSD arrays, and the critical role of both caching and auto-tiering in shared storage models.

If you’re a storage vendor whose primary value is in technologies that have less relevance in that transition, or you are unable or unwilling to adapt, you’re in for a rough ride.

Another story here is that the cheap and cheerful off-the-shelf hardware in the Nexenta ZFS storage stood toe to toe with EMC and NetApp storage costing four to eight times as much.

Powell said: "We stood tall alongside the largest, most established storage brands with a comparable solution that comfortably held its own, despite it costing just a fraction of the EMC and NetApp offerings."

Nexenta kicked serious arse at VMworld - and EMC, NetApp and VMware each know it. ®


Other stories you might like

  • VMware customers have watched Broadcom's acquisitions and don't like what they see
    It's not hard to find unpleasant precedents for what might happen to Virtzilla

    VMware customers have seen companies acquired by Broadcom Software emerge with lower profiles, slower innovation, and higher prices - a combination that makes them nervous about the virtualization giant’s future.

    The Register offers that assessment after spending the day at a VMware user group conference in Melbourne, Australia, where we interviewed over a dozen VMware customers to ascertain their reaction to Broadcom’s surprise acquisition of the virtualisation giant. The customers all requested that The Register not use their names, or those of their employers, as none were authorized to speak to the media.

    One of those customers was a sysadmin at a sporting organisation that has decided to drop Symantec products because product evolution has slowed under Broadcom’s ownership. The sysadmin has also heard, from multiple sources including Broadcom partners, that the company uses price hikes to discourage customers it does not want.

    Continue reading
  • Confirmed: Broadcom, VMware agree to $61b merger
    Unless anyone out there can make a better offer. Oh, Elon?

    Broadcom has confirmed it intends to acquire VMware in a deal that looks set to be worth $61 billion, if it goes ahead: the agreement provides for a “go-shop” provision under which the virtualization giant may solicit alternative offers.

    Rumors of the proposed merger emerged earlier this week, amid much speculation, but neither of the companies was prepared to comment on the deal before today, when it was disclosed that the boards of directors of both organizations have unanimously approved the agreement.

    Michael Dell and Silver Lake investors, which own just over half of the outstanding shares in VMware between both, have apparently signed support agreements to vote in favor of the transaction, so long as the VMware board continues to recommend the proposed transaction with chip designer Broadcom.

    Continue reading
  • Broadcom to 'focus on rapid transition to subscriptions' for VMware
    Offers comforting vision for core customers, products, channel – though warns efficiencies are coming

    Broadcom has signaled its $61 billion acquisition of VMware will involve a “rapid transition from perpetual licenses to subscriptions.”

    That's according to Tom Krause, president of the Broadcom Software Group, on Thursday's Broadcom earnings call. He was asked how the semiconductor giant plans to deliver on its guidance that VMware will add approximately $8.5 billion of pro forma EBITDA to Broadcom within three years of the deal closing – significant growth given VMware currently produces about $4.7 billion. And subscriptions was the answer.

    Krause also repeatedly said Broadcom intends to invest in VMware’s key product portfolio and is pleased to be acquiring a sales organization and channel relationships that give it reach Broadcom does not currently enjoy.

    Continue reading

Biting the hand that feeds IT © 1998–2022