This article is more than 1 year old
2021 in storage: We waited for a flash price revolution that never came. But about creativity? We can't complain
DPUs in the data centre, object and file get cosy, 3D layer Jenga plus cloud, capacity, and ransomware advances
Analysis While much of the world was in lockdown in 2021, storage boomed. It was a year of ransomware, tech advances, hybrid multi-cloud, a switch to subscriptions and services, hypergrowth in analytics startup funding, and building a DPU data centre makeover.
Suppliers grew, were acquired, struggled, were reborn, and a few (very few) went under. We also saw moves on the high-level exec dancefloor as CEOs and other execs darted between companies, looking to make the optimal career move.
The constant backdrop to the year was COVID-19 and a recovery from business lockdowns in spending, which provided growth opportunities for virtually all storage companies, despite the widespread supply chain issues. Strong growth rates in data, particularly unstructured data for analytics and AI, provided good business for data protecting and managing companies as well as block, file, and object storage suppliers.
Flash and disk suppliers had a good year and the tape media and library suppliers benefited as well.
The other constant backdrop was ransomware with an epidemic of attempts to invade organisations' data stores, encrypt them and extort cash. Even storage suppliers were not immune – witness Kaseya and ExaGrid. The data protection industry responded with near-universal immutable storage technology, virtual air gaps, and backup file scans. As examples we could cite Infinidat's InfiniGuard CyberRecovery, the Rubrik ransomware guarantee and Threat Hunter, Cohesity's security advisor, and the Quantum Scaler tape library ransomware block.
A third pervasive theme was the rise of hybrid multi-cloud products and services. It became generally understood that there is not going to be a mass migration from data centres, be they central or distributed edge sites, to the public cloud titans. Indeed, the public cloud titans, realising this, expanded their on-premises presence. Suppliers realised that multi-cloud appeal was strong; no one likes lock-in to a single supplier, and also that the so-called edge needs local processing and storage.
This is for the same reason that retail stores have local point-of-sale intelligence rather than green-screen cash tills hung off a central mainframe. Edge sites need intelligence and storage distributed to them for fast response and reaction to events.
Many companies supported the idea of offering their products as subscription-based services, like HPE GreenLake and Dell APEX, Pure Storage and Cohesity. Pure went further and offers its storage arrays as on-premises, cloud-style consumable resources
The fourth theme was the incredible strength of the database, data lake, warehouse and analytics industry as a magnet for VCs. It was exemplified as recently as 13 December, with distributed RDBMS startup Cockroach getting a $268m funding round and a $5bn valuation.
The mass adoption by incumbent players of Kubernetes and container technologies continued apace; we can mention HPE Ezmeral, NetApp astra, CTERA, Hitachi Kubernetes Service, Buurst, and Dell Container Storage Modules. Kubernetes support is a table stakes feature for storage incumbents.
The NAND industry concentrated on 150+ layers in 3D NAND manufacturing with industry leader Samsung pushing 176 layers like Micron and SK Hynix, and Kioxia and Western Digital pumping our BiSCS gen 6 162-layer chips.
The transition away from disk-influenced 2.5-inch drive bays to the EDSFF ruler format began with developments from Micron, Kioxia, Fadu, and NGD.
Jason Adrian, senior director of Azure Platform Architecture at Microsoft, said of Kioxia's XD6 announcement: "The EDSFF E1.S form factor is the future of flash storage in hyperscale data centres, including Azure platforms."
That seems pretty clear cut and servers will increasingly be offered with EDSFF bays rather than the legacy disk drive bay slots.
The disk drive manufacturers continued their dogged and determined advance up the nearline drive capacity ladder, with both Western Digital and Seagate announcing 20TB drives and Toshiba reaching the 18TB level.
All three manufacturers think HAMR (Heat-Assisted Magnetic Recording) is the required technology for capacity advances beyond the 30TB or so level. They each have their own incremental technology steps to get to that level, such as Toshiba's MAS-MAMR and Western Digital's OptiNAND. It seems likely, although unsaid by WD, that full MAMR may not happen. Instead OptiNAND 10 and 11-platter drives may get it to the HAMR gateway of 30TB-plus capacities.
One notion was debunked this year: the idea that there was going to be a flash-disk cost crossover with the $/TB of flash matching or even becoming lower than that of disk. Instead it is now pretty widely recognised that SSDs and HDDs will co-exist for the next decade, with each technology lowering its cost/TB more or less at the same rate.
Tape capacity boundaries continued being pushed back, with Fujifilm demonstrating a 1PB tape in February.
Optane's role continues to have some doubt attached to it, with Intel not saying much about its manufacturing future after Micron's departure from the 3D XPoint manufacturing field in March.
An Intel 10-Q SEC filing showed that its Optane business business lost $473m in the first nine months of 2020. Intel hinted Optane could be CXL-connected in the future. VAST Data, which uses Optane drives to store metadata and function as a write buffer, has qualified Kioxia's FL6 Optane-class SSD as an alternative.
In November Dr J Metz, technical director for systems design at AMD and SNIA board chair, said Intel announced 3DXP "years before its availability, stalling some of the market – including its own." He added: "I still like Optane a lot. PM (Persistent Memory) is going to play a huge role in both CS and SmartNICs. Advances in NVMe and SNIA are relying on that class of storage to not just exist, but play a central role. It's just a shame that it hasn't created the paradigm shift that was anticipated."
Optane buffs are looking forward to the gen 3 product, potentially an 8-decker with doubled capacity.
DNA storage progressed but it became increasingly obvious that its IO speed can be catastrophically slow. Unless this issue is solved, DNA storage will remain largely a scientific plaything.
In the interconnect area PCIe 4.0 started becoming established, with product announcements from Phison (controllers), Samsung (client SSD), GigaIO (PCIe 4 composable fabric), Kioxia (Exceria SSDs), and Micron (7400 SSD) as examples. The next doubling of PCIe speed to gen 5 started with a Marvell controller in June, and a Kioxia SSD in November.
The future CXL interconnect, based on PCIe gen 5, received a tremendous boost when the overlapping and competing Gen-Z consortium decided to merge its efforts into the CXL technology. Next year we may see the fist CXL-connected external memory stores appearing.
It is possible, if not probable, that the CXL bus will provide a boost to composable systems technology, the automated and dynamic setup and teardown of servers from pooled component resources of compute and memory, accelerators, storage, and networking.
Liqid made most of the composable systems running this year, and got a $100m funding boost in December as a reward for its product, partner, and market-building efforts.
Back in the legacy storage networking world we saw 64Gbit/s Fibre Channel with products from HPE, using Broadcom hardware, and Cisco.
There was a flurry of NVMe/TCP announcements from Lightbits Labs, Fungible, and NetApp, and its role as the iSCSI upgrade path seems solid.
High Bandwidth Memory, stacked memory dies connected via an interconnecting interposer layer to processors, made progress as well. Intel's coming Sapphire Rapids gen 4 Xeon processor will support HBM2e as will Intel's Ponte Vecchio GPU. Samsung also supports HBM2e.
SmartNICs and DPUs
There were several product and technology moves as suppliers positioned themselves in the data centre server offload market. The products are divided between SmartNICs and specialised processors for east-west data centre networking.
Intel made lots of noise about being what it called the biggest IPU (Infrastructure Processing Unit) leader through deals with hyperscalers for its server offload cards.
Nvidia ploughed ahead with its BlueField SmartNICs, publicising gen 3 and pushing ahead with VMware and its Project Monterey to have vSphere running on the BlueField cards. Fungible announced that its Storage Initiator Card supported NVMe/TCP, and can run at 6.5m IOPS to a single server using its FS1600 NVMe storage server and DPU chips.
Dell said in September that VxRail hyperconverged systems fitted with Project Monterey-style SmartNICs were coming in spring next year.
There were many storage array product pushes in the year, such as HPE's Alletra follow-ons to its Primera and Nimble arrays, Pure Storage's range-topping FlashArray//XL, and NetApp's similarly positioned AFF A900. All-flash arrays dominated, with Infinidat launching its first AFA in June. Kioxia plugged away with with its software-defined KumoScale flash boxes and VAST Data opened a new marketing front by positioning its arrays as FlashBlade-style fast restore and fast archive access boxes.
Seagate put a lot of effort into its LyveDrive business, which transports disk drives from endpoints to data centres. It set up LyveDrive receiving points in Equinix colos, which also saw Pure install FlashBlade systems to provide fast connectivity to the Azure cloud.
We are seeing Pure erecting a storage and container-orchestrating layer across the on-premises and multi-cloud environments. It has a similar vision to NetApp and we think that wherever an application executes, there will be Pure Storage resource that can effectively move around the hybrid multi-cloud environment and find Pure Storage available wherever it goes.
Notable company events include HPE CEO Antonio Neri taking a personal interest in GreenLake and giving it a kick up the ass. GreenLake boss Kumar Sreekanti retired in August so Neri reorganised the GreenLake business teams and leadership.
In September HPE said there would be additions to its GreenLake subscription, marking the company's shift into direct competition with public cloud data warehouses like Snowflake and SaaS-based data protectors such as Cohesity, Druva, HYCU, and others. It won Porsche as a GreenLake customer in October and Barclay's Bank in a 10-year, 100,000+ workload deal just days ago.
Object (and file) storage supplier Scality, which gained $20m in fresh funding in January, went into the cloud-native world by announcing Artesca in April. In general it appeared that object and file storage were preparing to get closer together.
Dell made a series of storage related announcements throughout the year but a significant upturn in storage revenues eluded it. For example, in November we wrote "Dell's quarterly revenues rose 21 per cent year-on-year to a record $28.4bn, but storage revenues were anaemic with a mere 0.9 per cent rise to $3.89bn." We suggested there might be problems in high-end array sales, low or no growth in Dell's PowerScale file and ECS object storage products.
NetApp did well in the year, growing revenues for the fifth quarter in a row in December, and appointing, for the first time, a chief product officer hired from Microsoft – not an ONTAP mothership member, in other words.
IBM storage sales declined throughout the year, despite upgrades of its products. In October it announced updates for Spectrums Virtualize, Protect, Protect Plus, Scale, AIOps for FlashSystem, and the ESS 3200, which runs Spectrum Scale software. But IBM then saw its storage CMO, Eric Herzog, desert to join competing Infinidat in the same month.
IBM invested in combined transactional and analytics SQL database company SingleStore in November. There's a Red Hat connection here as SingleStore is available in IBM's Cloud Pak for Data, and is certification on Red Hat OpenShift and available in the Red Hat Marketplace. Basically, IBM is waiting for the next mainframe cycle to shift more DS8000 arrays, hoping for Red Hat to move more storage products, and treading water elsewhere in storage after having updated its FlashSystem all-flash arrays in February.
We recorded 10 acquisitions in the year:
- DataCore buys Caringo – 22 Jan
- Twitter buys DriveScale – 25 Jan
- NetApp DataStax – 20 May
- NetApp buys Data Mechanics – 23 Jun
- IBM buys ECX from Catalogic – 24 May
- HPE buys Zerto – 1 July
- HPE buys Ampool – 8 July
- Quantum buys Pivot3 – 19 July
- NetApp CloudCheckr – 4 Oct
- DataCore buys MayaData – 18 Nov
Among the partnership deals, four stood out:
- Dell selling Druva products deal – 21 Jan
- Silk Azure co-sell – 10 Feb
- NetApp Rubrik resell deal – 22 Apr
- Nutanix-Citrix – 23 Sep
There were three failures we covered: Datera in February with its liquidation in March, InfiniteIO in May, and Pivot3 in July, with assets being bought by Quantum. China's Tsinghua Unigroup went bankrupt in July. That's a low number of failures overall.
NetApp end-of-lifed its SolidFire HCI business in March and SolidFire founder Dave Wright resigned from NetApp in July to join a fintech business. From our point of view, the ONTAP all-flash arrays killed the SolidFire all-flash arrays because ONTAP has such a massive presence in the market and inside NetApp. SolidFire revenues were a small percentage of ONTAP revenues and the ONTAP AFF product's performance was excellent, leaving no real need for a standalone SolidFire array other than as a niche filler.
- Flash? Nu-uh. Windows 11 users complain of slow NVMe SSD performance
- Toshiba to become Threeshiba – company split to spur growth as strife persists
- Do not try this at home: Man spends $5,000 on a 48TB Raspberry Pi storage server
- Fatal Attraction: Lovely collection, really, but it does not belong anywhere near magnetic storage media
Various new CEOs were appointed. Infinidat co-CEO Karel Sandler went on 5 January and Phil Bullinger was made CEO next day. We saw new CEOs at StorMagic, Pavilion, Acronis, Fungible, MayaData (before DataCore bought it), Arcserve and, on 16 December, Veeam.
As well as getting a new CEO, Acronis appointed ex-VMware CEO Paul Maritz as its board chairman in October. Is this a distant hint that an IPO is envisaged?
Notable exec moves were Fidelma Russo joining HPE as its CTO, coming from Dell. Strongbox's CEO and exec team left as it rebranded as StrongLink under a new CEO, Andrew Hall.
A few companies were reborn through refunding, pivots, and rebranding including Reduxio becoming Ionir, Panzura completing what it called its "refounding" – it has just launched a white glove managed migration service too; Kubernetes startup StorageOS became OnDat; and computational storage startup Nyriad became comprehensively reorganised under new management with a new CEO, execs, and funding.
DDN has rebooted its Tintri acquisition, helped by management changes, and is growing nicely. Among the new startups Ocient impressed.
In mid-November we said $5.84bn had been invested in 36 storage startups and post-startup-pre-IPO suppliers this year. With recent new funding for Liqid, Rockport, and Cockroach, the numbers are now $6.17bn and 39 companies.
Ironically, with only one IPO (that of Backblaze), more money has flowed into building up companies than into IPO-ing enterprises standing on their own feet at last. That means there is a fattening queue of potential IPO candidates: Cohesity, Databricks, Druva, Rubrik, Veeam, and WekaIO, for example. Perhaps even VAST Data and Acronis.
It has been a massively positive year for storage, overcoming the COVID-19 pandemic and associated supply chain issues to grow and prosper. The incumbent Kubernetes adoption phase is over and attention has turned to offering products as cloud-style services through subscriptions, and as hybrid, multi-cloud services.
Data protection is more highly thought of than ever as suppliers have adopted anti-ransomware measures with vigour and alacrity – here we've seen Acronis, Cohesity, Commvault, Druva. HYQU, Rubrik and Veeam, also Veritas, still a huge, huge player.
Two of the old guard storage array/filer incumbents, HPE and NetApp, are doing well, Dell not quite as well, Hitachi Vantara less so, and IBM not very well at all. Pure continues to grow strongly as does Nutanix. Both need to become profit-making.
Qumulo is growing strongly as is Infinidat. This storage editor feels, though, that the two champion growers are VAST Data and WekaIO, both file-based.
A final thought: storage in 2021 started with COVID and ransomware. It progressed through an ongoing switch to cloud-like services and is finishing strongly on a mix of flash and files. Stick that in your mental folder structure. ®