Can all-flash arrays, spinning disks and hybrid really live together?

AFA without tears

Conventional wisdom says that for performance-sensitive applications, you want all-flash arrays (AFAs), while for less-critical applications and bulk storage, disk is better. How can you use them in conjunction with each other? What would a joint AFA/hybrid/disk environment look like?

Broadly speaking, there are two types of flash array: the pure, all-flash kind that stores everything in a super-fast SSD; and then the hybrid array, which combines a relatively small proportion of flash with some spinning drives that provide more capacity. Then, of course, there’s your conventional spinning disk, which comes with its own RPM speeds. We typically understand the limitations of spinning disk, which revolves around input-outputs per second (IOPs). Disk IOPs usually run into the hundreds, while flash SSD registers in the tens of thousands.

Hybrid, of course, merges the two, and is supposed to provide the best of both worlds. One of the biggest advantages of hybrid drives overall is cost, point out experts. Given the still-high cost of flash memory, people don’t want to buy any more of the stuff than they must, points out Frank Berry, CEO at data centre infrastructure analyst firm IT Brand Pulse. “Everyone identifies their most frequently accessed data and stores that on flash,” he said.

Density in solid state storage is “zooming past” hard drives, in his estimation, but they’re still more expensive than HDDs, and most customers are way more sensitive about the cost per gigabyte. For this reason, the vast majority of solid state storage is still sold on a hybrid basis. IDC figures bear this out. “According to IDC's Worldwide Enterprise Storage tracker and Forecasts, the move towards flash-powered arrays in Western Europe is well under way: AFA accounted for 17 per cent of total revenue in 2016Q2, and HFA for an additional 46 per cent,” she said.

Where they use hybrid arrays, IT teams will prioritise the use of the SSD for frequently accessed files. One example is in databases, said Berry. “Within the databases there are index files that are accessed more so than in the data,” he says. “A typical solution would be to put index files on SSD drive in hybrid.” That way, the IT team doesn’t have to rely on expensive caching.

From hybrid to AFA

The balance between hybrid and AFA is slowly shifting, though, and databases are a good example of why. “A large database is in the 2-300Gb range,” said George Crump, founder and president of storage technology advisory firm Storage Switzerland, arguing that the falling cost of flash is making it more viable to put this entirely on an SSD. “And all that is before we factor in deduplication and compression,” he said. Databases may not be built for deduplication, but they decompress well, and this function normally happens inline with all-flash units.

By 2020, IDC expects 87 per cent of the total external storage market to represent flash-powered arrays, and 40 per cent of them will be all-flash. Often, hybrid flash will be a go-to choice among smaller to medium-sized customers who want better performance than pure spinning disk but simply aren’t ready to branch out into all flash arrays, points out Frank Reichart, senior director of product marketing at Fujitsu.

“In larger accounts or among data centre customers, all the production systems for mission-critical applications simply go on all-flash only, with the exception of online archives,” he says.

Jim Handy, general director at semiconductor market research firm Objective Analysis, agrees. While companies grapple with SSDs in general, hybrid arrays are typically finding their niche among smaller firms.

“Once you start having arrays of disks, then you don’t want to put the flash into the disks themselves. It’s less expensive to have a centralized SSD and then use software to manage the distribution of data from the disk to the SSD,” he said.

Every byte of flash storage that you buy from a hybrid vendor is more expensive than the same amount in an all-flash array, he argued. That makes hybrid still a good bet for smaller firms who are more cash constrained and can do without the large performance bump that an all-flash array can provide.

So what’s the point in using hybrid arrays at all? One answer is that they tier their data automatically. Smaller firms relying on a hybrid array for their entire backend system are unlikely to want to manage what data goes where. With a hybrid setup, they don’t have to – the controller takes care of all that under the hood.

This advantage is exacerbated by the variety of tiering options available in hybrid SSD products. Some offer a straight HDD/SSD split, although the proportion of each will vary. However, you can also get sub-tiers of SSD storage, each configured for read and write-intensive operations. And the hard drive layer can be tiered into different disk speeds, too. This creates a flexible array of configuration options for companies not yet ready to take the plunge into AFA.

As the industry does move to AFA, Simon Robinson, research veep at 451 Research, advises against trying to integrate them with hybrid SSD in any meaningful way. Although some firms like NetApp have been focusing on trying to make an all-flash version of a traditional hybrid SSD system, the two kinds of storage aren’t usually compatible, he warns.

“These are systems that run on entirely different operating systems. The way they store data, the way they dedupe data, the algorithms that they use, they’re all completely different,” he says, adding that this makes some idealistic visions of interplay between the two platforms difficult. Customers might want to use all-flash arrays as the primary system, and then send snapshots from that system to a hybrid one. In the real world, that’s not really possible, he says. It would take a lot of technical work to make it happen.

“My overriding problem in storage already is that I have too many silos. And you’re saying I need another silo because my flash array doesn’t integrate with the rest of my environment,” he says. For this reason, he argues that some companies will choose one or the other. Those that do use them together will find them uneasy bedfellows, and you’ll typically find administrators dividing AFA and hybrid along application-specific lines.

Next page: The ideal: pooling

Similar topics

Other stories you might like

  • Zuckerberg sued for alleged role in Cambridge Analytica data-slurp scandal
    I can prove CEO was 'personally involved in Facebook’s failure to protect privacy', DC AG insists

    Cambridge Analytica is back to haunt Mark Zuckerberg: Washington DC's Attorney General filed a lawsuit today directly accusing the Meta CEO of personal involvement in the abuses that led to the data-slurping scandal. 

    DC AG Karl Racine filed [PDF] the civil suit on Monday morning, saying his office's investigations found ample evidence Zuck could be held responsible for that 2018 cluster-fsck. For those who've put it out of mind, UK-based Cambridge Analytica harvested tens of millions of people's info via a third-party Facebook app, revealing a – at best – somewhat slipshod handling of netizens' privacy by the US tech giant.

    That year, Racine sued Facebook, claiming the social network was well aware of the analytics firm's antics yet failed to do anything meaningful until the data harvesting was covered by mainstream media. Facebook repeatedly stymied document production attempts, Racine claimed, and the paperwork it eventually handed over painted a trail he said led directly to Zuck. 

    Continue reading
  • Florida's content-moderation law kept on ice, likely unconstitutional, court says
    So cool you're into free speech because that includes taking down misinformation

    While the US Supreme Court considers an emergency petition to reinstate a preliminary injunction against Texas' social media law HB 20, the US Eleventh Circuit Court of Appeals on Monday partially upheld a similar injunction against Florida's social media law, SB 7072.

    Both Florida and Texas last year passed laws that impose content moderation restrictions, editorial disclosure obligations, and user-data access requirements on large online social networks. The Republican governors of both states justified the laws by claiming that social media sites have been trying to censor conservative voices, an allegation that has not been supported by evidence.

    Multiple studies addressing this issue say right-wing folk aren't being censored. They have found that social media sites try to take down or block misinformation, which researchers say is more common from right-leaning sources.

    Continue reading
  • US-APAC trade deal leaves out Taiwan, military defense not ruled out
    All fun and games until the chip factories are in the crosshairs

    US President Joe Biden has heralded an Indo-Pacific trade deal signed by several nations that do not include Taiwan. At the same time, Biden warned China that America would help defend Taiwan from attack; it is home to a critical slice of the global chip industry, after all. 

    The agreement, known as the Indo-Pacific Economic Framework (IPEF), is still in its infancy, with today's announcement enabling the United States and the other 12 participating countries to begin negotiating "rules of the road that ensure [US businesses] can compete in the Indo-Pacific," the White House said. 

    Along with America, other IPEF signatories are Australia, Brunei, India, Indonesia, Japan, South Korea, Malaysia, New Zealand, the Philippines, Singapore, Thailand and Vietnam. Combined, the White House said, the 13 countries participating in the IPEF make up 40 percent of the global economy. 

    Continue reading
  • 381,000-plus Kubernetes API servers 'exposed to internet'
    Firewall isn't a made-up word from the Hackers movie, people

    A large number of servers running the Kubernetes API have been left exposed to the internet, which is not great: they're potentially vulnerable to abuse.

    Nonprofit security organization The Shadowserver Foundation recently scanned 454,729 systems hosting the popular open-source platform for managing and orchestrating containers, finding that more than 381,645 – or about 84 percent – are accessible via the internet to varying degrees thus providing a cracked door into a corporate network.

    "While this does not mean that these instances are fully open or vulnerable to an attack, it is likely that this level of access was not intended and these instances are an unnecessarily exposed attack surface," Shadowserver's team stressed in a write-up. "They also allow for information leakage on version and build."

    Continue reading
  • A peek into Gigabyte's GPU Arm for AI, HPC shops
    High-performance platform choices are going beyond the ubiquitous x86 standard

    Arm-based servers continue to gain momentum with Gigabyte Technology introducing a system based on Ampere's Altra processors paired with Nvidia A100 GPUs, aimed at demanding workloads such as AI training and high-performance compute (HPC) applications.

    The G492-PD0 runs either an Ampere Altra or Altra Max processor, the latter delivering 128 64-bit cores that are compatible with the Armv8.2 architecture.

    It supports 16 DDR4 DIMM slots, which would be enough space for up to 4TB of memory if all slots were filled with 256GB memory modules. The chassis also has space for no fewer than eight Nvidia A100 GPUs, which would make for a costly but very powerful system for those workloads that benefit from GPU acceleration.

    Continue reading
  • GitLab version 15 goes big on visibility and observability
    GitOps fans can take a spin on the free tier for pull-based deployment

    One-stop DevOps shop GitLab has announced version 15 of its platform, hot on the heels of pull-based GitOps turning up on the platform's free tier.

    Version 15.0 marks the arrival of GitLab's next major iteration and attention this time around has turned to visibility and observability – hardly surprising considering the acquisition of OpsTrace as 2021 drew to a close, as well as workflow automation, security and compliance.

    GitLab puts out monthly releases –  hitting 15.1 on June 22 –  and we spoke to the company's senior director of Product, Kenny Johnston, at the recent Kubecon EU event, about what will be added to version 15 as time goes by. During a chat with the company's senior director of Product, Kenny Johnston, at the recent Kubecon EU event, The Register was told that this was more where dollars were being invested into the product.

    Continue reading

Biting the hand that feeds IT © 1998–2022