Oh no, you're thinking, yet another cookie pop-up. Well, sorry, it's the law. We measure how many people read us, and ensure you see relevant ads, by storing cookies on your device. If you're cool with that, hit “Accept all Cookies”. For more info and to customise your settings, hit “Customise Settings”.

Review and manage your consent

Here's an overview of our use of cookies, similar technologies and how to manage them. You can also change your choices at any time, by hitting the “Your Consent Options” link on the site's footer.

Manage Cookie Preferences

  • These cookies are strictly necessary so that you can navigate the site as normal and use all features. Without these cookies we cannot provide you with the service that you expect.

  • These cookies are used to make advertising messages more relevant to you. They perform functions like preventing the same ad from continuously reappearing, ensuring that ads are properly displayed for advertisers, and in some cases selecting advertisements that are based on your interests.

  • These cookies collect information in aggregate form to help us understand how our websites are being used. They allow us to count visits and traffic sources so that we can measure and improve the performance of our sites. If people say no to these cookies, we do not know how many people have visited and we cannot monitor performance.

See also our Cookie policy and Privacy policy.
Sign in

Hasta la vista Lustre, so long Spectrum Scale: Everyday HPC is here

Flash arrays could make parallel file systems redundant

Chris Mellor Mon 1 Aug 2016 // 11:27 UTC

Comment Parallel file systems were developed to overcome delays servers experienced when accessing files on disk storage systems. Flash arrays get rid of disk access latencies and so weaken the need for parallel file systems.

Spectrum Scale, the renamed GPFS (General Parallel File System) and Lustre are two such parallel file systems. Instead of waiting for one IO stream to fill a server with data from a file system, they use multiple simultaneous IO streams which fill the server much more quickly. Such technologies are used with parallel compute clusters for high-performance computing applications such as simulations, seismic analysis and reservoir modelling, and enable the storage resource to support many more IO requests over a period of time.

Disk arrays are good at streaming IO but poor at random IO as these involve more disk seeks. In an interview with the Next Platform, Pure Storage’s Max Kixmoeller, its VP for products, said Pure wanted to bypass parallel file systems, which involve extra layers of code on top of an underlying file or object system, and have the applications in accessing servers connect directly with the underlying file/object system on a Pure FlashBlade array. He said a base line with NFS is a good starting point.

That way there would be no need for extra hardware and software to cache random IO data and make it sequential before sending it to the parallel file system. Thus burst buffers would not be needed and the cost and complexity of running such applications would be reduced by using base flash storage arrays with NFS, rather than burst buffers fronting GPFS or Lustre with a back end disk drive array.

Talking about the FlashBlade beta program, he said: “I would say that the people who came into the early adopter program have found the upper boundaries of what NetApp and Isilon can do, and so FlashBlade is providing them a breakthrough in terms of dramatically more compute and simulation power than they had before.”

Let’s add in NVMe over fabric ideas, which effectively banish network access latency, and we could be seeing petabye-plus flash arrays providing pretty-near instant large dataset access as a matter of course in years to come, with none of today’s burst buffer/parallel file system complexity – yep, just some everyday HPC. ®

4 Comments

Other stories you might like

Biting the hand that feeds IT © 1998–2022

Your Consent Options Cookies Privacy Ts&Cs