Oh no, you're thinking, yet another cookie pop-up. Well, sorry, it's the law. We measure how many people read us, and ensure you see relevant ads, by storing cookies on your device. If you're cool with that, hit “Accept all Cookies”. For more info and to customize your settings, hit “Customize Settings”.

Review and manage your consent

Here's an overview of our use of cookies, similar technologies and how to manage them. You can also change your choices at any time, by hitting the “Your Consent Options” link on the site's footer.

Manage Cookie Preferences
  • These cookies are strictly necessary so that you can navigate the site as normal and use all features. Without these cookies we cannot provide you with the service that you expect.

  • These cookies are used to make advertising messages more relevant to you. They perform functions like preventing the same ad from continuously reappearing, ensuring that ads are properly displayed for advertisers, and in some cases selecting advertisements that are based on your interests.

  • These cookies collect information in aggregate form to help us understand how our websites are being used. They allow us to count visits and traffic sources so that we can measure and improve the performance of our sites. If people say no to these cookies, we do not know how many people have visited and we cannot monitor performance.

See also our Cookie policy and Privacy policy.

This article is more than 1 year old

Hadoop rigs are really supercomputers waiting for the right discipline

So says IBM, as it re-tunes its Spectrum HPC-wranglers for clustered workloads

IBM reckons the rigs assembled to run the likes of Hadoop and Apache Spark are really just supercomputers in disguise, so has tweaked some of its supercomputer management code to handle applications that sprawl across x86 fleets.

As explained to The Register by IBM's veep for software-defined infrastructure Bernie Sprang, apps resting on clusters need to optimise workloads across pools of compute and storage resources, and can benefit from templates that make it easier to deploy without dedicated hardware. That second point, Sprang says, is important because he's starting to see “cluster creep”, a phenomenon whereby different teams inside an organisation each cook up their own compute clusters that could perhaps be shared instead of hoarded.

Big Blue thinks workload deployment and scheduling tools can give organisations the chance to acquire and operate fewer clusters without running out of data-crunching grunt.

Enter the new IBM “Spectrum” tools, comprising just such a resource-sharing tool in the form of the “Conductor” product that can “templatise” clustered apps. There's also the “Spectrum LSF” tool for workload scheduling. IBM already offers Spectrum Storage, the filesystem formerly known as GPFS, to support large-scale workloads.

Big Blue's wrapped this new set of release up in software-defined jargon, but isn't going after mainstream workloads. It does think the Spectrum range is a fine idea for those contemplating cloudy or hybrid cloud analytics rigs, as it will happily span on-premises and public clouds.

Sprang likes to position the Spectrum suite as akin to a hypervisor, inasmuch as it sits between the server and the workload. But that's where the similarities end: this lot is aimed squarely at clustered apps and Big Blue hopes its high-end pedigree will interest those now wrestling with hyperscale workloads. ®

 

Similar topics

Similar topics

Similar topics

TIP US OFF

Send us news


Other stories you might like