HPE, Cerebras build AI supercomputer for scientific research

Wafer madness hits the LRZ in HPE Superdome supercomputer wrapper

HPE and Cerebras Systems have built a new AI supercomputer in Munich, Germany, pairing a HPE Superdome Flex with the AI accelerator technology from Cerebras for use by the scientific and engineering community.

The new system, created for the Leibniz Supercomputing Center (LRZ) in Munich, is being deployed to meet the current and expected future compute needs of researchers, including larger deep learning neural network models and the emergence of multi-modal problems that involve multiple data types such as images and speech, according to Laura Schulz, LRZ's head of Strategic Developments and Partnerships.

"We're seeing an increase in large data volumes coming at us that need more and more processing, and models that are taking months to train, we want to be able to speed that up," Schulz said.

"And then we're also seeing multi-modal problems, such as integration of natural language processing (NLP) and medical imaging or documents, so we have this complexity, we have this the need for faster, we have this need for bigger that's coming from our user side, from our facility side, and we need to make sure that we're constantly evaluating to have these different novel architectures, to have different usage models to be able to understand all that."

HPE-Cerebras graphic

Click to enlarge

The LRZ team decided that the Cerebras technology, with its large shared memory and scalability, was a good match for the "pain points" they were trying to resolve, she said.

"And then the combination of Cerebras with the Superdome Flex from HPE, this seemed to make sense. The Superdome Flex has a very efficient pre and post-data processing, the resource management, it keeps the Cerebras system fed, and it keeps it nice and happy and full of data."

The Cerebras technology is built on the concept of using an entire silicon wafer to make the central processor, rather than cutting it up into individual chips. The result is a Wafer-Scale Engine that has 850,000 cores optimized for sparse linear algebra operations with 40GB of on-chip memory, fabricated using a 7nm production process.

"Each one of these cores is identical and fully programmable, built, from the ground up to optimize performance for the sparse linear algebra, compute operations that are common both to large scale AI and HPC workloads," explained Cerebras VP of product management Andy Hock.

"Each one of those cores is also directly connected to its four nearest neighbours across the entire device, within a high bandwidth, low latency interconnect mesh, and the dataflow traffic pattern between cores is fully programmable at compile time. So not only are we bringing massive compute resources to bear but also very high bandwidth memory and high bandwidth communication between those processors."

This architecture contrasts with the typical approach to tackling large-scale AI problems, which is to build a large cluster of servers. But as the problems and the models get ever larger, this approach shows diminishing returns, according to Cerebras.

"The time to solution doesn't scale linearly. So you might bring to bear, say, hundreds of processors, but only get a result tens of times faster," Hock said.

HPE-Cerebras graphic

Click to enlarge

In contrast, the Wafer-Scale Engine, with its closely linked host of processors, allows for linear performance scaling out to models that are on the order of hundreds of millions of parameters or even billions of parameters, Hock claimed.

The CS-2 system has one of these Wafer-Scale Engines inside a 15U rack-mount chassis along with redundant power supplies, an internal liquid cooling system, and a dozen 100GbE network ports to link to the outside world.

In the new LRZ deployment, one CS-2 is linked using all those network ports to the HPE Superdome Flex via an SN3700M Switch using a fully non-blocking topology.

The Superdome Flex is fitted with eight InfiniBand HDR100 adapters to link it to the LRZ's backbone network and, according to HPE, "ensure a proper injection bandwidth from the file system to sustain the expected very high performance of the CS-2 and feed it smoothly."

The Superdome Flex itself has 16 Intel Xeon processors with 12TB of system memory and 100TB of NVMe local storage.

However, the key part of the whole system is the software stack, which Cerebras claims enables the LRZ scientists and researchers to use standard AI software tools to build code that will run on the CS-2.

"We know as researchers and scientists ourselves at Cerebras that a high performance computing platform is only good if it's easy to use, and we can only reach that broader user audience if we allow them to programme this machine with standard frameworks that they're using today," Hock said.

This is achieved through a compiler that allows users such as machine learning researchers and data scientists to develop in standard frameworks like TensorFlow, and PyTorch, and translates their code into an executable that can run on one or more CS-2 devices.

"We also have a lower level software development kit that many of our HPC audience are using to add custom kernels into our compiler to bring to life not just to AI applications, but also HPC applications for a wide range of projects ranging from signal processing to physics based modelling and simulation," Hock said.

Dr Dieter Kranzlmüller, chair of the Board of Directors at LZR, said that the new system would be used to combine aspects of traditional HPC and AI processing for some applications.

"We're following here an integrated supercomputer architecture, which means that the future HPC system is heterogeneous and we'll pick up all the help we can get from advanced technology," he said.

"So the idea is really that you will have your HPC application, but the HPC application will get additional benefits reaching out to specialized solutions for particular applications or a sort of a model where we improve the time to solution by replacing part of the complexity with AI models, then run these on a specialized chip, on specialized hardware."

The new deployment is not the first installation to pair a Cerebras system with an HPE Superdome server. As detailed by our sister site The Next Platform, the Neocortex supercomputer at the Pittsburgh Supercomputing Center at Carnegie Mellon University actually has two Cerebras systems, linked together and to the Bridges-2 supercomputer via the Superdome. ®

Broader topics

Other stories you might like

  • HPE thinks your next GreenLake deploy will be a private cloud
    Plus: IT giant expands relationship with Red Hat and SUSE, tackles hybrid data fabrics

    Extending a public-cloud-like experience to on-prem datacenters has long been a promise of HPE's GreenLake anything-as-a-service (XaaS) platform. At HPE Discover this week, the company made good on that promise with the launch of GreenLake for Private Cloud.

    The platform enables customers "to have a cloud in their premises wherever the data is, whether it's at the edge, it's at a colo datacenter, or is at any other location," Vishal Lall, SVP and GM for HPE GreenLake cloud services solutions, said during a press briefing ahead of Discovery.

    Most private clouds up to this point have been custom-built environments strapped together with some automation, he said. "It was somewhat of an improvement over the DIY infrastructure, but it really wasn't private cloud."

    Continue reading
  • HPE ordered to pay Oracle $30m for Solaris IP infringement
    Years long case closes after three-week jury trial, HPE considering next step

    Hewlett Packard Enterprise must pay Oracle $30 million for copyright infringement after a jury found it guilty of providing customers with Solaris software updates without Big Red's permission.

    The decision, which HPE may contest, is the culmination of a three-week trial in Oakland, California. However, the case was first raised years back when Oracle claimed HPE had offered illegal updates under a scheme devised by software support provider Terix, which settled its case in 2015 for almost $58 million.

    In proceedings at the start of this week, Oracle’s lawyer, Christopher Yeates of Latham & Watkins LLP, pressed the eight-person jury to award his client $72 million for HPE using software not covered by a support contract, and for pinching clients, including Comcast.

    Continue reading
  • HPE Q2 revenue growth held back by supply constraints
    'However, enterprise demand continues to persist across our entire portfolio,' says CEO

    Amid a delayed HPC contract and industry-wide supply limitations compounded by the lockdown in Shanghai, Hewlett Packard Enterprise reported year-on-year sales growth of $13 million for its Q2.

    That equated to revenue expansion of 1.5 percent to $6.713 billion for the quarter ended 30 April. Wall Street had forecast HPE to generate $6.81 billion in sales for the period and didn't look too kindly on the shortfall.

    "This quarter," said CEO and president Antonio Neri, "through a combination of supply constraints, limiting our ability to fulfill orders as well as some areas where we could have executed better, we did not fully translate the strong customer orders into higher revenue growth."

    Continue reading
  • Will Lenovo ever think beyond hardware?
    Then again, why develop your own software à la HPE GreenLake when you can use someone else's?

    Analysis Lenovo fancies its TruScale anything-as-a-service (XaaS) platform as a more flexible competitor to HPE GreenLake or Dell Apex. Unlike its rivals, Lenovo doesn't believe it needs to mimic all aspects of the cloud to be successful.

    While subscription services are nothing new for Lenovo, the company only recently consolidated its offerings into a unified XaaS service called TruScale.

    On the surface TruScale ticks most of the XaaS boxes — cloud-like consumption model, subscription pricing — and it works just like you'd expect. Sign up for a certain amount of compute capacity and a short time later a rack full of pre-plumbed compute, storage, and network boxes are delivered to your place of choosing, whether that's a private datacenter, colo, or edge location.

    Continue reading

Biting the hand that feeds IT © 1998–2022