Intel plans immersion lab to chill its power-hungry chips

AI chips are sucking down 600W+ and the solution could be to drown them.

Intel this week unveiled a $700 million sustainability initiative to try innovative liquid and immersion cooling technologies to the datacenter.

The project will see Intel construct a 200,000-square-foot "mega lab" approximately 20 miles west of Portland at its Hillsboro campus, where the chipmaker will qualify, test, and demo its expansive — and power hungry — datacenter portfolio using a variety of cooling tech.

Alongside the lab, the x86 giant unveiled an open reference design for immersion cooling systems for its chips that is being developed by Intel Taiwan. The chip giant is hoping to bring other Taiwanese manufacturers into the fold and it'll then be rolled out globally.

As the name suggests, immersion cooling involves dunking components in a bath of non-conductive fluid — mineral oil and certain specialized refrigerants being two of the most common — as opposed to using heat sinks or cold plates to move heat away from the chips. Intel claims its novel ideas on this established technology could reduce datacenter carbon emissions by 45 percent.

This is a big step forward for datacenter sustainability, Dell'Oro Group analyst Lucas Beran told The Register.

While the individual components and servers consume a substantial amount of power, keeping them cool accounts for upwards of 40 percent of a datacenter's energy consumption, he explained. "Simply reducing the energy consumption is a really big part of what liquid cooling and, more specifically, immersion cooling brings to the table."

Beyond, curbing energy consumption, these technologies offer several ancillary benefits. One is a substantial reduction in water usage, while another is that liquid cooling is just better than air at moving heat away, and can even be reused for things like district heating, Beran said.

Bytesnet, for example, recently announced plans to recycle heat generated by its datacenters to warm thousands of homes in the Groningen district of the Netherlands.

Balancing power, performance, and heat

One of the driving forces behind Intel's latest datacenter sustainability efforts is a trend toward ever higher power consumption by upcoming CPUs, GPUs, and AI accelerators.

Over the past few years, the thermal design power (TDP) commanded by many of these chips has more than doubled. Today, modern CPU architectures are pushing 300W, while GPUs and AI chips from Intel, AMD, and Nvidia are now sucking down 600W or more.

As these systems proliferate and find their way into mainstream datacenters, liquid or immersion cooling will eventually become unavoidable, not just to keep these systems from overheating, but to offset their exponential power consumption, Beran explained.

He highlighted one datacenter that adopted immersion cooling not because they were thermally bound, but because it enabled them to reallocate much of the power used to cool the systems to additional compute density. And this is how Beran expects most datacenter operators will approach immersion cooling in the near future.

"If you transition from a traditional rack air cooling system, to an immersion cooling system, you could just use less power," he said. But "I don't see many too many instances where folks say…'We have enough compute, we're just trying to cool it more efficiently.'"

More often than not, operators are running into trouble getting enough power into the rack, Beran added.

Can Intel drive adoption?

While immersion cooling isn't new, Beran argues Intel's involvement in developing an open reference design is still notable.


Cool building bro. Click to enlarge

"They play a really important role in developing technologies that are compatible with immersion cooling," he said. "It has the potential to have a massive impact because they can influence the server OEMs, like Dell or HPE, in terms of how they sell their products and the type of cooling infrastructure needed to support those products."

"Now they are designing products that will be born in liquids and born in immersion cooling," Beran added.

This is important because liquid and immersion cooling requires completely different form factors than what's used in air cooled datacenters today. This unfamiliarity remains one of the technology's biggest inhibitors.

The unknown is scary and many datacenter operators don't yet have a playbook for how to handle the various issues that can crop up in liquid and immersion cooled hardware. One of the most common concerns, Beran said, is associated with the tech is weight distribution, though it's rarely a problem.

"A big facility like this, a big playground if you will, where you can go and actually see this infrastructure firsthand, dip your toes in the fluid so to speak, understand how these systems operate in a datacenter like environment, has tremendous value to the industry," he said of the Intel lab. ®

Broader topics

Other stories you might like

  • Linux Foundation thinks it can get you interested in smartNICs
    Step one: Make them easier to program

    The Linux Foundation wants to make data processing units (DPUs) easier to deploy, with the launch of the Open Programmable Infrastructure (OPI) project this week.

    The program has already garnered support from several leading chipmakers, systems builders, and software vendors – Nvidia, Intel, Marvell, F5, Keysight, Dell Tech, and Red Hat to name a few – and promises to build an open ecosystem of common software frameworks that can run on any DPU or smartNIC.

    SmartNICs, DPUs, IPUs – whatever you prefer to call them – have been used in cloud and hyperscale datacenters for years now. The devices typically feature onboard networking in a PCIe card form factor and are designed to offload and accelerate I/O-intensive processes and virtualization functions that would otherwise consume valuable host CPU resources.

    Continue reading
  • Semiconductor industry growth to slow in 2022, warns IDC
    Chip price hikes keeping sector healthy but new fabs could lead to 'overcapacity'

    The global economy may be in a tenuous situation right now, but the semiconductor industry is likely to walk away from 2022 with a "healthy" boost in revenues, according to analysts at IDC. But beware oversupply, the analyst firm warns.

    Semiconductor companies across the world are expected to grow collective revenues by 13.7 percent year-on-year to $661 billion, IDC said in research published Wednesday. Global semiconductor revenue last year was $582 billion.

    "Overall, the semiconductor industry remains on track to deliver another healthy year of growth as the super cycle that began in 2020 continues this year," said Mario Morales, IDC group vice president of semiconductors.

    Continue reading
  • Will optics ever replace copper interconnects? We asked this silicon photonics startup
    Star Trek's glowing circuit boards may not be so crazy

    Science fiction is littered with fantastic visions of computing. One of the more pervasive is the idea that one day computers will run on light. After all, what’s faster than the speed of light?

    But it turns out Star Trek’s glowing circuit boards might be closer to reality than you think, Ayar Labs CTO Mark Wade tells The Register. While fiber optic communications have been around for half a century, we’ve only recently started applying the technology at the board level. Despite this, Wade expects, within the next decade, optical waveguides will begin supplanting the copper traces on PCBs as shipments of optical I/O products take off.

    Driving this transition are a number of factors and emerging technologies that demand ever-higher bandwidths across longer distances without sacrificing on latency or power.

    Continue reading
  • Intel offers 'server on a card' reference design for network security
    OEMs thrown a NetSec Accelerator that plugs into server PCIe slots

    RSA Conference Intel has released a reference design for a plug-in security card aimed at delivering improved network and security processing without requiring the additional rackspace a discrete appliance would need.

    The NetSec Accelerator Reference Design [PDF] is effectively a fully functional x86 compute node delivered as a PCIe card that can be fitted into an existing server. It combines an Intel Atom processor, Intel Ethernet E810 network interface, and up to 32GB of memory to offload network security functions.

    According to Intel, the new reference design is intended to enable a secure access service edge (SASE) model, a combination of software-defined security and wide-area network (WAN) functions implemented as a cloud-native service.

    Continue reading

Biting the hand that feeds IT © 1998–2022