Your top 5 liquid cooling quandaries answered, according to Omdia

Liquid cooling has its advantages, but it's not for everyone, says analyst

With generation after generation of chips and systems growing ever hotter, and datacenters increasingly under pressure to reduce their impact on the climate, liquid and immersion cooling technologies have steadily gained traction.

Chipmakers and OEMs alike are investing heavily in liquid cooling. Intel, for example, plans to build a $700 million "mega lab" in Oregon to develop and study novel cooling technologies.

However, there remain questions about the necessity, reliability, cost, and underlying technologies that come with adopting liquid or immersion cooling. So we asked Omdia analyst Moises Levy what datacenter operators' most pressing questions are when it comes to deploying these technologies.

1. What's the state of the market?

According to Levy, one of the first and most frequent questions he gets from clients is about what the datacenter liquid cooling market looks like today.

Broadly speaking, liquid cooling technologies fall into two categories. The first is direct liquid cooling, which involves passing water or a dielectric fluid through copper or aluminum cold plates attached to hotspots like CPUs, GPUs, or other accelerators. This is a well-established and understood market, with many OEMs already offering systems pre-configured in this form factor. Lenovo's Jupiter and HPE's Apollo systems are two such examples.

The second, and arguably more compelling, is immersion cooling, which involves physically dunking the server into a bath of non-conductive fluid – commonly oil or two-phase refrigerants. For the most part, the companies offering this technology are startups, Levy tells us. However, he notes that more recently oil companies – experts in producing dielectric fluids – have got involved.

What's more, unlike direct liquid cooling, immersion cooling is largely vendor agnostic.

Driving development in this arena are a wave of high-TDP processors, GPUs, and AI accelerators which have dramatically changed the cooling requirements and power density of the rack, he explained.

As such, Levy predicts the liquid and immersion cooling market will top $1 billion by 2025, while the broader thermal management market is expected to crest $7.7 billion during that same time frame.

2. How reliable is it?

Understandably, many datacenter operators aren't keen on the idea of a liquid-cooled system springing a leak and shorting something out.

Fortunately, liquid cooling isn't a new concept and most direct liquid cooled systems are quite reliable – enough so that the major OEMs are willing to stand behind them. "Nowadays direct liquid cooling technologies are well proven," says Levy.

The risk of a leak, while still present, can be further mitigated through the use of non-conductive fluids. This means that even if there is a spill, it won't pose a risk to the components.

Immersion cooling, however, introduces some unknowns of which customers need to be aware, Levy says. "Not all companies are producing servers that are guaranteed to operate in immersion."

This is a problem because the dielectric liquid used in these systems is relatively new and may breakdown or degrade certain components if they weren't designed to operate immersed in an oil or two-phase refrigerant.

3. Will this help me achieve my sustainability goals?

"Absolutely," Levy assures us. "At the end of the day, we're having more efficient systems," as a result of liquid cooling.

It's well known that liquid cooling has the potential to reduce the power required to keep systems cooled significantly compared to traditional air cooling. As much as 40 percent of a datacenter's power consumption can be directly attributed to running the air conditioning necessary to keep the servers at operating temperature.

However, liquid and immersion cooling are not a silver bullet for sustainability. While they can drastically reduce datacenter water consumption and cut the power required to cool the systems, customers should not confuse sustainability with efficiency, Levy says.

For example, an electric vehicle may be more sustainable, but if the driver has to make multiple trips to the store to get everything they need, it could end up being less efficient than a combustion engine-powered vehicle whose driver can finish their shopping in a single trip.

The same is true of datacenters, Levy says. "We can have the most efficient and sustainable datacenter, but if we're not producing, if we're not processing workloads, we're missing something."

While liquid and immersion cooling can improve the efficiency of the datacenter, operators should consider the big picture when talking about sustainability, he argues.

4. Is liquid cooling the best option for me?

"There is no one size fits all," when it comes to liquid or immersion cooling, Levy says.

Not every datacenter needs or will even benefit from liquid cooling, he explains. Two big factors that play into whether or not operators should consider liquid cooling, according to Levy, are datacenter location and workload.

"For example, if you're one of the Nordic countries, you have access to free cooling," Levy says, referencing environments where air conditioning may not be required due to the low ambient air temperature.

He adds that, depending on the location, datacenters in the US or elsewhere in the world could take advantage of this to offset power consumption by piping in cool evening air rather than relying on air conditioning 24/7.

However, if the datacenter isn't located in an environment where "free cooling" is a possibility, customers should also consider their workloads. Air cooling is still effective up to 40–50 kilowatts per rack, Levy says.

"Nowadays with high-performance computing, more compute-intensive workloads, we're seeing more and more power density," he said. "Now a server can hold hundreds or thousands of watts."

For customers that are deploying massive HPC or AI/ML workloads, it may make sense to liquid cool those systems, while continuing to rely on air cooling for other less-demanding systems.

5. What's it gonna cost me?

Location and workload also factors heavily into the cost of ownership and customers' return on investment, Levy says.

Customers always want to know what it's going to cost them to deploy liquid-cooled systems in their datacenters and how long it will take to pay that investment off. Unfortunately, there isn't always a straightforward answer, he adds.

"If you're adopting this technology because you don't have any other option – for example, the land is too expensive, or you don't have power capacity from other sources – that [investment] will pay off," Levy says.

However, it's a different story entirely when it comes to large hyperscale and cloud providers, many of which are already dabbling in liquid and immersion cooling tech. "For the big cloud service providers who are leading this race. It's not about cost. It's about the long term."

In other words, if the cloud providers can achieve higher system and power densities by using liquid cooling, that drives down the cost of compute.

Meanwhile, for colocation providers, liquid cooling is being deployed out of necessity rather than desire, Levy notes, adding that a failure to embrace the technology could lose them valuable customers that want or need liquid cooling to support AI/ML workloads.

There are also operational costs to using liquid cooling that need to be taken into account.

"People sometimes don't know they need a filtering system, they need software, they will be tracking the quality of dielectric fluids," he says. "It's another type of monitoring which requires higher skilled labor."

All of which means there is a lot to consider before you jump straight in. ®