Why chasing the AI dragon may force big tech to take sustainability seriously
Carbon offsets don't make you green when your datacenters are still rolling coal
Comment It's hard to suppress an eyeroll when the world's largest consumers of datacenter resources talk about sustainability. Putting the planet ahead of profits is often not at the top of the to-do list in large-scale, performance-driven environments.
Sure, the hyperscalers talk a good game. Carbon offsets, green bonds, and lofty carbon neutrality targets aren't nothing. But it's also the kind of thing that can only be measured on a spreadsheet. And it's certainly not the kind of tangible change that's needed.
It's great that these companies are offsetting their energy use by funding renewable energy projects, but it hardly feels sincere when your datacenters are still rolling coal.
Imagine if your company's sustainability director justified driving a gas guzzler because they invested in green power companies. It'd feel just a little hypocritical, wouldn't it?
And yet the past few weeks have left a glimmer of hope that the days of tech companies greenwashing their impact on the climate may be changing.
The announcements surrounding Computex and even the International Supercomputing Conference (ISC) this spring had a decidedly green undertone.
The issue of datacenter sustainability dominated Supermicro's Computex keynote, Nvidia unveiled new liquid-cooled accelerators, and Intel shared its plan to develop novel immersion-cooling tech in Oregon.
This is the kind of talk that usually crops up around Earth Day, not tech-heavy conferences like Computex or ISC. And at $700 million, Intel's new lab doesn't feel like a PR stunt. If it is, someone seriously needs to take Pat Gelsinger's company credit card away.
You mean hotter chips are a good thing?
The rise of AI/ML workloads are driving a semiconductor arms race for what chipmakers say will soon be a half-trillion-dollar market. And to compete, the latest CPUs, GPUs, and AI accelerators from the likes of Intel, Nvidia, AMD, and others are pulling down more power than ever before.
Immersion cooling no longer reserved for the hyperscalers, HPCREAD MORE
TDPs for these chips have effectively doubled over the last generation, now topping out at 700W in the case of Nvidia's H100 SXM GPUs. AMD and Intel aren't far behind. AMD's MI250X GPUs and Intel's Habana Gaudi2 AI accelerators all command around 600W of power under full load.
The good news is that power looks like it's going to translate into more grunt. We saw this at ISC when Los Alamos National Laboratory's AMD EPYC and Instinct-based Frontier supercomputer claimed not just the number-one spot on the TOP500, but the Green500 as well.
Unfortunately, that performance isn't free. You can't shove that much power into a single chip without it getting hot. So hot, in fact, that vendors are now seriously reassessing how they cool them. Liquid cooling isn't just for high-performance computing, hyperscalers or PC enthusiasts any more.
- All-AMD US Frontier supercomputer ousts Japan's Fugaku as No. 1 in Top500
- Compute responsibly: Yet another IT industry sustainability drive
- Small nuclear reactors produce '35x more waste' than big plants
- Immersion cooling no longer reserved for the hyperscalers, HPC
- Microsoft datacenter to heat homes in Finland
As strange as this might sound, this is great news for datacenters, where cooling accounts for upwards of 40 percent of their power consumption.
Liquid and immersion cooling tech is more efficient at moving heat around than air, Dell'Oro analyst Lucas Beran told The Register in an earlier interview. "Simply reducing the energy consumption is a really big part of what liquid cooling and, more specifically, immersion cooling brings to the table."
The tech also has other environmental benefits that may make it attractive to datacenter operators, including lower water consumption – a common sustainability target – and the opportunity to use datacenters to heat homes in the region.
That last one isn't theoretical either. Datacenters in the Netherlands and Finland are already recycling waste heat to warm thousands of homes.
More importantly, there's now a strong economic incentive for adopting these technologies. If you want to compete, you need to deploy the best chips, and if you want to deploy those chips without your energy bill taking a hike, you're going to need more efficient cooling.
This appears to have been the impetus behind Intel's $700 million liquid-cooling lab and immersion-cooling reference design, unveiled last month. If the chipmaker is to be believed, its take on this old concept can reduce datacenter carbon emissions by 45 percent. Intel wouldn't have dropped that kind of cash if it wasn't serious about bringing the tech to a mass market.
Faced with 300W CPUS and 700W GPUs coming down the pipe, OEMs like Lenovo, Supermicro, HPE, and others have been only too happy to parade their latest liquid-cooled chassis and rack infrastructure.
Where are my nuclear-powered datacenters?
With all that said, there's still plenty to be done. What about datacenters powered by wind, solar, hydroelectric, and other sustainable energy sources. There are miniature nuclear reactors too, though the jury is still out on their pros and cons.
There are also opportunities to reduce e-waste. Modular chassis and blade servers that can be upgraded rather than replaced are a good start. Supermicro's universal GPU servers and Nvidia's HGX reference designs are just two recent examples.
There are also intriguing efforts to improve system utilization by virtualizing GPUs or optimizing applications at runtime. What good are super-fast servers if they're sitting idle most of the time? Software has to be part of the solution.
But let's be clear – any change won't be for the good of mankind or to hold back an impending climate collapse. We as a species are too short-sighted for that, especially in the face of profits.
Ultimately, it comes down to money. Whether it's green or not is a public relations issue, regardless of how sincere these companies' climate initiatives may or may not be.
Only when there is an indisputable economic incentive to adopt green tech will we see these companies dispense with the lip service.
It seems like between fractured supply chains, rising energy prices, power-hungry silicon, and the insatiable demand for more compute capacity, the economics have changed, and green infrastructure is finally on the menu. ®