IBM has delivered a water-cooled supercomputer to the Zurich ETH* university, saying it uses up to 40 per cent less energy than an air-cooled machine, and has a reduced carbon footprint because its waste heat is used to warm adjacent buildings.
The 6Tflop experimental system is called Aquasar and has been designed and built by IBM researchers in Zurich. It is built from a combination of Cell and Nehalem processors, and has three chassis or frames, two of which use water-cooling.
Each chassis has 11 BladeCenter QS222 servers, each of which has two PowerXCell 8i processors, and three BladeCenter HS22 servers, each of which has two Intel NehalemEP processors. In general the PowerXCell 8is, which are also used in the Los Alamos Roadrunner system, seem to be intended for the massive number-crunching involved in the apps that Aquasar will run.
The PowerXCell 8i, introduced in May 2008, is based on cell microprocessor technology developed by IBM, Sony and Toshiba. It is a high-performance, double-precision floating point product with a 65nm feature size.
The water-cooled chassis feature water-blocks, cooling elements attached to the processors, which have water at 60 degrees C circulating through capillary tubes and picking up heat from the processors, rising to 65 degrees C and keeping the processors below their 85 degrees C upper limit. The water then goes through a heat exchanger, goes back down to 60 degrees, passing nine kilowatts of thermal power into the ETH building heating system, and returns to the processors.
In general there's nothing fancy here, IBM water-cooling this super-computer like it has mainframes. But intensive effort has been put into the design of the water-blocks to maximise the heat transfer from processors to the water.
Water-cooling is more efficient than air-cooling, and overall uses less energy than an air-cooling system based on IT equipment operating inside a cooled data centre. IBM reckons Aquasar will help it build more energy-efficient supercomputers, and quotes Professor Dimos Poulikakos, the Thermodynamics laboratory head at ETH: "In the future it will be important to measure how efficiently a computer is per watt and per gram of equivalent CO2 production."
Aquasar is said to have its carbon footprint reduced by up to 85 per cent because its waste heat is used to warm buildings. However, this has Aquasar's carbon footprint offset by the saved carbon that would have been used in heating the buildings, and Aquasar's waste heat is not much use in the summer. The better direct measure is that the Aquasar installation uses up to 40 per cent less energy than if it were air-cooled.
IBM researchers will monitor Aquasar's operation in ETH's Department of Mechanical and Process Engineering , comparing and contrasting the air-cooled and water-cooled chassis to further optimise the water-cooling system.
In the future it may be possible to over-clock supercomputer processors by using water-cooling to keep their temperature inside operating limits, and so gain both performance and energy consumption benefits. ®
ETH stands for Eidgenössische Technische Hochschule which boasts 21 Nobel Prize Laureates amongst its researchers, teachers and graduates.