Nvidia's next-gen AI GPUs will need 600kW per rack

Nvidia's next-gen AI GPUs will need a jaw-dropping 600kW of power per rack. Here's how liquid cooling keeps AI servers cool in data centres.

GPUs: Bigger, better, hotter

Keeping powerful GPU servers cool is a challenge. Due to how much power they need, the heat they produce is prodigious.

Did you see the new GPUs that Nvidia just talked about? It needs 600kW for a single server rack - or 600,000 watts. It's rising so quickly that normal data centres cannot cope.

Power consumption per rack:

• Estimate for 2018-2022: 5kW
• Average in 2023: 6kW
• Average in 2024: 8kW
• One rack of H200 GPUs: ~50kW
• GB200 NVL72 (2025): ~130kW
• Rubin Ultra NVL576 (2027): 600kW*

*Announced by Nvidia earlier today.

Be cool, not hot

As I've written previously, AI data centres are looking to liquid cooling. After all, liquid has 4x the heat capacity of air for the same mass. And liquid masses a lot more.

To be clear, air cooling is probably good enough for organisations deploying a handful of GPU servers. Or even hundreds of them.

Air cooling offers advantages such as:

• Straightforward to deploy.
• No need to worry about leaks.
• Supported by most data centres.

Liquid cooling in action

What if you prefer liquid cooling? How does it work in the data centre? At SuperComputing Asia 2025 last week, I took photos from Asus and Vertiv to show you.

Here's how "direct to chip" liquid cooling looks like.

Chilled liquid flows from cooling equipment in the data centre to the CDU (Cooling Distribution Unit), which is used to cool liquid in a secondary loop.

This is then piped to server racks where it is distributed via manifolds to individual servers. Inside each server, the liquid is channelled through specialised heat sinks.

As heat transfers to the liquid, the (warmed-up) liquid is piped back through the manifold back to the CDU where it is cooled again.

As with all technologies, proper care must be taken to implement liquid cooling properly. I'll write more in a future post.