Cooling & Thermal Management: The Other Half of the Build
Any apprentice could raise a fire. It was the master smith who understood the quenching — that the blade was not made in the heat, but in how carefully the heat was drawn back out of it. The forge that could not cool its work made nothing but slag.
The Heat Problem
The previous pieces were about getting power into the data centre. This one is about the law of physics that follows immediately: nearly every watt that goes in comes back out as heat, and it has to go somewhere. For most of computing history that was a trivial afterthought — a few fans, some cold air. AI broke that assumption in a single hardware generation.
The driver is rack density. A traditional server rack draws 5 to 15 kilowatts, and air cooling handles it comfortably. But an NVIDIA Blackwell GB200 rack draws around 140 kW — roughly ten times as much heat, in the same physical footprint. Air simply cannot carry that much energy away fast enough; at those densities, chips throttle their own clock speeds to avoid cooking, and the expensive GPUs you paid for stop delivering the performance you bought. Next-generation designs push toward 900 kW per rack and single chips near 2,000 watts. The industry’s own thermal standards body now recommends liquid cooling above 20 kW per rack — a line virtually every serious AI deployment crossed in 2025.
At modern AI densities, cooling is no longer a feature of the building. It is the building.
Once a rack crosses roughly 50–140 kW, the cooling architecture dictates the physical design of the entire facility — the floor loading, the piping, the power draw, the water supply, even the site selection. You do not build a data centre and then cool it; you design the cooling and wrap a data centre around it. That inversion is what turned thermal management from a maintenance line item into a distinct, investable layer of the AI build-out.
The Technologies — and How They Compare
The shift underway is from moving air to moving liquid, because liquid carries heat up to a thousand times more effectively. There are three broad approaches, in ascending order of density and complexity:
- Direct-to-chip (cold plate). A metal plate sits directly on the GPU, and coolant flows through it, carrying heat away at the source. It retrofits into existing racks with minimal disruption, which is why it dominates today — more than half the market.
- Immersion cooling. Entire servers are submerged in a non-conductive dielectric fluid. It handles the highest densities and slashes water use, but requires purpose-built tanks and specialised fluids — a bigger commitment.
- Air (the baseline). Still the incumbent for lower-density workloads, increasingly supplemented by rear-door heat exchangers as a transitional step.
The efficiency gap is stark, and it is measured in PUE — power usage effectiveness, the ratio of total facility power to the power actually reaching the computers. A PUE of 1.0 is the theoretical ideal (no overhead); everything above it is energy spent on cooling and losses.
| Method | Rack density | Typical PUE | Water use | Best fit |
|---|---|---|---|---|
| Air (containment) | Up to ~15 kW | 1.5–1.8 | High (evaporative) | Legacy / low-density |
| Rear-door exchanger | ~20–40 kW | 1.35–1.55 | Moderate | Transitional retrofits |
| Direct-to-chip | ~40–140 kW | 1.15–1.30 | Varies by heat-reject | Mainstream AI (today) |
| Single-phase immersion | 100 kW+ | 1.03–1.08 | 90–98% less | Max density / water-scarce |
The likely future is not one winner but a dual track: direct-to-chip serving the mainstream because it retrofits easily, immersion taking the ultra-dense and water-constrained deployments. Both are liquid; the air era is ending for anything running AI.
The Market — and the Business Model Underneath It
Cooling has crossed its inflection point. Liquid cooling penetration was around 3% in 2021; by 2026 it is roughly 37% — a more-than-tenfold jump in five years, driven by hardware that leaves no choice.
Today the market is roughly 70–80% hardware (the cooling systems themselves) and 20–30% services. Industry forecasts expect that mix to invert over the next four to five years, toward services-led revenue — monitoring, maintenance, fluid management, thermal-as-a-service. That shift matters more than the headline growth rate: hardware sales are cyclical and competitive, but the recurring service contract attached to a mission-critical cooling loop is an annuity. In the primer’s language, it is the difference between a one-off transaction and fee-bearing, recurring revenue — and the latter is what re-rates a business.
The Water Tradeoff — Where Build Meets Strain
Here is the catch that turns a growth story into a constraint. The cheapest way to reject heat is evaporative cooling — letting water evaporate to carry heat away — and it is thirsty. A conventional evaporatively-cooled data centre consumes on the order of 2 to 5 million gallons of water per megawatt, per year. Scale that across a gigawatt-class AI campus and the number becomes a genuine claim on a regional water supply.
In the American Southwest — Phoenix, Las Vegas, much of Texas — water rights are finite and increasingly contested, and local permitting reviews now scrutinise a data centre’s water draw as closely as its power draw. Water, in other words, has joined the interconnection queue as a gate that can stop a project before it starts. This is where the cooling choice becomes a siting decision, and where the Build thread runs straight into the Strain thread.
The escape is a genuine three-way tradeoff, with no free option:
- Evaporative cooling: lowest energy, highest water — fine where water is cheap, disqualifying where it isn’t.
- Dry / adiabatic coolers: minimal water, but higher energy use — you trade the water bill for the power bill.
- Immersion: cuts water use 90–98% and improves efficiency — but demands specialised fluid and purpose-built design.
Reading It Through the Frameworks
Where is the moat? Not in the commodity hardware — cold plates and pumps will commoditise. It sits in three places: proprietary thermal IP and the coolant-distribution systems that are hard to replicate; the retrofit lock-in of a cooling loop that, once installed, is expensive to swap; and the service annuity attached to it. Own the recurring relationship, not the box.
Where does policy become the cash flow? In two ways. Water-permitting rules increasingly mandate low-water cooling in scarce regions, effectively legislating demand for immersion and dry cooling. And PFAS regulation cuts the other way — the phase-out of certain fluorochemical fluids used in two-phase immersion creates a real transition risk for that specific technology, and a cost advantage for single-phase and fluid-free approaches.
Cooling is the half of the AI build-out that the power headlines skip, and it has quietly become mandatory infrastructure: at Blackwell densities, air cooling simply doesn’t work, so liquid is not a choice but a requirement. That has turned a maintenance line item into a sub-sector growing toward $27 billion, with the most durable value in the recurring service loop rather than the hardware.
But cooling is also where the build meets its limits. Every megawatt of AI compute is a claim on water as well as power, and in the places the data centres most want to be, water is exactly what’s scarce. The winners will be the ones who master the quenching — who reject heat without draining a river — and sell that capability as an annuity, not a box.
The great forges were not built beside the richest ore, nor the strongest fire. They were built beside cold, running water — for the masters knew that what a forge could make was limited, in the end, only by how well it could be cooled.
Leave a Reply