Data Centre Infrastructure for AI Demand

Data centre infrastructure can keep up with AI demand more efficiently by shifting from a general-purpose, CPU-centric design to a purpose-built, accelerator-first architecture that prioritises power density, liquid cooling, and software-defined resource orchestration. This means planning for 50–100 kW per rack instead of the traditional 5–10 kW, deploying direct-to-chip or immersion cooling, and using schedulers that place AI workloads on the right hardware at the right time. No single upgrade solves the problem; the gains come from co-designing power, cooling, and compute as one system.

Why AI Changes the Infrastructure Equation

Traditional data centres were built for web servers, databases, and virtual machines—workloads that are latency-sensitive but relatively low-power. A typical rack ran at 5–10 kW, and air cooling was sufficient. AI training clusters invert every assumption: they run thousands of GPUs or TPUs at near-total utilisation for days or weeks, drawing 30–100+ kW per rack and generating heat that air handling cannot remove economically. The shift is not incremental; it is a change in the physical limits of what a facility can deliver.

Inference workloads (running a trained model) are less intense but still far above traditional levels, and they require low latency, which pushes compute to the edge. The result is a bifurcated infrastructure: hyperscale training centres with extreme density, and edge nodes with compact, efficient accelerators. Both need different power and cooling strategies than a standard colocation hall.

Power Delivery: From 480V to 48V and Beyond

AI servers draw more current than older equipment, and the standard 208V or 240V distribution loses efficiency when converting down to the 12V or 48V that processors need. The industry is moving toward higher-voltage distribution (up to 400V DC in some designs) and a 48V bus inside the rack to reduce copper losses and improve power supply efficiency from around 94% to 97% or higher. This is not just a component change; it affects UPS design, busbar sizing, and the physical weight of cabling.

Another practical lever is power capping and scheduling. AI workloads are often flexible: a training job can pause and resume. A scheduler can shift jobs to times when renewable energy is abundant or when the grid is less strained, effectively using the data centre as a demand-response resource. This does not reduce total energy but it can lower cost and carbon intensity.

Cooling: The Bottleneck That Decides Location

Air cooling reaches its limit around 20–25 kW per rack. Above that, you need liquid cooling. Two main approaches are in production:

• Direct-to-chip (cold plate): a liquid loop runs through a metal plate mounted on the CPU/GPU, removing heat directly. It is retrofittable to many existing racks and uses about 40% less energy than air cooling.
• Immersion cooling: servers are submerged in a dielectric fluid, which absorbs heat with no fans. It handles 100+ kW per rack and can reduce cooling energy by up to 50%, but it requires specialised tanks and changes how you service hardware.

The choice depends on whether you are building new or retrofitting. Immersion is more efficient but harder to adopt; cold plates are the pragmatic middle ground. Either way, the heat can be reused—for building heating or even district heating—turning a waste product into an asset. That is a growing requirement in Europe, where energy costs are high and carbon reporting is mandatory.

Resource Orchestration: Software Is the Hidden Efficiency

Hardware is only half the story. AI workloads are often run on clusters where utilisation is surprisingly low—studies of production clusters show average GPU utilisation between 50% and 70% in many organisations, meaning nearly half the expensive hardware sits idle. The fix is a cluster scheduler that understands AI jobs: it can pack multiple small jobs onto one GPU, preempt low-priority work, and use checkpointing to resume interrupted training without losing progress.

Another software lever is model compression—quantisation, pruning, and distillation can shrink a model by 10–100x with little accuracy loss, which directly reduces the compute and memory needed at inference time. This is often cheaper than buying more GPUs. For example, running a quantised 7B-parameter model on a single consumer GPU is feasible, whereas the full-precision version would need a multi-GPU server.

How to Choose the Right Strategy

There is no universal answer; the right mix depends on your workload and budget. Consider these factors in order:

1. Workload type: training (bursty, high-power, long-running) vs. inference (steady, low-latency, many small requests).
2. Existing facility: can you upgrade power and cooling, or is a greenfield build cheaper?
3. Power cost and carbon goals: if energy is expensive, invest more in cooling efficiency and renewable procurement.
4. Utilisation: measure current GPU/CPU usage for a month before buying more hardware; scheduling fixes may be enough.
5. Future growth: design for 2–3x the density you need today, because AI models are not shrinking in size.

For most organisations, the fastest win is not new hardware but better scheduling and quantisation. The second is retrofitting cold-plate cooling on existing high-density racks. Only then consider new construction.

Common Problems and Misconceptions

A frequent mistake is assuming that a higher power capacity automatically means more compute. Power delivery and cooling must be matched to the actual hardware; a 100 kW rack with air cooling will throttle the GPUs, wasting the investment. Another misconception is that liquid cooling is risky or exotic—modern systems use non-conductive fluids and are designed for leak-free operation, and major cloud providers already deploy them at scale.

A second problem is stranded capacity: you may have 50 kW per rack of power but only 20 kW of cooling, so you cannot use the full power. Always plan power and cooling as a pair. A third issue is the power usage effectiveness (PUE) trap: PUE measures how much extra energy the facility uses for cooling and power distribution, but it does not measure how efficiently the IT equipment runs. A PUE of 1.1 with 50% GPU utilisation is worse than a PUE of 1.3 with 90% utilisation. Focus on total workload per kilowatt, not just the facility metric.

Costs and Trade-offs

The capital cost of a liquid-cooled AI rack is roughly 10–20% higher than an air-cooled equivalent, but the operating savings in cooling energy and the ability to run denser hardware usually pay back within 1–3 years. Immersion cooling has a higher upfront cost (specialised tanks and fluid) but the lowest ongoing energy use. The bigger cost is often the power connection itself—upgrading a facility from 1 MW to 5 MW can involve new transformers and utility fees that dwarf the IT hardware cost.

For smaller operators, the alternative is to avoid owning infrastructure entirely: rent GPU time from a cloud provider. This shifts the efficiency problem to the provider, who can amortise cooling and power investments over many customers. The trade-off is data transfer costs and latency, which matter for real-time inference but not for batch training.

Safety and Reliability Considerations

Liquid cooling introduces new safety considerations. Dielectric fluids are generally non-toxic but can be slippery on floors; containment trays and leak detection are essential. Electrical safety changes too: higher DC voltages (400V) require different training for technicians and careful grounding. Fire suppression in liquid-cooled rooms should use inert gas or water mist, not dry chemical, because the latter can damage electronics. Always follow manufacturer guidelines for fluid handling and disposal, and ensure that cooling loops are isolated from potable water systems.

Reliability is another angle: AI training jobs can run for weeks, and a single cooling pump failure can destroy thousands of dollars of compute time. Design for redundancy in cooling pumps and power feeds, and implement automatic checkpointing so a failure restarts from the last saved state, not from zero.

Frequently Asked Questions

Can I retrofit liquid cooling into an existing air-cooled data centre?

Yes, but it is not trivial. Direct-to-chip cold plates can be installed on many existing servers if the rack has enough space for the coolant manifolds. You will need to add a coolant distribution unit (CDU) and piping, and you must verify that the existing power distribution can handle the higher density. Immersion cooling is harder to retrofit because it requires removing servers from racks and placing them in tanks, which changes cabling and service procedures.

Is air cooling ever sufficient for AI workloads?

For small inference workloads and edge deployments, yes. A single GPU server drawing 2–3 kW can be air-cooled in a standard rack. But for training clusters with many GPUs, air cooling becomes inefficient above roughly 20 kW per rack—you would need massive airflow, which uses more fan energy and still cannot remove heat fast enough. If you plan to run modern AI accelerators at scale, liquid cooling is the practical path.

What is the single most cost-effective way to reduce AI energy use?

Using model quantisation and pruning before you buy hardware. Reducing the model size by 4x can cut inference energy by a similar factor, and it often requires no new infrastructure. For training, the biggest lever is improving GPU utilisation through better scheduling—many clusters run at 50–70% utilisation, and closing that gap is essentially free compute.

How does PUE relate to AI efficiency?

PUE measures facility overhead, not IT efficiency. A data centre with a PUE of 1.1 is very efficient at powering and cooling, but if the GPUs sit idle half the time, the overall efficiency is poor. For AI, you should also track GPU utilisation and workload throughput per kilowatt. A slightly worse PUE with much higher utilisation is often the better trade.

Should I build my own AI data centre or rent cloud capacity?

It depends on scale and predictability. If you have steady, large workloads and can invest in power and cooling, owning gives you lower marginal cost per GPU-hour. If your workloads are variable or you are experimenting, cloud rental avoids the huge capital outlay and lets the provider handle efficiency upgrades. Many organisations start with cloud, then move to owned infrastructure once usage is stable.

Related

• Samsung to Xiaomi Transfer: How to Move Your Data
• AI Agent System Access: What to Log and Monitor
• Prison Escape Codes: What They Are & How They Work