AI Data Center Requirements: What Makes a Site Suitable

A data center suitable for AI workloads must deliver extreme power density, low-latency high-bandwidth networking, and liquid cooling capability, because AI training clusters draw 5–10 times more power per rack than traditional servers and generate heat that air cooling cannot remove. You need to evaluate power infrastructure, cooling design, network topology, and physical site constraints before committing to a location. This article explains the technical requirements, how to assess a facility, and the common pitfalls to avoid.

Formula

The total facility power required for an AI data center is calculated as:

Total Facility Power (MW) = (GPU Power per Rack (kW) × Number of Racks) / 1000 × PUE

Where:

• GPU power per rack (kW) – the average power draw of one rack filled with AI servers (typically 30–120 kW for modern GPU clusters).
• Number of racks – the total count of racks in the facility.
• PUE (Power Usage Effectiveness) – the ratio of total facility power to IT equipment power. A PUE of 1.3 means 30% overhead for cooling, power distribution, and losses.

The IT load is simply the product of per-rack power and rack count, divided by 1000 to convert kW to MW. The cooling overhead is the difference between total power and IT load.

Worked Example

Using the default values from the calculator: 40 kW per rack, 100 racks, and a PUE of 1.3.

1. IT load = (40 kW × 100 racks) / 1000 = 4 MW
2. Cooling overhead = 4 MW × (1.3 – 1) = 1.2 MW
3. Total facility power = 4 MW × 1.3 = 5.2 MW

The calculator rounds to two decimals, so the result is 5.20 MW total, 4.00 MW IT load, and 1.20 MW cooling overhead.

Power Density: The Core Difference

Traditional enterprise data centers typically run 5–10 kW per rack. AI training clusters, especially those using NVIDIA H100 or A100 GPUs, require 30–40 kW per rack for air-cooled configurations and 60–120 kW per rack for liquid-cooled ones. This density shift affects everything from the electrical feed to the physical floor.

Key power considerations:

• Utility capacity – The site must have access to enough grid power, often 10–100 MW for a large AI facility. This may require new substations or direct high-voltage connections.
• Redundancy – AI workloads are typically run as long-running jobs that cannot tolerate interruption. Facilities often use 2N (duplicate) or 2N+1 (duplicate plus spare) UPS and generator systems.
• Voltage and distribution – Higher rack densities may require 415V or 480V three-phase power to reduce current and copper losses.
• Power factor correction – GPU power supplies can have poor power factor if not corrected, leading to higher distribution losses.

When evaluating a site, check the utility provider’s ability to deliver the required capacity and the lead time for any upgrades. Some regions have interconnection queues that can take years.

Cooling: Why Air Is Not Enough

At 40 kW per rack, air cooling becomes inefficient and often impossible. Liquid cooling is the standard for AI data centers because water has a much higher heat capacity than air. Two main approaches exist:

• Direct-to-chip cooling – Coolant flows through cold plates mounted on the GPU and CPU, removing heat directly from the silicon. This can handle 60–120 kW per rack.
• Immersion cooling – Servers are submerged in a dielectric fluid. This offers even higher densities but requires specialized hardware and maintenance.

For air-cooled facilities, the maximum practical density is around 30 kW per rack with hot/cold aisle containment. Above that, liquid cooling is necessary. A suitable site must have space for chillers, cooling towers, or dry coolers, plus a reliable water supply if using evaporative systems. Some locations use free cooling (outside air) during cooler months, which reduces energy use but adds humidity control complexity.

Networking and Latency

AI training involves thousands of GPUs communicating constantly. The network must have extremely high bandwidth and low latency to avoid idle GPU time. Key factors:

• Fabric topology – Most AI clusters use a fat-tree or torus topology with 400 Gbps or 800 Gbps links between switches. The data center must have enough space and power for the network switches (which can be 10–20% of total power).
• Proximity to users – For AI inference (running models), latency matters. A facility closer to end users reduces response time. For training, latency between GPUs in the same data center is critical, but the facility’s location relative to the rest of the world is less important.
• Dark fiber availability – If the facility is part of a larger network, having access to dark fiber or high-capacity leased lines is essential for moving training data and model checkpoints.

Many AI data centers are built in clusters or campuses to allow high-speed interconnects between buildings. A single large building may not be enough for the largest models.

Physical Site Selection

The physical location of the data center affects cost, reliability, and environmental impact. Important criteria:

• Climate – Cooler climates reduce cooling costs. Locations like the Nordics or high-altitude deserts are popular. However, extreme cold can also cause issues with humidity and condensation.
• Seismic and flood risk – Avoid fault lines and flood plains. Geotechnical surveys are essential.
• Water availability – Liquid cooling requires water for heat rejection. A site with limited water supply may need air-cooled chillers, which are less efficient.
• Energy cost and carbon regulations – Some regions offer renewable energy at low cost, but others have carbon taxes or grid stability issues. The calculator above only computes power, not cost.
• Connectivity – Proximity to internet exchange points and fiber routes reduces latency and improves resilience.

Many AI data centers are built on greenfield sites because existing buildings rarely have the floor loading (weight capacity) or ceiling height for the required cooling infrastructure.

How to Choose a Suitable Facility

When evaluating an existing or new data center for AI workloads, follow this checklist:

1. Calculate your power needs – Use the calculator above to estimate total facility power based on your GPU count and expected density.
2. Verify utility capacity – Confirm the site can receive the required megawatts without excessive upgrade costs or delays.
3. Inspect cooling systems – Check if the facility supports liquid cooling or can be retrofitted. Look for the presence of chilled water loops and space for heat exchangers.
4. Test network performance – Run latency and bandwidth tests between racks and to the outside world. A high-performance fabric is non-negotiable.
5. Review redundancy and maintenance – Ensure UPS, generators, and cooling have N+1 or 2N redundancy, and that maintenance can be done without shutting down the AI cluster.
6. Consider future expansion – AI workloads grow quickly. The site should have space and power capacity to double or triple the initial deployment.

If you are leasing space in a colocation facility, ask for a power density guarantee in the contract. Some providers advertise high density but cannot actually deliver it consistently.

Common Problems and Misconceptions

A common misconception is that a data center with good air cooling can simply add more fans to handle AI workloads. In reality, air cooling becomes physically impractical above about 30 kW per rack because the volume of air required would be enormous and the noise and dust would be problematic. Another issue is underestimating the power draw of the network switches. In a large AI cluster, the network can consume 10–20% of the total IT power, which is often forgotten in initial planning.

Another frequent problem is that the facility’s electrical distribution is designed for lower densities, with small busways or insufficient breaker capacity. Even if the utility feed is large enough, the internal distribution may need a complete overhaul. Also, many facilities have a PUE that is measured at low load, but at high AI density, the PUE can worsen because cooling systems operate less efficiently at extreme loads. Always ask for PUE data at the expected load level.

Costs and Alternatives

Building a new AI data center is capital-intensive, with costs per megawatt often exceeding $10 million (including construction, power infrastructure, and cooling). Retrofitting an existing facility can be cheaper but may be limited by the building’s structure and power path. Alternatives to building your own facility include:

• Colocation – Renting space in a provider’s AI-ready facility. This avoids upfront capital but locks you into a long-term contract.
• Cloud AI services – Using hyperscale cloud providers (AWS, Azure, Google Cloud) that already operate AI-optimized data centers. This is the fastest option but can be more expensive at scale.
• Modular data centers – Prefabricated modules that can be deployed quickly, but they still require the same power and cooling infrastructure.

Each option has trade-offs between control, speed, and cost. The calculator above helps you estimate the power requirement, but you must also factor in the cost of electricity, water, and maintenance, which vary by region.

Frequently Asked Questions

What is the typical power density for AI racks?

Modern AI training racks using GPUs like the H100 typically draw 30–40 kW with air cooling and 60–120 kW with liquid cooling. Inference racks may be lower, around 10–20 kW, but still higher than traditional servers.

Can a traditional data center be upgraded for AI?

Sometimes, but it depends on the building’s structural floor loading, electrical capacity, and cooling system. Upgrading to liquid cooling often requires new piping and heat exchangers, which may not fit in existing spaces. A feasibility study is essential.

How much water does an AI data center use?

Water usage varies widely. A 10 MW facility with evaporative cooling towers can use millions of gallons per year. Direct-to-chip cooling still requires water for the chiller plant, but immersion cooling may use less. Some facilities use closed-loop dry coolers to avoid water consumption entirely.

What is PUE and why does it matter?

PUE is the ratio of total facility energy to IT equipment energy. A PUE of 1.2 is excellent, while 1.5 is poor. For AI workloads, a high PUE means higher operating costs and more environmental impact. Always ask for PUE measured at your expected load, not just at low load.

Is location important for AI inference?

Yes, for inference (running models), latency to end users matters. A facility in a central location reduces response times. For training, location is less critical, but network connectivity to the rest of your infrastructure still matters.

What is the difference between AI training and inference in terms of data center needs?

Training is compute-intensive and runs for days or weeks, requiring maximum power density and cooling. Inference is more latency-sensitive and often runs continuously at lower utilization, so it can use lower-density racks but needs robust network connectivity to serve many users.

Related

• Python AI Agent Tracing: Build a Parent-Child Timeline
• Older Flagship vs Newer Midrange Chip: Which to Choose?
• Word AI Summarize: What It Does and How to Use It

in t | GPU power per rack | kW | 40 | 0 | 200
in n | Number of racks | racks | 100 | 1 | 10000
in p | Power Usage Effectiveness (PUE) | - | 1.3 | 1 | 3
out total | Total facility power | MW | (t * n / 1000) * p | 2
out it | IT load | MW | t * n / 1000 | 2
out cooling | Estimated cooling overhead | MW | (t * n / 1000) * (p - 1) | 2