keepcalculator

06 Numbers and Powers

Sizing Power & Cooling for an AI Training Cluster

Card No. 06·4718 · Updated · Standard formula, shown below

This guide walks you through estimating the power consumption and cooling requirements for an AI training cluster, so you can plan your facility’s electrical and HVAC capacity. You’ll learn how to calculate total IT load, account for cooling overhead, and size the necessary infrastructure. Before starting, you’ll need basic knowledge of your planned GPUs, servers, and facility constraints.

What You’ll Need

  • List of GPU models and their typical power draw (TDP)
  • Server chassis specifications (CPU, memory, storage power)
  • Number of servers and GPUs planned
  • Facility voltage (e.g., 208V, 240V, 480V) and cooling type (air or liquid)
  • Local climate data (for efficiency estimates)
  • Calculator or spreadsheet

Step-by-Step

  1. Calculate GPU power draw: For each GPU model, note its maximum power consumption (TDP) in watts. Multiply by the number of GPUs per server and the number of servers. For example, 8 GPUs at 350W each = 2800W per server.
  2. Add server overhead: Each server also has CPUs, memory, storage, and fans. Estimate 300–500W per server for these components, depending on CPU model and memory count. Add this to the GPU total.
  3. Include network and storage: Add power for top-of-rack switches (typically 200–400W each) and any shared storage systems (estimate 1–2kW per storage node).
  4. Sum the IT load: Add all the above to get the total IT equipment power in watts. This is the load that the cooling system must remove.
  5. Apply utilization factor: Training workloads often run at high utilization, but not always 100%. Use a factor of 0.8–0.9 for realistic peak load. Multiply the total IT load by this factor.
  6. Calculate cooling load: The cooling system must remove the heat generated by the IT load, plus the heat from the cooling equipment itself (fans, pumps, compressors). For air cooling, estimate a cooling overhead of 30–50% of the IT load. For liquid cooling, the overhead is lower, around 10–20%.
  7. Total facility power: Add the IT load and the cooling load to get the total power required from the utility. Also add a safety margin of 10–20% for future expansion or inefficiencies.
  8. Convert to cooling capacity: If you need cooling capacity in tons or kW, use the conversion: 1 ton of cooling = 12,000 BTU/hr = 3.517 kW. Divide the cooling load (in watts) by 3517 to get tons.
  9. Check electrical infrastructure: Verify that your facility’s electrical panels, breakers, and UPS can handle the total load. Consider both power and amperage at your voltage.
  10. Plan for redundancy: For critical training, consider N+1 redundancy for cooling and power. This means adding one extra unit (e.g., one extra CRAC unit) to ensure operation if one fails.

Common Mistakes

  • Using theoretical TDP instead of actual measured power, which can be higher during sustained training.
  • Forgetting to include power for networking and storage, which can add 5–10% to the total.
  • Underestimating cooling overhead, especially in hot climates or with poor airflow.
  • Not accounting for power distribution losses (PDU, UPS inefficiency) – add 5–10%.
  • Ignoring peak vs. average power – training can have bursts, so size for peak.
  • Assuming all servers run at 100% utilization – use a realistic factor.

Tips

  • Use manufacturer power calculators or reference designs for your specific hardware.
  • Consider liquid cooling for high-density GPU clusters – it’s more efficient and reduces cooling load.
  • Monitor actual power draw after deployment to refine your estimates.
  • Plan for future expansion – leave extra capacity in your electrical and cooling systems.
  • Use power usage effectiveness (PUE) to measure efficiency – aim for below 1.3 for modern facilities.

Troubleshooting

If your calculations seem off or you’re hitting limits:

  • Check for hidden loads: Lights, monitors, and other facility equipment add to cooling load – include them.
  • Verify voltage and phase: A 3-phase system delivers more power per amp than single-phase – recalculate if unsure.
  • Measure actual power: Use a power meter on a single server to get real numbers, then extrapolate.
  • Review airflow: Hot spots can cause cooling inefficiency – improve rack layout and blanking panels.
  • Consult a professional: For large clusters, an electrical engineer or data center consultant can validate your numbers.

Frequently Asked Questions

How much power does a single AI server with 8 GPUs draw?

Typically 3–5 kW, depending on GPU model and CPU. For example, 8× 350W GPUs + 500W CPU/memory = 3300W.

What is a good PUE for AI training clusters?

A PUE below 1.3 is considered efficient, meaning only 30% extra power for cooling and overhead. Liquid cooling can achieve closer to 1.1.

Should I use air or liquid cooling for AI training?

Liquid cooling is more efficient and allows higher density, but air cooling is simpler and cheaper for smaller clusters. Choose based on scale and budget.

How do I convert kW to tons of cooling?

Divide the cooling load in kW by 3.517 to get tons. For example, 100 kW of cooling load is about 28.4 tons.

What safety margin should I add to my power estimate?

Add 10–20% for future expansion and unexpected spikes. Also account for power distribution losses.

Can I run an AI cluster on standard 120V outlets?

No – high-power GPUs require 208V or 240V circuits. A single server may draw 20–30A, so you need dedicated circuits.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →