Estimate AI Data Center Costs: Electricity, GPUs, Cooling

This guide walks you through estimating the major costs of an AI data center: electricity, GPU hardware, and cooling. You’ll learn what data to gather, how to calculate each component, and how to combine them into a realistic monthly or annual budget. Before you start, have your GPU specifications, expected utilization, local electricity rate, and facility dimensions ready.

What You’ll Need

• GPU model specs: power draw (TDP), number of GPUs planned, and typical utilization rate (often 60-90% for AI workloads).
• Electricity rate from your utility (cost per kWh, e.g., $0.12/kWh).
• Cooling system type: air, direct-to-chip liquid, or immersion cooling, and its efficiency (PUE – Power Usage Effectiveness).
• Facility details: total floor area, average temperature, and any existing cooling capacity.
• Optional: server count and other IT equipment power draw.

Step-by-Step

1. Determine GPU power draw and count. Look up the TDP (thermal design power) of your GPU model. For example, an NVIDIA H100 has a TDP around 700 W. Multiply TDP by the number of GPUs to get total GPU power draw in watts. For 100 GPUs: 100 × 700 W = 70,000 W = 70 kW.
2. Account for utilization. AI workloads rarely run at 100% power 24/7. Estimate the average utilization, say 70%. Multiply the total GPU power by the utilization factor. 70 kW × 0.7 = 49 kW average GPU power draw.
3. Add other IT equipment power. Servers also include CPUs, memory, storage, and networking. A common rule of thumb is to add 20-30% on top of GPU power. Using 25%: 49 kW × 1.25 = 61.25 kW total IT load.
4. Calculate electricity cost for IT equipment. Convert kW to annual energy: 61.25 kW × 24 hours × 365 days = 536,550 kWh per year. Multiply by your electricity rate. At $0.12/kWh: 536,550 × $0.12 = $64,386 per year.
5. Factor in cooling and facility overhead. Use PUE (Power Usage Effectiveness) to account for cooling, power distribution, and other losses. A typical air-cooled facility has PUE of 1.3-1.5; liquid cooling can achieve 1.1-1.2. Multiply the IT energy by PUE to get total facility energy. For PUE 1.3: 536,550 kWh × 1.3 = 697,515 kWh per year. Then multiply by electricity rate: 697,515 × $0.12 = $83,702 per year.
6. Estimate cooling system capital cost. Cooling hardware cost depends on the system type and capacity. For air cooling, budget roughly $100-$200 per kW of cooling capacity. For liquid cooling, $200-$400 per kW. If your total IT load is 61.25 kW, air cooling might cost $6,000-$12,000; liquid cooling $12,000-$24,000. (These are rough estimates; get quotes from vendors.)
7. Estimate GPU acquisition cost. GPU prices vary widely. Use current market prices for your chosen model. For example, if an H100 costs $30,000, then 100 GPUs cost $3,000,000. Include server chassis, networking, and installation costs, often 10-20% extra.
8. Sum all costs. Add annual electricity (from step 5), annualized cooling capital (divide by expected lifespan, say 5 years), and annualized GPU cost (also over 5 years). For a rough monthly cost, divide annual totals by 12.

Common Mistakes

• Using TDP as actual draw. TDP is the maximum under load, not average. Always apply a utilization factor.
• Ignoring PUE. Forgetting cooling overhead can underestimate electricity cost by 30-50%.
• Mixing up kW and kWh. kW is power, kWh is energy. Always multiply by hours to get energy.
• Overlooking idle power. Even when idle, GPUs draw power (often 10-20% of TDP). Include idle time in your utilization estimate.
• Forgetting other IT equipment. CPUs, memory, and networking add significant load.

Tips

• Get real utilization data. If you have existing workloads, measure actual power draw over a week to refine your estimate.
• Consider time-of-use rates. Some utilities charge more during peak hours. If you can shift workloads to off-peak, you may save money.
• Explore liquid cooling. Though higher upfront, liquid cooling can lower PUE and reduce electricity bills, especially in warm climates.
• Use cloud pricing as a benchmark. Compare your on-prem estimate to cloud GPU rental costs to see if building is worth it.
• Plan for redundancy. Add 10-20% extra capacity for cooling and power to handle failures and maintenance.

Troubleshooting

If your estimate seems too high or low, check the following:

• Electricity cost seems too high? Verify your PUE assumption. If you have a modern cooling system, PUE might be lower. Also check if you applied the utilization factor correctly.
• GPU cost seems too low? Confirm you included all necessary components: server boards, memory, storage, networking, and installation labor. Add a 10-15% contingency.
• Cooling cost confusing? For a rough check, cooling typically accounts for 30-40% of total facility electricity. If your cooling share is much higher, your PUE may be too high.
• Utilization unknown? Start with 70% as a baseline. For training workloads, it can be higher; for inference, lower. Adjust once you have real data.

Frequently Asked Questions

Q: What is PUE and why does it matter?
PUE (Power Usage Effectiveness) is the ratio of total facility energy to IT equipment energy. A PUE of 1.0 means all power goes to IT; 1.5 means 50% extra for cooling and losses. Lower PUE means lower electricity cost.

Q: How do I know my GPU’s actual power draw?
Use the TDP as a starting point, but measure with tools like nvidia-smi on Linux or GPU-Z on Windows. Average the power draw over a representative workload.

Q: Can I use this for cloud AI costs?
No, this guide is for on-premises data centers. Cloud costs are based on per-hour GPU rental prices, which you can estimate separately.

Q: What is a good PUE for an AI data center?
Modern facilities with liquid cooling can achieve PUE of 1.1-1.2. Air-cooled facilities often run 1.3-1.5. Older or poorly designed ones may be higher.

Q: How do I account for cooling water costs?
If you use water-based cooling, add water consumption costs. Estimate water usage based on cooling load (typically 0.5-1 gallon per kWh of heat removed) and your local water rate.

Q: What about maintenance and staff costs?
These are significant but vary widely. A common rule is to budget 10-20% of total hardware cost per year for maintenance and operations, including staff salaries.

Related

• AI Sea Ice Forecasts: Evidence & Limits
• Gigabyte X870 Aorus DDR5 Specs: What They Promise
• Why Can’t I Open Word Document? Fixes & Causes