AI Data Centers: Cut Electricity Use Without Downtime

This guide explains how to reduce electricity consumption in AI data centers without interrupting running workloads. You’ll learn practical techniques that lower power draw while keeping training and inference jobs stable, from hardware tuning to cooling and scheduling adjustments.

What You’ll Need

• Access to the data center’s monitoring system (power usage effectiveness, server utilization, temperature sensors).
• Ability to modify server BIOS/UEFI settings (or a management interface like iDRAC, iLO, or IPMI).
• Knowledge of the workload scheduler (e.g., Slurm, Kubernetes) to adjust job placement and timing.
• Permission to change cooling setpoints (if you have control over HVAC or liquid cooling).
• Basic understanding of power capping and frequency scaling (Intel Speed Shift, AMD CPPC).

Step-by-Step

1. Measure baseline power and utilization. Before making changes, record current power draw per server and overall facility power. Identify idle or underutilized servers (e.g., CPU utilization below 30%) that are still drawing significant power.
2. Enable power capping at the server level. Set a power cap slightly above the average power draw of your typical workload (e.g., 80% of max TDP). Use tools like `ipmitool` or vendor-specific interfaces to set the cap. This prevents power spikes without killing jobs.
3. Adjust CPU frequency scaling governor. Set the governor to “powersave” or “ondemand” (instead of “performance”) for non-latency-critical tasks. For training jobs that are not bottlenecked by CPU (e.g., GPU-heavy), this can save watts with negligible impact. Test with a small job first.
4. Use dynamic voltage and frequency scaling (DVFS) for GPUs. Many AI workloads don’t need maximum GPU clock. Use `nvidia-smi -lgc` to lock GPU clocks to a lower frequency, or use tools like `nvml` to set power limits. For example, setting a 200W limit on a 300W GPU can reduce power by 20% with minimal performance loss on memory-bound operations.
5. Consolidate workloads onto fewer servers. If you have multiple underutilized servers, move jobs to a subset of nodes and power off the rest (or put them in deep sleep). Use your scheduler’s backfill and bin-packing features to maximize utilization on active nodes.
6. Optimize cooling setpoints. Raise the inlet air temperature to the ASHRAE recommended range (up to 27°C / 80°F) if your hardware allows. For every 1°C increase, cooling energy can drop by 2-5%. Monitor GPU/CPU temperatures to ensure they stay below throttling limits (usually 85-90°C).
7. Implement workload scheduling to flatten power peaks. Stagger job starts to avoid simultaneous power spikes. Use your scheduler to delay non-urgent jobs by a few minutes or hours. This reduces the peak demand and allows you to use a smaller power supply or avoid demand charges.
8. Use predictive power management. If you have a workload that varies (e.g., inference with traffic spikes), use machine learning to predict demand and scale up/down resources accordingly. Even simple rules (e.g., scale down after 6 PM) can help.
9. Monitor and adjust iteratively. After each change, observe power draw and workload performance (throughput, latency). If a job fails or slows down significantly, revert the change. Keep a log of what works.
10. Consider liquid cooling for high-density areas. If you have a hot aisle or GPU cluster, liquid cooling can reduce cooling power by up to 40% compared to air. This is a larger investment but doesn’t interrupt workloads during installation if done in phases.

Common Mistakes

• Setting power caps too low. If the cap is below the workload’s minimum requirement, jobs will fail or hang. Always test with a small job first.
• Ignoring memory and network power. Power capping only the CPU may not save much if memory and network cards are running at full speed. Use memory power management (e.g., DDR4/5 low-power modes) and disable unused network ports.
• Forgetting to update cooling setpoints after raising server inlet temps. If you raise the setpoint but don’t adjust the cooling system, you may still be overcooling.
• Applying changes to all servers at once. A single bad setting can cause a cascade of failures. Roll out changes gradually.
• Not accounting for power spikes during job startup. Many AI jobs have a high initialization phase (loading data, compiling). A power cap that’s fine for steady-state may cause startup failures.

Tips

• Use the “power” performance state (P-state) that matches your workload. For example, if your workload is memory-bound, a lower CPU frequency may not hurt performance.
• For GPU workloads, use the GPU’s power management API to set a power limit that is 80-90% of the default; this often yields near-peak performance with significant savings.
• Use energy-efficient network switches and turn off unused ports.
• Implement “power-aware” scheduling: your scheduler can be configured to prefer nodes with lower current power draw.
• Regularly review your Power Usage Effectiveness (PUE) and aim to reduce it by improving cooling and power distribution efficiency.

Troubleshooting

• Workloads are slower after power capping. Increase the power cap gradually until performance is acceptable. Check if the bottleneck is CPU, GPU, or memory.
• Servers are overheating after raising inlet temps. Check for hot spots using sensor data. You may need to adjust airflow or add baffles. If temperatures exceed 90°C, lower the setpoint.
• Jobs are being killed due to power limits. Look at the power draw during the job’s initialization phase. Set a temporary higher cap for the first few minutes, or use a scheduler that allows per-job power limits.
• Power savings are minimal. Ensure that power capping is actually engaging (check with `ipmitool sensor`). Also check if your servers are using “turbo” modes that override caps.
• Cooling system is not responding to setpoint changes. Verify that the cooling control system is set to a “setpoint” mode, not a fixed manual mode. Some systems require a delay to respond.

Frequently Asked Questions

Will power capping affect my AI model training time?

It can, but often less than you’d expect. If the workload is memory-bound or I/O-bound, a 10-20% power reduction may cause only a 2-5% slowdown. Test with your specific workload.

Is it safe to raise data center temperatures to 27°C?

Yes, for most modern hardware. ASHRAE recommends up to 27°C for air-cooled equipment. Just monitor component temperatures and ensure they stay below the manufacturer’s maximum (usually 85-95°C).

Can I use renewable energy to reduce electricity use?

Renewable energy reduces carbon footprint, but not electricity consumption. However, pairing with battery storage can shift loads to off-peak times, reducing demand charges.

Does turning off idle servers really save money?

Yes. An idle server can draw 30-50% of its peak power. Powering it off can save that energy, but only if you consolidate workloads effectively. Consider the cost of restarting vs. the savings.

What is the best way to measure power savings?

Use a power meter at the rack or PDU level, and compare before/after readings over a week. Also track your PUE and total facility energy consumption.

Can I use software tools to automate power management?

Yes, tools like Kubernetes with power-aware scheduling, or vendor-specific software (e.g., HPE OneView, Dell OpenManage) can automate capping and scaling. Start with manual controls to understand your environment.

Related

• Best Tech Websites to Follow for Pros & Beginners
• Second Wind Quest Walkthrough: Where Winds Meet
• Enshrouded Weapon Upgrade Guide: How to Improve Gear