AI Data Center Networking FAQ: Ethernet Speeds & Optics

This article answers the most common questions about AI data center networking, focusing on Ethernet speeds, optical transceivers, and how to plan an upgrade. You’ll learn what speeds are currently used, how optics work, and the key factors to consider when planning a network upgrade for AI workloads.

Background: Why AI Needs Different Networking

Traditional data center traffic is mostly north-south (client to server) with some east-west (server to server). AI training and inference workloads, especially large language models (LLMs), are different. They use distributed training across thousands of GPUs or TPUs that must synchronize frequently. This creates massive, sustained east-west traffic with low latency requirements. A single training job can generate more internal traffic than a traditional web service. This is why AI data centers are built with high-speed, low-latency fabrics, often using Ethernet or InfiniBand, and why understanding Ethernet speeds and optics is critical for planning.

How Ethernet Speeds and Optics Work

Ethernet speeds are defined by IEEE standards. The most common in modern AI data centers are 100G, 400G, and increasingly 800G per port. These speeds can be achieved using different numbers of electrical lanes and optical wavelengths. For example, 400G can be delivered as 8x50G or 4x100G electrical lanes. Optics convert electrical signals to light and back. A 400G optical module can use 4 wavelengths at 100G each (400G-DR4) or 8 wavelengths at 50G each (400G-FR8). The choice depends on distance and fiber type.

Key optical terms include:

• Transceiver: The module that plugs into a switch port and converts electrical to optical signals.
• Form factor: The physical size and connector type, e.g., QSFP-DD (Quad Small Form-factor Pluggable Double Density) for 400G, OSFP (Octal Small Form-factor Pluggable) for 800G.
• Reach: How far the signal travels, e.g., 100m (SR), 500m (DR), 2km (FR), 10km (LR).
• Fiber type: Multimode (MMF) for short distances, single-mode (SMF) for longer distances.

Choosing Ethernet Speeds for AI

Selecting the right speed depends on your workload, budget, and existing infrastructure. For AI training clusters, the current sweet spot is 400G per port for GPU-to-GPU communication, with 800G starting to appear in new builds. For inference, 100G or 200G may suffice. A common approach is to build a leaf-spine topology where each GPU server connects at 400G to a leaf switch, and leaf switches connect to spine switches at 400G or 800G. The goal is to avoid oversubscription, meaning the network has enough bandwidth to handle the peak traffic without dropping packets.

Consider these factors:

• GPU type: Newer GPUs like NVIDIA H100 have 400G network interfaces. Older A100s have 200G.
• Scale: The number of GPUs in a cluster affects the required fabric bandwidth.
• Budget: 800G optics are more expensive per port than 400G.
• Future-proofing: 800G may be worth the cost if you plan to upgrade GPUs soon.

Upgrade Planning: Steps and Considerations

Upgrading a data center network for AI is not just swapping switches. It requires careful planning to avoid downtime and bottlenecks. Here is a typical process:

1. Assess current network: Measure current utilization, identify bottlenecks, and inventory existing fiber and optics.
2. Define requirements: Determine the required bandwidth per GPU, the number of GPUs, and the traffic pattern (e.g., all-to-all vs. pipeline parallel).
3. Design the topology: Choose a leaf-spine or other non-blocking architecture. Decide on oversubscription ratio (1:1 is ideal but costly).
4. Select hardware: Choose switches that support the required port speeds and density. Consider power and cooling.
5. Plan the migration: Stage the upgrade to minimize disruption. Use a test environment first.
6. Test and validate: After deployment, run stress tests to ensure the network performs as expected.

A common mistake is forgetting about fiber types. 400G-DR4 optics require single-mode fiber (OS2), while 400G-SR8 uses multimode (OM4/OM5). If you have existing OM3 fiber, you may need to replace it.

Common Problems and Misconceptions

One major misconception is that higher speed always means lower latency. While higher speeds reduce serialization delay, the physical distance and switch forwarding latency dominate. Another issue is signal integrity at high speeds. At 800G, even small imperfections in fiber connectors can cause bit errors. Common problems include:

• Incorrect fiber type: Using multimode fiber with single-mode optics (or vice versa) causes high loss.
• Dirty connectors: Dust on fiber end-faces is a leading cause of failure. Always inspect and clean.
• Power budget: The optical power at the receiver must be within range. Too much or too little causes errors.
• Switch compatibility: Not all optics work with all switch brands. Check vendor support lists.

Ethernet vs. InfiniBand for AI

InfiniBand has been popular in HPC and AI due to its low latency and high bandwidth, but Ethernet is now a strong competitor. Ethernet advantages include:

• Interoperability: Works with any vendor’s equipment.
• Familiarity: Existing network teams know Ethernet.
• Cost: Often cheaper per port than InfiniBand.
• Ecosystem: Better support for standard TCP/IP and RDMA over Converged Ethernet (RoCE).

InfiniBand still offers lower latency and higher efficiency for tightly coupled workloads, but Ethernet is closing the gap. For many AI data centers, Ethernet is the practical choice due to cost and flexibility.

Cost and Safety Considerations

Cost is a major factor. 800G optics can cost several times more than 400G. However, the total cost of ownership includes power consumption, cooling, and the cost of fiber. Always calculate the cost per bit per second. For safety, always follow the manufacturer’s guidelines for handling optics and fiber. Never look directly into a fiber end-face or laser source, as it can cause eye damage. Use proper laser safety glasses when working with active optics.

Frequently Asked Questions

What is the difference between 400G-DR4 and 400G-FR8?

400G-DR4 uses 4 wavelengths at 100G each over single-mode fiber, reaching up to 500 meters. 400G-FR8 uses 8 wavelengths at 50G each, reaching up to 2 kilometers. DR4 is cheaper and uses less power, but FR8 is better for longer distances.

Can I use existing Cat6 copper cables for AI networking?

No. AI data centers require fiber optics for speeds above 25G. Copper cables like Cat6 are limited to 10G and are not suitable for high-speed AI fabrics.

How do I know if my network is oversubscribed?

Oversubscription means the total bandwidth of downstream ports exceeds the upstream bandwidth. For AI, you want a 1:1 non-blocking fabric. Check your leaf-to-spine bandwidth versus leaf-to-server bandwidth. If the upstream is less, you have oversubscription.

What is RoCE and why is it important for AI?

RoCE (RDMA over Converged Ethernet) allows direct memory-to-memory transfers over Ethernet, reducing CPU overhead and latency. It is essential for AI training because it enables fast GPU-to-GPU communication without the kernel involvement.

How often should I clean fiber connectors?

Inspect and clean connectors every time you plug or unplug a cable. Even a tiny dust particle can cause significant loss at 400G or 800G. Use a one-click cleaner or lint-free wipes.

Is 800G worth the investment now?

If you are building a new AI cluster with the latest GPUs, yes. But if you are upgrading an existing 400G network, consider whether the GPUs can actually use 800G. If not, 400G is still a solid choice.

Related

• Ringconn Gen 3 Sleep Tracking: What Your Data Means
• Evaluate 800VDC Power for AI Data Centers
• Zero TTFB: What It Means for Static Sites

in t | Finish time | min | 50 | 0
in d | Distance | km | 10 | 0 | 100000
out pace | Average pace | min/km | t / d | 2
out speed | Average speed | km/h | d / (t / 60) | 2