AI data center interconnects are chosen primarily by the required reach (distance between switches or clusters) and the integration point (where the optics plug into the system: pluggable module, co-packaged optics, or mid-board optics). Reach dictates the optical modulation scheme and fiber type, while the integration point determines power, thermal, and serviceability trade-offs. This article explains both decision axes and how they interact.
Formula
The calculator above estimates the relative power penalty of different modulation schemes over a given reach. The formula is:
Power Penalty (dB) = 10 × log₁₀(Modulation Order)
Where:
- Modulation Order = 1 for NRZ (on-off keying), 2 for PAM4 (4-level pulse amplitude modulation), 3 for coherent (as a relative complexity factor, not a physical constant).
- Reach (m) = the distance between the optical transceiver and the switch ASIC, or between two switches, in meters.
The power penalty is a simplified way to compare the optical signal-to-noise ratio (OSNR) requirements of each modulation format. Higher-order modulation requires more OSNR for the same bit error rate, which translates to a higher power penalty. The reach input is used only to flag whether the distance is within typical 500m limit for multimode fiber (MMF) — it does not affect the modulation penalty directly.
Worked Example
Using the default values: Reach = 100 m, Modulation = PAM4 (2).
- Identify the modulation order: PAM4 = 2.
- Compute the power penalty: 10 × log₁₀(2) = 10 × 0.3010 = 3.01 dB.
- Check reach: 100 m is less than 500 m, so the output shows “0” (meaning within limit).
The result is a power penalty of 3.01 dB and a reach flag of 0 (OK).
Background: Why Reach and Integration Point Matter
AI clusters are built from thousands of GPUs or accelerators that need to exchange data at very high bandwidth — typically 400G or 800G per link today, with 1.6T in development. The physical layer that carries this data is the optical interconnect. Two fundamental choices define that layer:
- Reach: the distance the optical signal must travel. This can be as short as a few meters within a rack (spine-to-leaf) or as long as hundreds of meters across a data center hall (leaf-to-spine) or even kilometers between buildings (DCI).
- Integration point: where the optical engine (laser, modulator, photodetector) is placed relative to the switch ASIC. The three main options are pluggable modules (e.g., QSFP-DD, OSFP), co-packaged optics (CPO, where the optics are on the same package as the ASIC), and mid-board optics (MBO, where the optics are on the same board but not on the ASIC package).
These two factors are intertwined. Longer reach generally requires more advanced modulation (PAM4 or coherent) and more powerful optics, which consume more power and generate more heat. That heat and power budget, in turn, limits how closely you can integrate the optics to the ASIC.
How Reach Determines the Optical Scheme
Reach is the first filter in the selection process. The industry has standardized on a few reach categories:
- ≤ 2 m (chip-to-chip, on-board): This is the domain of electrical interconnects (copper) or very short reach optics like VCSELs (vertical-cavity surface-emitting lasers) over multimode fiber. Here, integration point is almost always on-board (MBO or CPO) because pluggables add too much latency and power.
- 2 – 100 m (within a rack): Multimode fiber with VCSELs and NRZ or PAM4 modulation. Pluggable modules are common, but CPO is also viable because the distances are short enough that the optical link budget is not strained.
- 100 – 500 m (across a row or hall): Single-mode fiber (SMF) with PAM4 or coherent-lite (e.g., 400ZR). Pluggable modules dominate here because they allow easy upgrades and servicing. CPO becomes harder because the required optical power and thermal management are more demanding.
- 500 m – 2 km (data center interconnect, DCI): Coherent optics (e.g., 400ZR, 800ZR) are standard. These are almost exclusively pluggable modules (QSFP-DD or OSFP) because they need sophisticated DSPs and tunable lasers that are too large and power-hungry to co-package with a switch ASIC.
This is why the calculator’s reach flag uses 500 m as a threshold: it separates the MMF-friendly zone from the SMF/coherent zone.
Integration Point: Pluggable, Co-Packaged, or Mid-Board
The integration point is a physical and thermal engineering decision. Here are the three options:
- Pluggable modules (front-panel or near-ASIC): These are the classic QSFP-DD, OSFP, or CFP modules that plug into a cage on the switch faceplate. They are hot-swappable, which means you can replace a failed module without taking the switch offline. They also allow the switch to support multiple generations of optics (e.g., 400G today, 800G tomorrow) by just swapping the module. The downside is that they sit far from the ASIC, requiring high-speed electrical traces (often with retimers) that consume power and add latency.
- Co-packaged optics (CPO): The optical engines are placed on the same substrate as the switch ASIC, usually within the same package. This eliminates the long electrical traces and reduces power and latency significantly. However, it is not serviceable — if an optical engine fails, the entire switch package is replaced. It also requires advanced thermal management because the ASIC and optics share the same heat sink.
- Mid-board optics (MBO): A middle ground where the optical engines are on the same printed circuit board (PCB) as the ASIC but not on the same package. This allows some serviceability (the optical module can be replaced) while still keeping the electrical traces short. It is often used in leaf switches where the reach is moderate and the power budget is tight.
The choice between these is not just about reach; it is also about the data rate per lane, the switch ASIC’s power envelope, and the operator’s maintenance philosophy. For example, a hyperscaler that runs a massive AI cluster may prefer CPO to save power, even though it means the switch is a single point of failure. An enterprise with a smaller cluster might prefer pluggables for flexibility.
How to Choose: A Practical Decision Flow
When selecting an interconnect for a specific AI deployment, follow this sequence:
- Define the reach: Measure the maximum distance between the two endpoints (switch-to-switch, or switch-to-GPU). Include patch panels and slack in the cable path.
- Determine the data rate per lane: Higher rates (e.g., 100G per lane vs. 50G) require more advanced modulation (PAM4 vs. NRZ) and tighter link budgets.
- Check the power and thermal budget: The switch ASIC and the optics must fit within the chassis’ power envelope. CPO saves about 30-50% of the optical power compared to pluggables, but adds thermal complexity.
- Consider serviceability: If the system must be up 24/7, pluggables are easier to maintain. If the system is part of a large, redundant cluster, CPO’s lower power might be worth the risk.
- Evaluate the fiber type: For reach ≤ 100 m, MMF with VCSELs is cheapest. For longer reach, SMF with edge-emitting lasers (DFB or coherent) is required.
In practice, most AI clusters today use pluggable modules for the spine and leaf layers, with reach between 100 m and 2 km. CPO is emerging for the top-of-rack (ToR) switch to GPU connections, where reach is under 10 m and power is critical.
Common Problems and Misconceptions
One common misconception is that reach is the only factor that determines the integration point. In reality, the integration point also depends on the switch ASIC’s power density and the cooling solution. For example, a 51.2T switch ASIC that dissipates 500 W cannot easily co-package optics that add another 200 W, so pluggables are the only viable option.
Another issue is that reach is often quoted as a maximum, but real-world links have penalties from connectors, splices, and aging. A 2 km link might work at 400G with coherent, but if you add 10 connectors, the loss might exceed the budget. Always add a 2-3 dB margin.
Also, many people confuse the integration point with the form factor. A QSFP-DD pluggable can be used for both short-reach (100 m) and long-reach (2 km) by using different optics (SR4 vs. DR4 vs. LR4). The integration point is about where the module sits, not the connector type.
Cost and Power Trade-offs
Cost is often the deciding factor. Pluggable modules are commoditized and benefit from economies of scale; a 400G QSFP-DD DR4 (500 m reach) costs significantly less than a 400G ZR (2 km reach). CPO has a higher upfront engineering cost but lower per-port power, which can save millions of dollars in a large cluster over its lifetime.
Power is the other major variable. A pluggable 800G module can consume 15-20 W, while a CPO engine might consume 5-10 W per port. Over a 10,000-port cluster, that difference is 100 kW — enough to require additional cooling infrastructure. This is why hyperscalers are pushing for CPO despite the serviceability challenges.
The calculator above gives a rough power penalty, but the real power comparison requires looking at the entire link, including the electrical drivers, retimers, and DSPs. For a full cost analysis, you need to know the specific transceiver prices, which vary by vendor and volume.
Alternatives and Future Trends
Beyond the three integration points, there are emerging alternatives:
- Linear-drive pluggables (LPO): These remove the DSP from the pluggable module, using a linear amplifier instead. This reduces power and latency but requires the switch ASIC to have a better electrical interface. LPO is seen as a bridge between pluggables and CPO.
- Optical circuit switching (OCS): Instead of electrical packet switching, OCS uses MEMS mirrors to route optical signals directly. This is used in some AI clusters for dynamic topology reconfiguration, but it is not a replacement for the interconnect itself.
- Silicon photonics: This is a manufacturing technology, not an integration point, but it enables CPO and MBO by integrating lasers and modulators on a silicon chip. Most CPO products use silicon photonics.
The trend is clear: as data rates increase to 1.6T and beyond, the power and latency of pluggables become untenable, so the industry is moving toward CPO and LPO. However, the transition is slow because of the installed base of pluggables and the need for standards.
Frequently Asked Questions
What is the difference between reach and integration point?
Reach is the distance the optical signal travels (e.g., 100 m, 2 km). Integration point is where the optical engine is physically located relative to the switch ASIC (pluggable, co-packaged, or mid-board). They are related because longer reach often requires more complex optics that are harder to integrate closely.
Can I use a pluggable module for a 2 km reach?
Yes, but you need a coherent pluggable like 400ZR or 800ZR, which is larger and consumes more power than a short-reach module. For 2 km, you would typically use single-mode fiber and a coherent or long-reach PAM4 module.
Why is co-packaged optics not the default choice?
Co-packaged optics are not serviceable — if an optical engine fails, the entire switch must be replaced. They also require advanced thermal management and are more expensive to develop. Pluggables are easier to upgrade and maintain, which is why they dominate.
What does the power penalty in the calculator mean?
The power penalty is a relative measure of how much more optical signal power is needed for a given modulation scheme compared to NRZ. For example, PAM4 needs about 3 dB more power than NRZ for the same bit error rate, which means you need a higher-quality laser or a shorter reach.
Do I need to worry about reach if I use only copper cables?
Copper (DAC) cables are limited to about 3-5 meters and are used only within a rack. For any distance beyond that, you need optical fiber. The reach in the calculator is for optical links.
How does the integration point affect the switch’s power budget?
Pluggable modules consume power for the electrical drivers and retimers that bridge the distance to the ASIC. CPO eliminates that, saving 30-50% of the optical power, but adds heat to the ASIC package. The total chassis power is often lower with CPO, but the thermal density is higher.
Related
Formula used
in reach | Reach (m) | m | 100 | 1 | 2000 in mod | Modulation | NRZ=1, PAM4=2, Coherent=3 out power_penalty | Relative power penalty (dB) | dB | 10 * log10(mod) | 2 out reach_ok | Reach within 500m? | - | 0 + 0 * sqrt(500 - reach) | 0
Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.
More cards in this drawer
- AI for Playwright Locators: Draft, Verify, and Refine
- How to Fact-Check Artificial Intelligence Claims
- Sony Bravia 3 II vs Hisense U6SF Pro: Which to Buy?
- Best Fire Emblem Games Without Monastery Grind
- Are AI Agents Safe for Private Information?
- Best MMORPG Games: Top Picks for Every Play Style