MCP Connection Latency: Why P90 Session Wait Hit 35.5s

This article explains why a developer’s P90 session wait time for MCP (Model Context Protocol) connections reached 35.5 seconds, and what that number actually means. The short answer: a 35.5s P90 indicates that 90% of session initialization requests completed within 35.5 seconds, but the remaining 10% took even longer—likely due to a combination of network round-trips, authentication handshakes, and server-side resource contention, not a single bottleneck.

Formula

The calculator above computes the average wait time per request and a naive P90 estimate if all requests had identical latency. The real P90 is the 90th percentile of a sorted list of individual request latencies. To calculate it manually: collect all session wait times, sort them ascending, and pick the value at index ceil(0.9 × N) where N is the total number of requests. For example, with 100 requests, the 90th value in sorted order is the P90.

Worked Example

Using the defaults: total wait time = 35.5 seconds, number of requests = 100. Average wait per request = 35.5 / 100 = 0.355 seconds. The naive P90 = 35.5 × 0.9 = 31.95 seconds. However, this only holds if every request took the same time. In reality, if you have 100 requests and the 90th slowest took 35.5 seconds, that means 10 requests were slower than 35.5 seconds—potentially much slower.

Background: What Is MCP and Why Latency Matters

Model Context Protocol (MCP) is an open standard that lets AI assistants like Claude or custom LLM applications connect to external tools and data sources. A “session” is established when the client and server exchange initialization messages, authentication tokens, and capability declarations. The time from the client’s first connection attempt to the moment the session is ready is the session wait time. High P90 values indicate that a significant minority of users experience noticeably slower startup, which can degrade the perceived responsiveness of AI-powered features.

How MCP Session Establishment Works

MCP sessions typically follow a sequence:

• Client sends a initialize request with protocol version and capabilities.
• Server responds with its own capabilities and protocol version.
• Client sends initialized notification.
• Optional: OAuth or API key exchange happens before or during this handshake.
• Server may load resources, connect to databases, or spawn subprocesses.

Each step adds latency. If any step involves a network call to a remote service (e.g., a database or a third-party API), that latency compounds. A single slow dependency can push the P90 up dramatically.

Why the P90 Reached 35.5 Seconds: Common Causes

Several factors can contribute to a 35.5s P90 session wait:

• Network round-trips: If the MCP server is far from the client or behind a proxy, each handshake message incurs RTT. With 10 round-trips at 500ms each, that’s 5 seconds alone.
• Authentication overhead: Token validation against an external identity provider (like OAuth) can take seconds if the IdP is slow or rate-limited.
• Server-side resource contention: If the server is busy handling other sessions, initialization requests queue up. A P90 of 35.5s suggests that during peak load, some requests waited a long time in a queue.
• Cold starts: Serverless or containerized deployments may need to spin up new instances, adding 10-30 seconds of cold start time.
• Inefficient code: Synchronous calls to external APIs during initialization, or heavy computation like loading large models, can block the session setup.

How to Diagnose and Reduce High P90 Latency

To pinpoint the cause, measure each step of the handshake separately. Use distributed tracing to see where time is spent. Common fixes:

• Cache authentication tokens to avoid repeated IdP calls.
• Move heavy initialization (e.g., loading reference data) to a background task that doesn’t block session readiness.
• Use connection pooling for databases and external APIs.
• Pre-warm serverless instances or use a minimum instance count.
• Consider protocol-level optimizations like HTTP/2 multiplexing to reduce round-trips.

Comparing P90 vs. Average vs. P99

P90 is a good indicator of the worst-case experience for the majority, but P99 (the 99th percentile) reveals the true tail. If P90 is 35.5s and P99 is 60s, that means 1 in 100 users wait a full minute. Average latency is misleading because a few very slow requests skew it upward. For session wait times, monitoring both P90 and P99 is recommended, because users who experience the tail are more likely to abandon the feature.

Common Misconceptions About MCP Latency

One misconception is that MCP latency is solely a network issue. In reality, server-side processing often dominates. Another is that increasing client-side timeouts solves the problem—it only hides it. Also, some assume that P90 = 90% of requests take that long; actually, 90% of requests are faster than that value. A 35.5s P90 means 90% of requests completed in under 35.5 seconds, not that 90% took exactly that long.

Frequently Asked Questions

What does P90 session wait mean in MCP?

P90 session wait is the 90th percentile of the time taken to establish an MCP session. If you collect the session wait times for 100 requests, sort them, the 90th value is the P90. It tells you that 90% of your users experienced a wait time at or below that number.

Is 35.5 seconds a bad P90 for MCP?

Yes, for most interactive applications. A session wait of 35.5 seconds means that 10% of users wait even longer, which is far beyond the 2-3 second threshold for perceived responsiveness. It suggests a serious bottleneck that needs investigation.

Why is my P90 latency so much higher than my average?

Because a small number of requests take disproportionately long (e.g., due to cold starts or queueing). The average is pulled down by the many fast requests, while the P90 is set by the slowest 10%. This is normal for skewed distributions, but a large gap indicates a tail-latency problem.

Can I reduce MCP session wait by using a faster network?

Possibly, but only if the bottleneck is network round-trips. If the delay is server-side (CPU, database, or external API calls), a faster network won’t help. Measure first, then optimize the actual bottleneck.

Should I use P90 or P99 to monitor MCP latency?

Both are useful. P90 gives a general sense of the experience for the majority, while P99 reveals the worst-case tail. If you can only monitor one, start with P99 because it catches rare but severe issues that P90 might miss.

What is a good P90 target for MCP session wait?

For most interactive AI applications, aim for a P90 under 2 seconds. If your use case is non-interactive (e.g., batch processing), a few seconds may be acceptable, but 35.5 seconds is almost always too high.

Related

• Robot Submarines and Sonar: Public Evidence
• Complacent Photographer Signs & How to Overcome
• How Astronomers Use Webb & Hubble for Star Formation

in t | Total wait time | s | 35.5 | 0 | 3600
in n | Number of requests | - | 100 | 1 | 100000
out avg | Average wait per request | s | t / n | 2
out p90 | P90 wait (if all equal) | s | t * 0.9 | 2