FastAPI WebSocket 1006: Why LLM Streams Drop

A WebSocket close code 1006 in a FastAPI app means the connection ended without the client receiving a normal WebSocket close frame; it does not identify the component that failed. During an LLM stream, the drop can originate in the browser, application, model provider, proxy, or network, so trace the connection across each layer rather than treating 1006 as a diagnosis.

What WebSocket 1006 means

Code 1006 is a reserved status used locally to describe an abnormal closure. It is not sent as a close code in a WebSocket close frame. If the browser reports 1006, it generally means that the connection disappeared before a valid close handshake reached it—for example, because a TCP connection was reset or a network intermediary closed it.

This distinction matters when reading logs. A server may record a disconnect while the browser reports 1006, and neither observation proves that FastAPI or the LLM caused the failure. A normal close has a close frame and may carry a status such as 1000; 1006 instead describes the absence of that orderly exchange.

How an LLM stream crosses the stack

A typical streaming request passes through several independent connections. The browser opens a WebSocket to the FastAPI service. The application may then make a separate streaming HTTP request to an LLM provider, read incremental events, and forward text or structured messages over the WebSocket. A reverse proxy, ingress controller, load balancer, firewall, or hosting platform may sit between the browser and FastAPI.

That means there are at least two streams to investigate: client-to-application WebSocket traffic and application-to-provider traffic. If the provider request stalls or raises an exception, the application might stop sending messages, close the socket, or leave it idle until an intermediary times out. If the browser-side connection drops first, the provider request may keep running unless the application cancels it.

Streaming does not guarantee that data is continuously transmitted. A model can take time to produce its first token, tool calls can pause output, and application code can buffer events before forwarding them. An idle timeout may therefore occur even when the overall request is still working.

Trace the failure from both ends

Start by recording a timestamp and a request or connection ID when the socket is accepted. Log connection establishment, each meaningful stream event, disconnect observation, provider completion or error, and cancellation. Avoid logging prompts, credentials, or other sensitive content; event counts, elapsed times, and safe identifiers are often enough.

1. Check the browser. Use the browser’s developer tools to inspect the WebSocket request and its Frames or Messages view. Note the time of the last received event, whether the request was upgraded successfully, and whether the browser saw a close frame or only a network failure.
2. Check FastAPI and ASGI logs. Record when the endpoint accepts the socket and when it detects a disconnect. In Starlette-based WebSocket handling, receive operations can raise a disconnect exception; code should handle it and stop work tied to that client.
3. Check the provider stream. Log when the upstream request starts, when its first event arrives, and whether it ends normally, errors, or is cancelled. If upstream activity continues after the browser is gone, the application may not be propagating cancellation.
4. Inspect intermediaries. Compare application timestamps with proxy or load-balancer access and error logs. Look for idle timeout, upstream reset, connection reset, or worker restart evidence. Confirm that the deployed path supports WebSocket upgrades and is not applying an unexpectedly short timeout.
5. Reproduce with a minimal stream. Send a small, periodic test message through the same route and infrastructure. If that survives while the LLM stream drops, investigate upstream waits, buffering, and application flow control; if both drop at a similar idle interval, suspect transport or intermediary behavior.

Common causes and practical fixes

• Idle timeouts: A proxy or hosting layer may close connections that send no data for a configured period. Check the actual deployed timeout settings. If appropriate, send application-level heartbeat messages more frequently than the relevant idle timeout, and ensure the browser understands them.
• Blocked or buffered forwarding: Code that waits for the entire provider response before sending anything defeats streaming and can leave the WebSocket idle. Forward each event as it arrives, and check whether middleware or a proxy buffers traffic.
• Uncaught task errors: An exception in the provider-reading task can silently stop output if it is not logged and handled. Log the exception and, while the client is still connected, send a clear application-level error message before closing cleanly where possible.
• Missing cancellation: When a client disconnects, cancel or close the upstream stream and associated tasks. Otherwise, work and provider connections may continue after the result can no longer be delivered.
• Backpressure and slow consumers: A client that reads slowly can cause queued messages and memory growth. Use bounded queues or another deliberate flow-control strategy, and decide what to do when the consumer cannot keep up.
• Deployment changes: Worker restarts, process crashes, network changes, or a proxy configuration difference can terminate sockets abruptly. Compare failures by instance, route, duration, and deployment event rather than assuming the model is responsible.

Heartbeats, reconnects, and delivery guarantees

WebSocket ping/pong control frames can help detect an unresponsive peer, but support and configuration depend on the ASGI server and infrastructure. Application-level heartbeat messages are another option when intermediaries need to see ordinary traffic. Neither method guarantees that a connection will remain alive: a network can still fail, and heartbeats should not be treated as proof that the LLM is making progress.

Clients should be prepared to reconnect, but reconnecting does not automatically resume a generation. If resumability is needed, the application must design it: assign a stream ID, retain or reproduce events safely, track sequence numbers, and let a reconnecting client request events after its last confirmed position. Otherwise, present reconnection as a new request and avoid implying that no output was lost.

Also distinguish transport delivery from application completion. A WebSocket may carry text successfully and still disconnect before a final “done” event arrives. A robust protocol should make completion explicit and allow clients to recognize an interrupted stream.

Frequently Asked Questions

Does WebSocket code 1006 mean FastAPI crashed?
No. It means the connection ended without a close frame reaching the observer. A process crash is one possible cause, but so are a proxy timeout, network interruption, client closure, or upstream-related application behavior.

Can I send close code 1006 from FastAPI?
No. It is reserved to represent abnormal closure and is not a valid close status to transmit in a WebSocket close frame. Log the disconnect and use an appropriate valid close code when your application can perform a normal close.

Why does the stream stop after a consistent amount of silence?
A repeatable idle duration points toward a timeout in a proxy, load balancer, server, or hosting platform. Compare the observed interval with configured timeouts and verify whether messages are actually passing through each layer.

Will ping/pong fix every LLM stream drop?
No. Heartbeats can help with idle detection, but they cannot repair a process crash, provider failure, broken network, or incorrect cancellation logic. Diagnose which connection and layer failed first.

Should the LLM request be cancelled when the browser disconnects?
Usually, if the result is no longer needed. Handle the disconnect, cancel the task that reads the provider stream, and close upstream resources; confirm cancellation behavior for the specific client library you use.

Can a browser reconnect and continue the same generation?
Not by default. Resumption requires application-level state and a protocol for identifying the generation and replaying or continuing events. Without that design, a reconnect is a new connection, not a continuation.

A useful diagnosis

Treat 1006 as a symptom at the point where it was observed, not as the root cause. Correlate browser, FastAPI, provider, and intermediary events by timestamp and connection ID; identify the last layer that demonstrably received data; then test the likely timeout, exception, buffering, or cancellation path. That approach separates a true LLM stall from a WebSocket transport drop and points to a fix that addresses the failing layer.

Related

• Measure Whether AI Collaboration Tools Improve Your Team
• OS X Yosemite Zoom Button: What It Does
• NotebookLM Export Limits: Should You Switch?