Stop Claude SSE Buffering in API Gateway

To stop Claude’s Server-Sent Events (SSE) responses from buffering in API Gateway, you must disable response buffering for the route and set the correct `Content-Type` header (`text/event-stream`) on the integration response. This article explains exactly which settings to change in the AWS console or CloudFormation, why buffering happens, and how to verify the fix.

Why API Gateway Buffers SSE Responses

API Gateway (both REST and HTTP APIs) is designed to buffer the entire response from a backend before forwarding it to the client. This is fine for JSON or XML payloads, but it breaks SSE, which relies on a continuous stream of events sent over a long-lived HTTP connection. When buffering is enabled, the client receives nothing until the stream ends, which defeats the purpose of real-time updates. The root cause is that API Gateway’s default behavior is to wait for the backend to finish writing the response body before sending it downstream.

How to Disable Buffering for HTTP APIs

For HTTP APIs (the newer, cheaper option), the fix is straightforward. You need to disable buffering on the route and ensure the integration response returns `Content-Type: text/event-stream`.

1. Open the API Gateway console and select your HTTP API.
2. Go to Routes and select the route that handles the SSE endpoint (e.g., GET /stream).
3. Under Integration, click Edit.
4. Scroll down to Advanced settings and check the box Disable response buffering.
5. Save the changes.

That’s it for HTTP APIs. The setting tells API Gateway to pass the response through as it arrives, without buffering. You must also make sure your backend (e.g., a Lambda function or a custom service) sets the `Content-Type` header to `text/event-stream` in the response. If you’re using Lambda, you’ll need to return a streaming response (see below).

How to Disable Buffering for REST APIs

REST APIs are older but still common. The process is slightly more complex because you must modify the integration response.

1. In the API Gateway console, select your REST API.
2. Go to Resources and select the method (e.g., GET) for your SSE endpoint.
3. Click Integration Response.
4. Expand the response (usually status 200) and look for HTTP Headers.
5. Add a header mapping: Content-Type mapped to 'text/event-stream' (note the single quotes to force a literal value).
6. Also, under Integration Response, ensure that Content Handling is set to Passthrough (not “Convert to binary” or “Convert to text”).
7. Deploy your API to a stage.

There is no direct “disable buffering” checkbox for REST APIs. The trick is to set the `Content-Type` header to `text/event-stream` on the integration response. When API Gateway sees this content type, it automatically streams the response instead of buffering it. This is a known behavior, though not heavily documented. If you’re using CloudFormation, you can set the `ContentHandling` property to `PASSTHROUGH` and add the header mapping in the `IntegrationResponse` resource.

Lambda Streaming with Claude

If your backend is a Lambda function, you must enable Lambda response streaming. Without it, Lambda will buffer the entire Claude response before returning it, even if API Gateway is configured to stream.

1. Create a Lambda function with the python3.12 (or newer) runtime, or use the Node.js 20.x runtime.
2. In the Lambda console, go to Configuration → Function URL or API Gateway trigger.
3. For the function itself, you need to use the streaming response feature. This is done by returning a `StreamingResponse` object (in Python) or using the `awslambda.streamify` decorator.
4. Set the `Content-Type` header to `text/event-stream` in the response.

Here’s a minimal Python example that streams Claude’s response:

import json
from anthropic import Anthropic
from awslambda.streamify import streamify

client = Anthropic()

@streamify
def handler(event, context):
    def generate():
        with client.messages.stream(
            model="claude-3-5-sonnet-20241022",
            max_tokens=1024,
            messages=[{"role": "user", "content": "Tell me a story"}]
        ) as stream:
            for text in stream.text_stream:
                yield f"data: {json.dumps({'text': text})}\n\n"
    return generate()

Note: The `awslambda.streamify` module is available in the Lambda runtime when streaming is enabled. You must also configure the function’s Basic settings to have a timeout that allows the stream to complete (e.g., 5 minutes).

Common Pitfalls and Troubleshooting

• Content-Type not set: Even with buffering disabled, if the backend returns `application/json`, API Gateway will buffer. Always set `text/event-stream`.
• Compression: If you enable compression on your API, it will buffer the stream. Disable compression for the SSE route.
• Proxy integrations: For HTTP APIs with a proxy integration (e.g., Lambda proxy), the buffering setting is on the route, not the integration. Make sure you edit the route’s integration settings.
• Testing with curl: Use `curl -N` to disable client-side buffering and see the stream in real time. If you see nothing, check the API Gateway logs.
• CloudFront: If you’re using CloudFront in front of API Gateway, you must also disable buffering there. CloudFront buffers by default unless you set `CachePolicy` to one that doesn’t buffer (e.g., `Managed-CachingDisabled`).

Comparing HTTP vs REST APIs for SSE

HTTP APIs are generally simpler for SSE because they have a direct “disable buffering” checkbox. REST APIs require the header trick and are more finicky. If you’re starting fresh, use HTTP APIs. However, REST APIs offer more advanced features like custom domain names with mutual TLS, and some legacy systems require them. Both can work, but HTTP APIs are the recommended path for real-time streaming.

Cost and Performance Considerations

Disabling buffering does not change the cost of API Gateway; you still pay per request and per minute for HTTP APIs. However, streaming responses can increase the duration of the connection, which may affect Lambda costs if you’re using Lambda. With Lambda streaming, you pay for the time the function runs, including the time spent streaming. For long streams, this can add up. Also, be aware that API Gateway has a maximum integration timeout of 30 seconds for REST APIs and 30 seconds for HTTP APIs (unless you’re using a custom domain with HTTP APIs, which allows up to 5 minutes). For longer streams, consider using a different service like WebSocket or a dedicated streaming server.

Frequently Asked Questions

Why does my SSE stream work in local but not through API Gateway?

Local servers don’t buffer responses by default, but API Gateway does. You need to explicitly disable buffering and set the correct content type. Also, if you’re using a Lambda function locally, you might be using a different runtime that doesn’t require streaming, but in the cloud, Lambda’s default behavior is to buffer.

Can I use SSE with API Gateway and Lambda without streaming?

No. Lambda functions that don’t use the streaming response feature will collect the entire Claude response and return it as a single JSON object. You must enable Lambda streaming and return a streaming response.

Do I need to set `Content-Type` on the integration request or response?

On the integration response. The request content type is irrelevant for SSE. The response must have `text/event-stream`.

What if I use CloudFront? Do I need to configure it too?

Yes. CloudFront buffers responses by default. You need to create a cache policy that disables caching and buffering, and associate it with your behavior. The `Managed-CachingDisabled` policy works.

Does API Gateway support SSE with WebSocket?

No, SSE is a different protocol. API Gateway WebSocket is for full-duplex communication. For SSE, you use HTTP APIs with the streaming configuration described above.

Why do I see a 504 error after 30 seconds?

API Gateway has a 30-second integration timeout for HTTP and REST APIs (unless you use a custom domain for HTTP APIs, which allows up to 5 minutes). If your Claude stream takes longer, you’ll need to break it into chunks or use a different service.

Related

• OLED Laptops in India: Prices & Configs Compared
• Zimbra Backup Mailbox: Full Guide
• Increase Disk Partition: How to Resize in Windows