Payment Idempotency Keys: Redis & Duplicate Charges

Payment idempotency keys prevent duplicate charges when a client retries a request, and Redis is a common choice for storing them because it offers fast reads, atomic operations, and built-in expiration. You send a unique key with each payment request; the server checks Redis for that key and, if found, returns the original response instead of processing the payment again. This article explains how Redis implements idempotency keys, how to choose the right TTL and storage strategy, and what pitfalls cause duplicate charges despite using them.

Formula

The calculator estimates how many idempotency keys accumulate in Redis over a retry window, and the memory they consume. The formulas are:

Estimated keys stored = n × (t × 3600 / 60)

Memory used (MB) = (estimated keys × 200 bytes) / 1,048,576

Where:

• t = retry window in hours (how long you keep keys)
• n = max requests per key (how many unique keys you expect per minute)
• 200 bytes = rough average size of a Redis key-value pair with a short UUID and a small JSON response

Worked Example

Using the defaults: retry window = 24 hours, max requests per key = 100 per minute.

1. Convert hours to minutes: 24 × 3600 = 86,400 seconds; 86,400 / 60 = 1,440 minutes.
2. Multiply by requests per minute: 100 × 1,440 = 144,000 keys.
3. Memory: 144,000 × 200 = 28,800,000 bytes; 28,800,000 / 1,048,576 ≈ 27.47 MB.

The calculator shows 144,000 keys and 27.47 MB. That is a trivial amount of RAM for a Redis instance, which is why idempotency keys are considered cheap to store.

Why Idempotency Keys Matter in Payments

Payment networks are unreliable by design. A client may send a request, the server processes it, but the response is lost in transit. The client times out and retries. Without an idempotency key, the second request looks like a new payment and the customer is charged twice. With a key, the server recognizes the retry and returns the stored result from the first attempt.

Idempotency keys are not the same as transaction IDs. A transaction ID is generated by the payment processor after the charge. An idempotency key is generated by the client (or your server) before the request, and it must be unique per logical operation. For example, if a user clicks “Buy� once, you generate one key. If they click again, you send the same key. The server sees the same key and does not charge again.

How Redis Stores Idempotency Keys

Redis is a natural fit because it is an in-memory data store with atomic commands. The typical pattern is:

1. Client generates a UUID (or any unique string) and sends it in the Idempotency-Key header.
2. Server checks Redis: GET key. If a value exists, return it (the stored response).
3. If not, server uses SET key value NX EX ttl (set if not exists, with expiration). If the SET returns OK, this is the first request; process the payment.
4. After processing, store the response under the same key with the same TTL.

The NX flag is crucial. It ensures only one concurrent request with the same key can succeed. Without it, two simultaneous retries could both pass the GET check and both process the payment. Redis’s single-threaded command execution makes this atomic.

Choosing a TTL for Idempotency Keys

The TTL (time-to-live) determines how long you keep a key. It must be longer than the maximum time a client might retry. Common choices:

• 24 hours – safe for most payment flows; matches many payment processor recommendations.
• 7 days – for subscriptions or delayed retries (e.g., a mobile app that queues requests offline).
• 1 hour – for internal services with short timeouts, but risky if a client retries after a network partition.

If the TTL expires and the client retries with the same key, the server treats it as a new request and charges again. That is a duplicate charge. To avoid this, set the TTL at least as long as your client’s maximum retry window, plus a safety margin. Some systems use 48 hours to cover a full day of retries plus overnight maintenance.

Common Pitfalls That Cause Duplicate Charges

Even with Redis, duplicates happen. The most common causes:

• Not using NX – a check-then-set race condition allows two concurrent requests to both see “no keyâ€� and both process.
• Key collision – using a weak key like a timestamp or user ID. Two different payments for the same user in the same millisecond collide, and the second is wrongly blocked (or worse, if you don’t check, you get a duplicate).
• Storing only a boolean – if you store “processedâ€� but not the response, a retry after a crash returns nothing useful, and the client may send a new key.
• TTL too short – the key expires before the client stops retrying.
• Redis outage – if Redis is down, some implementations fail open (process the payment) to avoid blocking sales, which defeats idempotency. Fail closed (reject the request) is safer.

A concrete example: a payment gateway times out after 10 seconds. The client retries at 5 seconds, 15 seconds, and 60 seconds. If your TTL is 30 seconds, the third retry arrives after the key expired, and the customer is charged twice. Set TTL to at least 24 hours to cover this.

Redis vs. Database vs. In-Memory Map

You can implement idempotency without Redis, but each alternative has trade-offs:

• Redis – fast, atomic, automatic expiration. Requires running a Redis instance and handling its failure.
• PostgreSQL/MySQL – use a unique constraint on the key column. Durable, but slower (disk I/O) and requires a cleanup job to delete old keys.
• In-memory map (e.g., ConcurrentHashMap) – fastest, but lost on restart, and cannot be shared across multiple server instances.

Redis is the best balance for most payment services because it is shared, fast, and has TTL. The main risk is Redis availability; use Redis Sentinel or a managed service with persistence to reduce data loss.

Cost and Capacity Considerations

Storing idempotency keys is cheap. A key (UUID) plus a small JSON response (status, transaction ID) is typically under 200 bytes. For 1,000 payments per minute with a 24-hour window, you store about 1.44 million keys (1,000 × 1,440 minutes), which uses roughly 288 MB. A small Redis instance handles that easily. The calculator above lets you estimate your own numbers.

Memory grows linearly with request rate and TTL. If you have a burst of 10,000 requests per minute for an hour, that is 600,000 keys. If you set TTL to 7 days, you accumulate 4.2 million keys. Still manageable, but you should monitor Redis memory and set maxmemory with an eviction policy. Do not use allkeys-lru for idempotency keys, because evicting a key can cause a duplicate charge. Use noeviction or a separate Redis instance for idempotency data.

Frequently Asked Questions

What happens if two requests with the same idempotency key arrive at the exact same time?

Redis’s SET NX is atomic, so only one request gets the key. The other request sees the key exists and waits for the stored response. If the first request fails (e.g., payment declined), you should store the failure response too, so the retry returns the same failure instead of retrying the charge.

Should I use the payment processor’s idempotency key or my own?

Use your own. Payment processors like Stripe accept your key, but you need one before you call them. Generate a UUID per logical operation (e.g., one checkout session) and reuse it for all retries of that operation. Do not generate a new key on each retry.

Can I use the same idempotency key for two different payments?

No. Each key must be unique per operation. If you reuse a key for a different amount or currency, the server will return the original response, and the second payment will not be processed. This is a feature, but it means you must generate a fresh key for each new payment intent.

What if Redis loses the key before the TTL expires?

If Redis restarts and persistence is not enabled, all keys are lost. A retry with the same key will be treated as new, causing a duplicate charge. Enable Redis AOF (append-only file) with appendfsync everysec to minimize loss, or use a managed Redis with automatic backups. For critical payments, some systems also store a hash of the key in a database as a fallback.

Is it safe to use a timestamp as an idempotency key?

No. Timestamps are not unique enough. Two requests in the same millisecond collide, and a user can trigger two different payments within the same second. Use a UUID v4 or a random 128-bit string. If you need a human-readable key, combine a user ID with a UUID.

How long should I keep idempotency keys after a successful payment?

Keep them at least as long as your refund or dispute window, because a user might retry the same action (e.g., double-click) after the payment succeeds. Most systems use 24 to 48 hours. Some keep them for 7 days to cover chargebacks. After that, you can delete them; the client should not retry that key anymore.

Related

• GeForce Experience Scanning Failed: Fixes That Work
• Star Wars Zero Company Den Upgrades: Effects, Unlocks, Build Order
• Picture to Text: How It Works & Best Ways to Use It

in t | Retry window (hours) | h | 24 | 1 | 168
in n | Max requests per key | - | 100 | 1 | 10000
out keys | Estimated keys stored | keys | n * (t * 3600 / 60) | 0
out mem | Memory at 200 bytes/key | MB | keys * 200 / 1048576 | 2