Idempotency with Redis sessions: a race condition and what we did about it.

I was working on a public endpoint. Nothing fancy — accept a request, write an audit record of the user's selection preferences to the database, return a response. The application was a modular monolith on Spring Boot that had recently been migrated off an older Struts 2 codebase. Redis had replaced the legacy in-JVM Java caches, and was also being used as the session store via Spring Session.

Because the endpoint was public and writing to the database, it needed to be idempotent — if the same client fires the same request twice (retry, double-click, network hiccup), it should execute once. Textbook stuff. I reached for the textbook solution: a single-use token.

It worked beautifully in testing. Then I load-tested it, and every assumption I had about how Spring Session + Redis actually behaves fell over. This post is the honest walkthrough of every attempt that didn't fix the race condition, the one that finally did, and what I now believe about distributed state after shipping it.

Attempt 1: single-use token in the session

The initial design was the one you'd sketch on a whiteboard in forty seconds. Generate a short-lived token when the client first lands on the page. On submit, send the token with the request. The server checks the token exists in the session, processes the request, and invalidates the token. Second request with the same token → token already consumed → reject.

// Simplified. Spring MVC controller — do NOT ship this.
@PostMapping("/preferences")
public ResponseEntity<?> savePreferences(
    @RequestBody PreferenceRequest body,
    HttpSession session
) {
    String submitted = body.getIdempotencyToken();
    String stored = (String) session.getAttribute("preference_token");

    if (stored == null || !stored.equals(submitted)) {
        return ResponseEntity.status(409).body("Token invalid or consumed");
    }

    // Mark the token as consumed, then process.
    session.removeAttribute("preference_token");
    preferenceService.save(body);

    return ResponseEntity.ok().build();
}

Fine in isolation. Fine in Postman. Fine in unit tests. Fine when a human clicks the submit button twice.

Then I pointed k6 at it with twenty concurrent virtual users all sending the same token. Twenty requests, twenty audit records written. The database happily absorbed every single one.

Why it broke

Classic time-of-check-to-time-of-use (TOCTOU) race. Every one of the twenty concurrent requests ran this sequence, interleaved:

1. Request handler fetches the session from Redis (via Spring Session's filter).
2. Reads preference_token — present, matches submitted token.
3. Processes the request. Writes the audit record.
4. Removes the token from the session.
5. Writes the session back to Redis.

Twenty requests racing through this sequence all observed the token as present in step 2, because none of them had reached step 4 yet. By the time any of them wrote back to Redis, every other request had already decided the token was valid and had kicked off the audit write.

Attempt 2: Java locks around the critical section

If the problem is that concurrent requests overlap on the read- decide-invalidate-write path, surely the fix is to put a lock around it. First attempt: synchronise on the session itself.

// Still broken — do NOT ship this.
@PostMapping("/preferences")
public ResponseEntity<?> savePreferences(
    @RequestBody PreferenceRequest body,
    HttpSession session
) {
    synchronized (session) {
        String stored = (String) session.getAttribute("preference_token");
        if (stored == null || !stored.equals(body.getIdempotencyToken())) {
            return ResponseEntity.status(409).body("Token consumed");
        }
        session.removeAttribute("preference_token");
    }
    preferenceService.save(body);
    return ResponseEntity.ok().build();
}

This looks correct. It isn't. The problem is that session is not a stable Java object across requests. Spring Session's request filter resolves the session from Redis on each request — the handler receives a different Java object instance each time, so synchronized(session) acquires a monitor on a different object each time, and no two requests actually contend.

Fine, I thought — lock on something stable instead. The session ID is literally the same string every request for a given session, by definition:

// Still broken. Different reason this time.
synchronized (session.getId()) {
    // ...
}

Also broken, and the reason trips up a lot of Java engineers: synchronized locks on object identity (reference equality, ==), not on value equality (.equals()). Two String objects with identical characters are still two different objects in the JVM with two different monitors. session.getId() is reconstructed per request — deserialised from Redis, parsed from a cookie, pulled from a UUID generator — so each call returns a fresh String object even though a.equals(b) is true. Each thread locks a different monitor. Neither waits.

OK — force interning. String.intern() canonicalises the string against the JVM's string pool, returning the single shared reference for a given value. Two threads calling "abc123".intern() get the same reference, and therefore the same monitor:

// Finally works! ...on one node.
synchronized (session.getId().intern()) {
    // ...
}

This does fix the single-JVM race. Load-tested on one node, no more duplicates. We shipped it. Then we deployed to the actual multi-node environment and the duplicates came back.

The reason: the string pool is per-JVM. Each node has its own StringTable. Node 1's interned “abc123” and Node 2's interned “abc123” are two different objects living in two different processes on two different machines. With a non-sticky load balancer, two concurrent requests for the same session can land on different nodes, each interning the ID into its own pool, each successfully acquiring its own lock — and neither knowing the other exists.

The detour: Spring Session's flush mode

Reading the Spring Session docs, I noticed flush mode. By default, Spring Session uses FlushMode.ON_SAVE: session attribute changes are buffered and written to Redis at the end of the request. Switch it to FlushMode.IMMEDIATE and every session.setAttribute() / removeAttribute() call writes to Redis immediately.

Promising-looking, but it doesn't solve the race on its own. Flush mode controls when writes are sent to Redis; it does not give you atomic compare-and-swap on an attribute. Two concurrent requests can still both read the token as present and then both race the invalidation write. One wins the write, both have already decided to process the request.

What flush mode does give you is a shorter window for the race — if your handler reads the token, immediately invalidates (which flushes to Redis), and only then does the work, a second request arriving even a millisecond later will miss the token. But “shorter window” is not the same as “no window.” Under real load, I still saw duplicates.

The right distributed answer: SET … NX EX on Redis

What I should have reached for from day one is a Redis primitive that is atomic by design: SET key value NX EX ttl. This sets a key only if it does not already exist, with an expiry. Redis guarantees atomicity across every client, every node, every connection pool. First request to send the command wins; everyone else gets back nil.

// The distributed-correct version.
@PostMapping("/preferences")
public ResponseEntity<?> savePreferences(
    @RequestBody PreferenceRequest body
) {
    String key = "idem:preferences:" + body.getIdempotencyToken();
    // Try to claim the token atomically with a 10-minute TTL.
    Boolean claimed = redisTemplate.opsForValue()
        .setIfAbsent(key, "consumed", Duration.ofMinutes(10));

    if (!Boolean.TRUE.equals(claimed)) {
        return ResponseEntity.status(409).body("Token already consumed");
    }

    preferenceService.save(body);
    return ResponseEntity.ok().build();
}

This is the textbook distributed-idempotency pattern. It works across any number of nodes with no sticky-session requirement because the arbitration happens in Redis, not in any JVM. If I were writing this post about what the solution should look like in a general distributed system, this would be the ending.

Why we actually chose an in-memory cache

Two reasons we went with an in-process bounded cache instead:

1. Sticky sessions were already a requirement. The application also used Spring Session for authenticated-user session state, and the load balancer was already configured for session affinity. Given that constraint, every request for a given session was already landing on the same node — the multi-node argument for distributed arbitration disappeared.
2. The token-check runs on every request to a public endpoint under attack conditions. Burning a Redis round-trip on every public-endpoint request felt wasteful when the arbitration could happen in O(1) in-process. Under a token- replay attack we didn't want to inflate Redis traffic too.

What we needed was a concurrent, bounded, TTL-aware map of consumed tokens, per node, with sticky sessions pinning the race to a single node. That is Caffeine's wheelhouse exactly:

import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;

@Configuration
public class IdempotencyConfig {
    @Bean
    Cache<String, Boolean> consumedTokens() {
        return Caffeine.newBuilder()
            // Match the session timeout — a token can never outlive its session.
            .expireAfterWrite(Duration.ofMinutes(30))
            // Hard ceiling so a replay attack can't balloon the heap.
            .maximumSize(1_000)
            .build();
    }
}

@PostMapping("/preferences")
public ResponseEntity<?> savePreferences(
    @RequestBody PreferenceRequest body
) {
    String token = body.getIdempotencyToken();

    // Atomic test-and-set, in-process, O(1).
    Boolean alreadyConsumed =
        consumedTokens.asMap().putIfAbsent(token, Boolean.TRUE);

    if (alreadyConsumed != null) {
        return ResponseEntity.status(409).body("Token already consumed");
    }

    preferenceService.save(body);
    return ResponseEntity.ok().build();
}

ConcurrentHashMap.putIfAbsent is atomic within a JVM — the first caller inserts, every subsequent caller gets back the existing value. Caffeine's backing map gives us the same semantics plus the TTL and the hard size cap for free. Under twenty concurrent k6 users, one request wins the insert and processes; the other nineteen each see a non-null return and get a 409.

What I'd reach for first next time

If I had to design this feature again from scratch, the order of attempts would reverse. I would reach for SET … NX EX on Redis first, specifically because it is correct across any deployment topology — single node, multi-node with sticky sessions, multi-node without sticky sessions, horizontally autoscaling. It does one job and the job is atomic.

I would reach for the Caffeine in-memory cache only when I had a specific reason to:

• Sticky sessions were already enforced for other reasons, so the multi-node correctness argument didn't apply.
• The endpoint's latency budget made a Redis round-trip per-check meaningful.
• The attack surface made the write-amplification to Redis a real cost.

All three applied in this project. In most projects, only the first does, and SETNX is the right call.

The three things I actually took away

1. Distributed sessions do not make a system distributed-safe. Moving a classic in-JVM session store to Redis-backed Spring Session feels like a drop-in upgrade. It isn't. Code that was previously safe against concurrent requests on the same session (because the JVM really was sharing one object) is no longer safe once each request gets its own snapshot from Redis. Audit every pattern that read-then-wrote to the session.
2. Atomic primitives beat locks for distributed coordination. SET NX EX, INCR, Lua scripts over Redis — these are atomic by design and work across every node. Locks (database or Redis- based) are a tool, but reach for atomic primitives first; they are cheaper to reason about and harder to deadlock.
3. Sticky sessions are a load-balancer choice with correctness consequences. If a system is designed with the assumption of sticky routing, that assumption needs to live in the architecture document, not implicit in whichever LB config shipped. Switching off sticky sessions to improve load distribution could silently reintroduce the race condition this post describes. We have an architecture note that explicitly calls this out now.