​

SPEC-032 v1.2 — Redis Session Plane

​

Status

​

Changelog

​

1. Summary

​

2. Motivation — a real bug, caught live

​

3. The NetBIOS analogy, taken seriously

​

4. Architectural commitments (load-bearing)

• MCP at the boundary — no change. MCP clients talk to the backend via HTTP + gRPC; they never open Redis connections.
• gRPC bidirectional streams inside the control plane — no change; the event-push channel under them switches from asyncpg LISTEN to Redis SUBSCRIBE (backend-side).
• Backend is the only router — no change, and reinforced. All Redis access flows through the backend. External clients authenticate via API keys against the backend; the backend translates to Redis operations. Review flag (Lola): re-evaluate when SPEC-028 (TS MCP) lands. This commitment survives the TS migration more cleanly now that clients never need Redis credentials.
• Capability tokens + leases are DB rows — no change; Postgres retains them.
• Master election is project-scoped — no change.
• CD always wins election when present — no change; implemented via atomic Lua CAS (see §5.2.1).
• Single-master invariant — no change; enforced by atomic SETNX on a single key plus Lua compare-and-swap for preemption.
• Redis is backend-only (new, v1.2) — MCP clients, CLI tools, and any future external consumer reach coordination state through backend HTTP / gRPC endpoints. Redis credentials live in one env (backend); no client ever sees PRISM_REDIS_URL.

​

5. What moves to Redis

​

5.1 Key schema

# Per-session hash. TTL = PRISM_SESSION_TTL_SECONDS (default 90).
prism:session:{session_id} → HASH {
  tenant_id, project_id, agent_identity, agent_surface,
  machine_id, is_master (0|1), registered_at (ISO8601)
}

# Master lock per project. Value is the session_id holding master.
# Same TTL as the session hash. Claimed via SETNX; preempted via Lua CAS (§5.2.1).
prism:master:{tenant_id}:{project_id} → STRING session_id

# Per-project active-session set. SADD on register, SREM on release/expire.
prism:project:{tenant_id}:{project_id}:sessions → SET of session_ids

# Pub/sub channels.
prism:events:project:{tenant_id}:{project_id}   # project-scoped broadcast
prism:events:session:{session_id}                # session-targeted push

​

5.2 Semantics — register / elect (prism_start)

1. Generate session_id (UUID).
2. HSET prism:session:{session_id} + EXPIRE to TTL (90s).
3. Try SETNX prism:master:{tenant}:{project} session_id + EXPIRE to TTL.
   • Success → caller is master; HSET is_master 1.
   • Failure → master exists. If caller is claude_desktop, preempt via the Lua CAS in §5.2.1 — NOT a plain SET ... XX.
4. SADD prism:project:{tenant}:{project}:sessions session_id.
5. Return controller_status assembled from Redis reads.

​

5.2.1 Preemption CAS — atomic via Lua

-- KEYS[1] = prism:master:{tenant}:{project}
-- ARGV[1] = expected incumbent session_id (the master we read moments ago)
-- ARGV[2] = new master session_id (the preempting claude_desktop)
-- ARGV[3] = TTL seconds
-- Returns: 1 on success (we atomically replaced the expected incumbent),
--          0 if the incumbent had already changed (retry or join as peer).
local current = redis.call("GET", KEYS[1])
if current == ARGV[1] then
  redis.call("SET", KEYS[1], ARGV[2], "EX", ARGV[3])
  return 1
end
return 0

1. GET prism:master:... → incumbent_session_id.
2. Run the Lua script with incumbent_session_id as the expected value.
3. Script returns 1 → caller is master. PUBLISH prism:events:session:{incumbent_session_id} with MasterPreempted payload.
4. Script returns 0 → another preemption beat us, OR the incumbent expired. Re-read the current master; retry if appropriate, or join as peer. Bounded retry (max 3) to avoid livelock.

​

5.3 Semantics — heartbeat (Option B: via backend, never direct)

MCP client (any machine)                  backend (inside Redis network)
        │                                        │
        │  POST /api/v1/controller/heartbeat      │
        │  Headers: Authorization: Bearer <key>   │
        │           X-Prism-Session-Id: <sid>     │
        ├────────────────────────────────────────▶│
        │                                        │
        │                                        │  PIPELINE:
        │                                        │    EXPIRE prism:session:{sid} TTL
        │                                        │    EXPIRE prism:master:{t}:{p} TTL
        │                                        │      (if session holds master)
        │                                        │
        │            200 {ok: true,               │
        │                 ttl_remaining: 90}      │
        │◀────────────────────────────────────────┤

• Cloud security. Managed Redis (Upstash/ElastiCache) never exposes credentials to user machines. Only the backend holds the URL. A leaked user API key risks one user’s session; a leaked Redis URL risks the whole coordination plane.
• LAN security. ufw rule on server1 stays locked to admin-IP for Redis port 46379. No need to broaden to the LAN subnet for MCP reachability.
• Single place for Redis knowledge. Backend owns the client, connection pool, Lua registry, retry policy, observability. If we swap Redis for another store later, only the backend changes.
• Per-user authentication + audit. Backend authenticates each heartbeat via API key; can rate-limit and log per-user activity. Direct Redis has no per-user story.
• Client dependency footprint. MCP stays lightweight — no redis library, no PRISM_REDIS_URL env.

# Pseudocode — heartbeat runs on its own event loop in a daemon thread.
# Talks HTTP to the backend, NOT Redis directly.

async def _heartbeat_loop(
    http_client: httpx.AsyncClient,
    api_url: str,
    api_key: str,
    session_id: str,
    stop: asyncio.Event,
) -> None:
    """Runs on a dedicated asyncio loop. Isolated from the MCP main loop."""
    headers = {
        "Authorization": f"Bearer {api_key}",
        "X-Prism-Session-Id": session_id,
    }
    while not stop.is_set():
        try:
            r = await http_client.post(
                f"{api_url}/api/v1/controller/heartbeat",
                headers=headers,
                timeout=5.0,
            )
            if r.status_code >= 400:
                log.warning("heartbeat rejected (status=%s)", r.status_code)
        except httpx.HTTPError as exc:
            log.warning("heartbeat HTTP error (will retry): %s", exc)
        try:
            await asyncio.wait_for(stop.wait(), timeout=30.0)
            return
        except asyncio.TimeoutError:
            pass


def spawn_heartbeat(api_url, api_key, session_id):
    """Returns a stop_event + thread handle. Called from prism_start."""
    loop = asyncio.new_event_loop()
    stop = asyncio.Event()

    def run():
        asyncio.set_event_loop(loop)
        client = httpx.AsyncClient()
        try:
            loop.run_until_complete(
                _heartbeat_loop(client, api_url, api_key, session_id, stop)
            )
        finally:
            loop.run_until_complete(client.aclose())
            loop.close()

    thread = threading.Thread(target=run, daemon=True, name=f"hb-{session_id[:8]}")
    thread.start()
    return _HeartbeatHandle(stop_event=stop, thread=thread, loop=loop)

# backend/app/routers/controller.py
@router.post("/heartbeat", response_model=HeartbeatResponse)
async def heartbeat(
    x_prism_session_id: str = Header(...),
    ctx: AuthContext = Depends(require_auth),
    session_store=Depends(get_session_store),  # Redis-backed
) -> HeartbeatResponse:
    """Refresh the caller's session (and master, if held) TTL in Redis."""
    refreshed = await session_store.refresh_ttl(
        tenant_id=ctx.tenant_id,
        session_id=x_prism_session_id,
    )
    if not refreshed:
        # Session key expired between last heartbeat and this one — caller
        # must re-register via prism_start.
        raise HTTPException(410, "session expired, re-register via prism_start")
    return HeartbeatResponse(ok=True, ttl_remaining=refreshed.ttl)

​

5.3.1 Re-election trigger via keyspace notifications

1. Redis is configured with notify-keyspace-events Ex (see §8.6) — enables generic key-expiration events on the __keyevent@{db}__:expired channel.
2. The backend-grpc ControllerEventListener subscribes to this channel at startup, filters for prism:master:* key patterns.
3. On a master-key expiration event, the backend publishes a master_released event on prism:events:project:{tenant}:{project} with the expired session_id.
4. Active peers on that project receive the event via their own subscription (or at their next prism_start via controller_status showing an empty master). Whichever peer acts first wins SETNX on the now-empty master key. No polling, no timer.

​

5.4 Semantics — release (prism_wrap)

PIPELINE:
  DEL prism:session:{session_id}
  DEL prism:master:{tenant}:{project}          # only if holding
  SREM prism:project:{tenant}:{project}:sessions session_id
  PUBLISH prism:events:project:{tenant}:{project}  # session_ended event

​

5.5 Semantics — checkpoint (prism_checkpoint, SPEC-031)

​

5.6 Server-push events

• PUBLISH prism:events:session:{target} {json_payload} — session-targeted (backend-side)
• PUBLISH prism:events:project:{tenant}:{project} — broadcast (backend-side)

• prism:events:session:{caller_session_id} — targeted channel
• prism:events:project:{tenant}:{project} — broadcast channel

​

5.7 prism_status

​

6. What stays in Postgres

​

7. What this eliminates

• controller_registrations table → becomes optional append-only audit log, then retired (Phase D).
• ControllerSweepWorker → delete.
• controller_service.stamp_heartbeat → delete.
• grpc_runtime/listener.py asyncpg LISTEN → replaced by Redis SUBSCRIBE.
• Partial UNIQUE index ix_controller_single_master → moot.
• The silent-release bug → fixed structurally via TTL.
• The gRPC-vs-HTTP heartbeat split brain → one path, one primitive.
• The X-Prism-Session-Id heartbeat side effect in authforge → removed. Heartbeat is explicit via /controller/heartbeat, not a side effect of other verbs.

​

8. What this adds

​

8.1 Runtime dependencies

• Redis runtime dependency — new container in all compose stacks.
• redis>=5.0 in backend/requirements.txt (async Python client). Backend only — MCP does NOT take this dep under Option B.
• New package backend/app/session_store/ — Redis client, pipeline builders, pub/sub machinery, Lua script registration.
• New module mcp/heartbeat.py — dedicated asyncio loop on daemon thread, HTTP client talking to backend /controller/heartbeat (no direct Redis access).
• New backend endpoint POST /api/v1/controller/heartbeat — thin wrapper that validates session + issues EXPIRE.
• PRISM_REDIS_URL env var — backend only. MCP reads only PRISM_API_URL + PRISM_API_KEY (unchanged from today).

​

8.2 Local mode (personal install)

redis:
  image: redis:7-alpine
  container_name: prism-personal-redis
  restart: unless-stopped
  command: >
    redis-server
    --save ""
    --appendonly no
    --notify-keyspace-events Ex
    --requirepass "prism_personal"
  # No host port published — Redis reachable only from the compose network.
  # Backend connects via internal DNS `redis:6379`.
  healthcheck:
    test: ["CMD", "redis-cli", "-a", "prism_personal", "ping"]
    interval: 5s
    timeout: 5s
    retries: 10

​

8.3 LAN mode (server install)

redis:
  image: redis:7-alpine
  container_name: prism-server-redis
  restart: unless-stopped
  command: >
    redis-server
    --save ""
    --appendonly no
    --notify-keyspace-events Ex
    --requirepass "${PRISM_REDIS_PASSWORD:-prism_server}"
  # No host port published under Option B. Backend access via compose network.
  healthcheck:
    test: ["CMD", "redis-cli", "-a", "${PRISM_REDIS_PASSWORD:-prism_server}", "ping"]
    interval: 5s
    timeout: 5s
    retries: 10

• New config constant: PRISM_REDIS_PASSWORD (generated via openssl rand -base64 24).
• No ufw rule needed for Redis — port isn’t published to the host. Simpler than v1.1’s admin-IP rule.
• /etc/prism-server.conf template grows PRISM_REDIS_PASSWORD line.
• Post-install smoke: docker compose exec redis redis-cli -a "$PRISM_REDIS_PASSWORD" ping

​

8.4 Cloud mode (hosted)

• Operator provisions managed Redis, gets URL + credentials.
• PRISM_REDIS_URL set in the backend’s deployment env. Never in client / MCP env.
• MCP clients authenticate to the backend via API key (unchanged from today). Backend uses its Redis credentials to do the actual coordination work.
• A leaked API key compromises one user’s session plane (revocable). A leaked backend Redis URL would compromise the whole cluster — which is why only the backend ever sees it.
• install.py --backend=cloud hard-fails if the backend deployment doesn’t have PRISM_REDIS_URL set.
• TLS via rediss:// is transparent through redis-py.

​

8.5 SPEC-019 env resolution for PRISM_REDIS_URL

@property
def effective_redis_url(self) -> str:
    if self.prism_redis_url:
        return self.prism_redis_url
    mode = self.prism_mode.lower()
    if mode in ("local", "personal", "development"):
        return "redis://:prism_personal@redis:6379/0"
    if mode == "lan":
        pwd = os.environ.get("PRISM_REDIS_PASSWORD", "prism_server")
        return f"redis://:{pwd}@redis:6379/0"
    if mode in ("cloud", "hosted"):
        raise RedisConfigError(
            "cloud mode requires PRISM_REDIS_URL — point at managed Redis"
        )
    return "redis://127.0.0.1:6379/0"

​

8.6 Redis configuration

• Persistence: AOF off, RDB off. State is ephemeral by definition — restart = re-register.
• Keyspace notifications: notify-keyspace-events Ex (generic key-expiration). Required for §5.3.1 re-election. Set via compose command: arg in local/lan; manual config for cloud (provider-specific).
• Eviction policy: maxmemory-policy noeviction. Session keys are TTL-managed; eviction would corrupt the coordination plane.
• Memory budget: ~100KB per active project — negligible.
• Volume mount: Optional, for operator debugging only.

​

9. Redis availability posture — hard prerequisite

• prism_start errors on coordination features. It does NOT silently degrade to an in-process singleton.
• POST /controller/heartbeat returns 503; MCP heartbeat thread logs a warning and retries.
• HTTP + gRPC domain verbs that hit Postgres continue working.
• prism_status returns an error for controller status, not a degraded response.

​

10. Phased migration

​

Phase A — Dual-write, Redis-authoritative read

• Add Redis to all compose stacks + install scripts.
• Add session_store package (backend-side Redis client + Lua registry + keyspace listener).
• Add POST /api/v1/controller/heartbeat endpoint on backend.
• Add mcp/heartbeat.py (dedicated-asyncio-loop-on-thread pattern, HTTP client).
• Register preemption Lua script at backend startup; cache SHA.
• controller_service.register writes BOTH Redis AND Postgres.
• Reads come from Redis.
• ControllerSweepWorker continues as safety net (loosened to 30 min TTL).
• MCP heartbeat spawned on prism_start.
• Redis keyspace notifications enabled in compose configs.
• Exit criterion: application-level SLI — see §14 criterion #13.

​

Phase B — Retire Postgres path

• Remove ControllerSweepWorker.
• Remove stamp_heartbeat.
• Postgres row becomes append-only audit.
• Remove X-Prism-Session-Id heartbeat side effect from authforge.

​

Phase C — Pub/sub migration

• Dual-path: producers PUBLISH to Redis AND pg_notify.
• Consumers (backend-grpc) subscribe to Redis; LISTEN path logs-only.
• Keyspace-notification-driven re-election wired into backend-grpc ControllerEventListener.
• After one week green, remove LISTEN infrastructure.

​

Phase D — Optional audit-table retirement

• Drop controller_registrations via forward migration if unused.

​

11. File-by-file diff summary

​

12. Relationship to other specs

• Supersedes SPEC-030 §5 + §11. Does NOT supersede §8, §9, §13.
• Complements SPEC-031 (checkpoint = heartbeat-without-release via the same /controller/heartbeat endpoint).
• Informs SPEC-028 (TS MCP gets a tiny HTTP-ping heartbeat model — no Redis client library needed in TypeScript).
• Uses SPEC-019 for env resolution.
• Corrects wrap-rate inflation from sweep-released sessions.

​

13. Resolved design questions

​

14. Acceptance criteria

1. PRISM_REDIS_URL resolves per SPEC-019 mode profiles on the backend only.
2. All compose stacks gain Redis service (no host port published).
3. backend/app/session_store/ exists with typed client + pipeline helpers + Lua script registry.
4. prism_start writes Redis (+ Postgres in Phase A).
5. prism_start returns identical controller_status shape.
6. MCP heartbeat spawns on prism_start via mcp/heartbeat.py; POSTs to backend /controller/heartbeat every 30s on its own asyncio loop in a daemon thread.
7. Backend /controller/heartbeat endpoint refreshes session + master TTLs via Redis EXPIRE.
8. TTL expiry → next prism_start wins master — no worker.
9. Redis DOWN → prism_start errors; /controller/heartbeat returns 503.
10. Multi-container backend-grpc: PUBLISH lands on all subscribers.
11. Smoke test covers full session-plane round trip: register, heartbeat via HTTP, natural expiry, preempt (via Lua), release, pub/sub delivery to gRPC stream.
12. Silent-release bug is fixed structurally (no sweep worker required).
13. Phase A exit SLI: over a continuous 7-day window, ≥99% of prism_start calls complete without a Redis-coordination error, AND zero code paths read from controller_registrations for live decisions (grep-verified). Application-level SLI, measurable via request logs.
14. prism_checkpoint issues heartbeat-refresh without DEL.
15. Concurrent-CD preemption test (§5.2.1 Lua CAS) passes.
16. Keyspace-notification-driven re-election test passes.
17. MCP does NOT take redis as a dependency. mcp/requirements.txt unchanged from pre-SPEC-032 state.
18. mcp/heartbeat.py tested in isolation — verify HTTP ping cadence, retry on failure, clean shutdown on stop_event.

​

15. Authorship + review trail

• Original author: Donna (Claude Code, session a54a1f65) 2026-04-23 16:19Z.
• v1.0 reviewer: Lola (Claude Desktop, session 97e32dbf) 2026-04-23 18:29Z.
• v1.1 implementation reviewer: Donna (Claude Code, session a54a1f65) 2026-04-23 PM. Scope: concurrent-CD race (§5.2.1), heartbeat concurrency model (§5.3), re-election trigger (§5.3.1), measurable acceptance criterion (§14 #13), rationale column in §13.
• v1.2 architectural refinement: Donna (Claude Code) + Frank 2026-04-23 PM. Scope: Option B (backend-mediated heartbeat) adopted after Frank’s steering during implementation review. Redis stays backend-only — cloud credentials never exposed to clients, LAN ufw stays tight, single source of Redis knowledge.
• Steering: Frank — questioned Postgres choice, directed Redis-as-anchor posture, confirmed Option B at implementation review.
• Trigger: silent-master-release bug during SPEC-030 wrap-discussion prep.