> ## Documentation Index
> Fetch the complete documentation index at: https://prism.ntecdev.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Spec 166 v0 1 marconi project broadcast fanout

# SPEC-166: Marconi Project Broadcast Fan-Out

> **Current authority notice:** This SPEC covers project-scope broadcast fan-out. The consolidated current Marconi signaling source-of-authority document is [Marconi](/marconi). Broadcast work must remain inside Marconi or record a new approved exception; Redis pubsub must not re-enter targeted delivery.

**Status:** proposed
**Owner:** Donna (PO + implementation); Texi (architecture review)
**Plan ref:** #46 P1d
**Origin:** Texi Plan #46 P1a Marconi-routing inventory (Journal #520, signal `a7548957`) flagged `publish_project_event` as the one disposition class lacking a Marconi equivalent. Full Redis pubsub retirement from signal-routing blocks on closing this gap.

## Motivation

SPEC-163 v0.7 (PR #742, commit `4401d8d`) retired Redis pubsub from `send_signal` targeted delivery — Marconi is the single transport for per-recipient sends. But the same `send_signal` retains one Redis publish call at `backend/app/services/signal_service.py:2325`: `publish_project_event` for project-scope broadcasts (signal\_type='\*' targets). Marconi's in-memory routing is session-keyed (point-to-point) and has no project-broadcast primitive yet. Until that primitive ships:

* The `publish_project_event` / `publish_org_event` / `publish_tenant_event` helpers in `backend/app/session_store/client.py:607-682` stay alive.
* `routers/session_stream.py:214-243` retains the Redis pubsub subscribe branch (it subscribes to tenant/org/project channels for the broadcast path even when Marconi handles the targeted path).
* The `PRISM_MARCONI_HOT_PATH_SEND` flag, although de-gated for targeted sends in PR #742, can't be retired from `config.py` because the broadcast path still inspects it.

This SPEC defines the Marconi-side broadcast primitive so the remaining Redis dependency in signal-routing can be deleted.

## Constraints (architectural)

1. **Marconi remains in-process.** No new external dependency; the fan-out walks the existing attached-WS-queue registry.
2. **Best-effort delivery semantics preserved.** Project broadcast is fire-and-forget today (Redis publish returns 0 on no subscribers; sender does not retry). Marconi fan-out matches: per-session put\_nowait, QueueFull / exception suppressed at DEBUG.
3. **Scope is project broadcast only.** This SPEC ports `publish_project_event` to a Marconi fan-out and deletes the Redis publisher + call site for that scope. `publish_org_event` and `publish_tenant_event` are **out of scope** and stay in source pending a separate SPEC (Marconi routing cache does not provide efficient org/tenant membership today per Texi G1 verdict). Targeted delivery is already on Marconi-only (PR #742).
4. **Wire envelope shape unchanged.** Subscribers (`cli/native/signal_stream.cjs`, `native-console/src/renderer/renderer.tsx`) see the same envelope structure regardless of fan-out origin.
5. **No persistence changes.** Project broadcasts are not persisted today (no signal\_queue row written for `to_identity='*'` sends per current send\_signal flow); this SPEC does not introduce persistence.

## Acceptance criteria

### AC-1 — Marconi delivery exposes a project-scoped queue enumerator

`backend/app/marconi/delivery.py` adds an async function `iter_project_ws_queues(tenant_id: str, project_id: str) -> AsyncIterator[tuple[str, asyncio.Queue]]` (or equivalent) that yields `(session_id, queue)` for every attached WS queue belonging to the project's live sessions.

**Membership source (Texi G1 verdict 2026-06-04):** project membership is resolved via the **Marconi routing cache / SessionStore project session set** — that's the existing in-memory record of which sessions belong to which project. `_session_queues` is the attached-WS-queue map ONLY, not the membership oracle. The flow is:

1. Read the project's session set (existing data structure).
2. Snapshot the matching `session_id → Queue` entries from `_session_queues` under `_session_queues_lock`.
3. Release the lock.
4. Yield / fan out from the snapshot.

**Lock invariant (Texi G1 verdict):** `_session_queues_lock` is held ONLY for the snapshot construction. It is **never** held across `put_nowait`, logging, exception handling, or any other fan-out work. This matches the targeted-send profile (`get_ws_queue(session_id)` holds the lock only for the read) and keeps fan-out off the lock's critical path.

**Org/tenant scope NOT covered by this G1 verdict.** The Marconi routing cache does not efficiently provide org/tenant membership today. `iter_org_ws_queues` and `iter_tenant_ws_queues` are **deferred** to a separate SPEC + indexing phase. This SPEC's Phase 3 (`AC-5`/`AC-6` deletions) is scoped to project broadcast only; the org/tenant `publish_*_event` helpers stay in source pending that follow-up SPEC.

### AC-2 — Marconi delivery exposes a project-scope broadcast publisher

`backend/app/marconi/delivery.py` adds:

* `async def publish_project_via_marconi(*, tenant_id, project_id, envelope) -> int` — iterates `iter_project_ws_queues`, `put_nowait`s the envelope to each, returns the count successfully queued.

Per-queue exceptions (`QueueFull` / arbitrary) are caught and logged at DEBUG; loop continues. Return count = sessions reached, not sessions attempted.

Org and tenant publishers are deferred per AC-1 note.

### AC-3 — `signal_service.py` broadcast call site replaced (project-scope only)

The call at `backend/app/services/signal_service.py:2325` (`store.publish_project_event(...)`) is replaced with `publish_project_via_marconi(...)`. **Project-scope only this PR.** Org/tenant call sites (`publish_org_event` / `publish_tenant_event`) are NOT migrated in this PR per Constraint 3, AC-1 G1, AC-6, and Out of Scope — they are deferred to a follow-up SPEC and remain in source unchanged. Wire envelope shape matches the existing Redis pubsub payload so subscribers continue to deserialize without change.

### AC-4 — `routers/session_stream.py` Redis subscribe branch deleted

After AC-3 lands, the Redis subscribe branch at `backend/app/routers/session_stream.py:214-243` (subscribing tenant/org/project channels) has no remaining publisher. The branch is deleted; the Marconi-only path (line 174-212) becomes the only WS handler.

### AC-5 — Flag retirement

`PRISM_MARCONI_HOT_PATH_SEND` is removed from `backend/app/config.py` (field deleted; env var no longer read anywhere in the codebase). Any remaining references to `marconi_hot_path_send` are removed.

**Diagnostic field disposition (Texi G1 verdict):** `backend/app/marconi/diagnostics.py:148`'s `hot_path_send` field is **deleted or renamed**, not stubbed to return a constant. A retired env flag should not surface as a perpetually-true field in operator diagnostics — that's misleading observability. Pick one:

* Delete the field entirely (preferred — the flag no longer exists).
* Rename to a more accurate post-retirement label (e.g. `marconi_targeted_unconditional: true`) if downstream consumers reference the key.

### AC-6 — Helper deprecation / deletion

After AC-3, `publish_project_event` and the targeted `_publish_targeted_event` (now unused since SPEC-163 v0.7) helpers in `backend/app/services/signal_service.py` and `backend/app/session_store/client.py` have zero call sites. They are deleted from source. The `session_store/client.py:18` docstring referencing them is updated. Pin test added so the symbols can't reappear.

**`publish_org_event` / `publish_tenant_event` are NOT deleted by this SPEC** — they stay in `session_store/client.py` pending the deferred org/tenant SPEC (per AC-1 G1 note). Their dead-code status is documented in their docstrings.

### AC-7 — Backend test sweep

Full backend test sweep green (modulo the three pre-existing failures in `test_spec140_process_architecture_registry` + `test_version_sot` that are unrelated to signal). New tests cover:

* `iter_project_ws_queues` over a fixture registry returns the expected (session\_id, queue) tuples
* `publish_project_via_marconi` fan-out delivers to all attached queues
* `publish_project_via_marconi` continues past QueueFull / arbitrary exceptions
* Pin test in `test_spec163_marconi_queue_fallback.py` extended: `publish_project_event` symbol does not exist on the deployed `session_store/client.py` module after this SPEC ships. (`publish_agent_event` / `publish_session_event` symbols are retired by the SPEC-163 v0.7 P1b sweep PR #742 + its follow-up cleanup, not by this SPEC. `publish_org_event` / `publish_tenant_event` remain present per Constraint 3 scope.)

## Test scenarios

Headline scenario: send a Question signal with `to_identity='*'` to a project with 3 attached console\_cc sessions (e.g. Lafonda + Candi + Porsche on mini3). Confirm:

1. Each recipient's WS frame receive log shows the broadcast envelope.
2. **No Redis publish call** is logged during the send (verify via `redis-cli MONITOR`).

The pre-revision draft included a "drain or piggyback eventually surfaces the broadcast in `pending_signals`" check. **That check is removed (Texi G1 verdict):** broadcast is attached-WS-only best-effort. The no-persistence semantics in the "Out of scope" section explicitly exclude broadcast persistence and late-joiner replay. A receiver not attached to its WS at fan-out time misses the broadcast — that is the design, not a defect. Re-introducing a persistence-or-drain check would conflict with the SPEC's own out-of-scope clause.

## Implementation phases

* **Phase 1 (this SPEC + AC-1, AC-2):** Add the Marconi-side enumerators + broadcast publishers. Net-new code. No call-site changes.
* **Phase 2 (AC-3):** Replace `signal_service.py:2325` with the new publisher. Add fan-out tests.
* **Phase 3 (AC-4, AC-5, AC-6):** Delete the Redis subscribe branch, the flag, the dead helpers. Pin tests close the door.

Phases ship as separate PRs to keep each diff reviewable. Phase 1 + 2 = no behavior change at the WS layer (subscribers still get the same wire envelope). Phase 3 = the structural cleanup.

## Pre-PR gate

**G1 — Marconi registry enumeration cost.** Before AC-1 ships, confirm the in-memory session registry can be enumerated O(sessions\_in\_project) without holding a global lock that affects targeted-send latency. If the existing registry is keyed only by session\_id, AC-1 may need an auxiliary project\_id → session\_id index built incrementally on attach/detach. Texi architecture review verdict required.

## Out of scope (deferred)

* **Marconi audit fan-out** — persistence of broadcast envelopes for late-joiner replay. Tracked separately by SPEC-101 Tier 2/3.
* **Cross-tenant broadcast** — explicitly disallowed by tenant-isolation invariants; this SPEC does not weaken that.
* **Broadcast obligations** — `to_identity='*'` sends do not create signal\_obligations rows today; this SPEC preserves that.

## Source refs

* `backend/app/services/signal_service.py:2325` — broadcast `publish_project_event` call site
* `backend/app/session_store/client.py:607-682` — Redis publish helpers
* `backend/app/routers/session_stream.py:214-243` — Redis subscribe branch
* `backend/app/marconi/delivery.py` — current targeted delivery + queue registry
* Journal #520 — Texi Plan #46 P1a inventory
* PR #742 (commit `4401d8d`) — SPEC-163 v0.7 targeted Marconi-only sweep
* Plan #46 P1d — execution tracker

## Supersession

Closes the gap left by SPEC-163 v0.7. Together with PR #742, this SPEC's full execution retires Redis pubsub from signal-routing end to end.