​

SPEC-166: Marconi Project Broadcast Fan-Out

Current authority notice: This SPEC covers project-scope broadcast fan-out. The consolidated current Marconi signaling source-of-authority document is Marconi. Broadcast work must remain inside Marconi or record a new approved exception; Redis pubsub must not re-enter targeted delivery.

Status: proposed Owner: Donna (PO + implementation); Texi (architecture review) Plan ref: #46 P1d Origin: Texi Plan #46 P1a Marconi-routing inventory (Journal #520, signal a7548957) flagged publish_project_event as the one disposition class lacking a Marconi equivalent. Full Redis pubsub retirement from signal-routing blocks on closing this gap.

​

Motivation

SPEC-163 v0.7 (PR #742, commit 4401d8d) retired Redis pubsub from send_signal targeted delivery — Marconi is the single transport for per-recipient sends. But the same send_signal retains one Redis publish call at backend/app/services/signal_service.py:2325: publish_project_event for project-scope broadcasts (signal_type=’*’ targets). Marconi’s in-memory routing is session-keyed (point-to-point) and has no project-broadcast primitive yet. Until that primitive ships:

• The publish_project_event / publish_org_event / publish_tenant_event helpers in backend/app/session_store/client.py:607-682 stay alive.
• routers/session_stream.py:214-243 retains the Redis pubsub subscribe branch (it subscribes to tenant/org/project channels for the broadcast path even when Marconi handles the targeted path).
• The PRISM_MARCONI_HOT_PATH_SEND flag, although de-gated for targeted sends in PR #742, can’t be retired from config.py because the broadcast path still inspects it.

This SPEC defines the Marconi-side broadcast primitive so the remaining Redis dependency in signal-routing can be deleted.

​

Constraints (architectural)

1. Marconi remains in-process. No new external dependency; the fan-out walks the existing attached-WS-queue registry.
2. Best-effort delivery semantics preserved. Project broadcast is fire-and-forget today (Redis publish returns 0 on no subscribers; sender does not retry). Marconi fan-out matches: per-session put_nowait, QueueFull / exception suppressed at DEBUG.
3. Scope is project broadcast only. This SPEC ports publish_project_event to a Marconi fan-out and deletes the Redis publisher + call site for that scope. publish_org_event and publish_tenant_event are out of scope and stay in source pending a separate SPEC (Marconi routing cache does not provide efficient org/tenant membership today per Texi G1 verdict). Targeted delivery is already on Marconi-only (PR #742).
4. Wire envelope shape unchanged. Subscribers (cli/native/signal_stream.cjs, native-console/src/renderer/renderer.tsx) see the same envelope structure regardless of fan-out origin.
5. No persistence changes. Project broadcasts are not persisted today (no signal_queue row written for to_identity='*' sends per current send_signal flow); this SPEC does not introduce persistence.

​

Acceptance criteria

​

AC-1 — Marconi delivery exposes a project-scoped queue enumerator

backend/app/marconi/delivery.py adds an async function iter_project_ws_queues(tenant_id: str, project_id: str) -> AsyncIterator[tuple[str, asyncio.Queue]] (or equivalent) that yields (session_id, queue) for every attached WS queue belonging to the project’s live sessions. Membership source (Texi G1 verdict 2026-06-04): project membership is resolved via the Marconi routing cache / SessionStore project session set — that’s the existing in-memory record of which sessions belong to which project. _session_queues is the attached-WS-queue map ONLY, not the membership oracle. The flow is:

1. Read the project’s session set (existing data structure).
2. Snapshot the matching session_id → Queue entries from _session_queues under _session_queues_lock.
3. Release the lock.
4. Yield / fan out from the snapshot.

Lock invariant (Texi G1 verdict): _session_queues_lock is held ONLY for the snapshot construction. It is never held across put_nowait, logging, exception handling, or any other fan-out work. This matches the targeted-send profile (get_ws_queue(session_id) holds the lock only for the read) and keeps fan-out off the lock’s critical path. Org/tenant scope NOT covered by this G1 verdict. The Marconi routing cache does not efficiently provide org/tenant membership today. iter_org_ws_queues and iter_tenant_ws_queues are deferred to a separate SPEC + indexing phase. This SPEC’s Phase 3 (AC-5/AC-6 deletions) is scoped to project broadcast only; the org/tenant publish_*_event helpers stay in source pending that follow-up SPEC.

​

AC-2 — Marconi delivery exposes a project-scope broadcast publisher

backend/app/marconi/delivery.py adds:

• async def publish_project_via_marconi(*, tenant_id, project_id, envelope) -> int — iterates iter_project_ws_queues, put_nowaits the envelope to each, returns the count successfully queued.

Per-queue exceptions (QueueFull / arbitrary) are caught and logged at DEBUG; loop continues. Return count = sessions reached, not sessions attempted. Org and tenant publishers are deferred per AC-1 note.

​

AC-3 — signal_service.py broadcast call site replaced (project-scope only)

The call at backend/app/services/signal_service.py:2325 (store.publish_project_event(...)) is replaced with publish_project_via_marconi(...). Project-scope only this PR. Org/tenant call sites (publish_org_event / publish_tenant_event) are NOT migrated in this PR per Constraint 3, AC-1 G1, AC-6, and Out of Scope — they are deferred to a follow-up SPEC and remain in source unchanged. Wire envelope shape matches the existing Redis pubsub payload so subscribers continue to deserialize without change.

​

AC-4 — routers/session_stream.py Redis subscribe branch deleted

After AC-3 lands, the Redis subscribe branch at backend/app/routers/session_stream.py:214-243 (subscribing tenant/org/project channels) has no remaining publisher. The branch is deleted; the Marconi-only path (line 174-212) becomes the only WS handler.

​

AC-5 — Flag retirement

PRISM_MARCONI_HOT_PATH_SEND is removed from backend/app/config.py (field deleted; env var no longer read anywhere in the codebase). Any remaining references to marconi_hot_path_send are removed. Diagnostic field disposition (Texi G1 verdict): backend/app/marconi/diagnostics.py:148’s hot_path_send field is deleted or renamed, not stubbed to return a constant. A retired env flag should not surface as a perpetually-true field in operator diagnostics — that’s misleading observability. Pick one:

• Delete the field entirely (preferred — the flag no longer exists).
• Rename to a more accurate post-retirement label (e.g. marconi_targeted_unconditional: true) if downstream consumers reference the key.

​

AC-6 — Helper deprecation / deletion

After AC-3, publish_project_event and the targeted _publish_targeted_event (now unused since SPEC-163 v0.7) helpers in backend/app/services/signal_service.py and backend/app/session_store/client.py have zero call sites. They are deleted from source. The session_store/client.py:18 docstring referencing them is updated. Pin test added so the symbols can’t reappear. publish_org_event / publish_tenant_event are NOT deleted by this SPEC — they stay in session_store/client.py pending the deferred org/tenant SPEC (per AC-1 G1 note). Their dead-code status is documented in their docstrings.

​

AC-7 — Backend test sweep

Full backend test sweep green (modulo the three pre-existing failures in test_spec140_process_architecture_registry + test_version_sot that are unrelated to signal). New tests cover:

• iter_project_ws_queues over a fixture registry returns the expected (session_id, queue) tuples
• publish_project_via_marconi fan-out delivers to all attached queues
• publish_project_via_marconi continues past QueueFull / arbitrary exceptions
• Pin test in test_spec163_marconi_queue_fallback.py extended: publish_project_event symbol does not exist on the deployed session_store/client.py module after this SPEC ships. (publish_agent_event / publish_session_event symbols are retired by the SPEC-163 v0.7 P1b sweep PR #742 + its follow-up cleanup, not by this SPEC. publish_org_event / publish_tenant_event remain present per Constraint 3 scope.)

​

Test scenarios

Headline scenario: send a Question signal with to_identity='*' to a project with 3 attached console_cc sessions (e.g. Lafonda + Candi + Porsche on mini3). Confirm:

1. Each recipient’s WS frame receive log shows the broadcast envelope.
2. No Redis publish call is logged during the send (verify via redis-cli MONITOR).

The pre-revision draft included a “drain or piggyback eventually surfaces the broadcast in pending_signals” check. That check is removed (Texi G1 verdict): broadcast is attached-WS-only best-effort. The no-persistence semantics in the “Out of scope” section explicitly exclude broadcast persistence and late-joiner replay. A receiver not attached to its WS at fan-out time misses the broadcast — that is the design, not a defect. Re-introducing a persistence-or-drain check would conflict with the SPEC’s own out-of-scope clause.

​

Implementation phases

• Phase 1 (this SPEC + AC-1, AC-2): Add the Marconi-side enumerators + broadcast publishers. Net-new code. No call-site changes.
• Phase 2 (AC-3): Replace signal_service.py:2325 with the new publisher. Add fan-out tests.
• Phase 3 (AC-4, AC-5, AC-6): Delete the Redis subscribe branch, the flag, the dead helpers. Pin tests close the door.

Phases ship as separate PRs to keep each diff reviewable. Phase 1 + 2 = no behavior change at the WS layer (subscribers still get the same wire envelope). Phase 3 = the structural cleanup.

​

Pre-PR gate

G1 — Marconi registry enumeration cost. Before AC-1 ships, confirm the in-memory session registry can be enumerated O(sessions_in_project) without holding a global lock that affects targeted-send latency. If the existing registry is keyed only by session_id, AC-1 may need an auxiliary project_id → session_id index built incrementally on attach/detach. Texi architecture review verdict required.

​

Out of scope (deferred)

• Marconi audit fan-out — persistence of broadcast envelopes for late-joiner replay. Tracked separately by SPEC-101 Tier 2/3.
• Cross-tenant broadcast — explicitly disallowed by tenant-isolation invariants; this SPEC does not weaken that.
• Broadcast obligations — to_identity='*' sends do not create signal_obligations rows today; this SPEC preserves that.

​

Source refs

• backend/app/services/signal_service.py:2325 — broadcast publish_project_event call site
• backend/app/session_store/client.py:607-682 — Redis publish helpers
• backend/app/routers/session_stream.py:214-243 — Redis subscribe branch
• backend/app/marconi/delivery.py — current targeted delivery + queue registry
• Journal #520 — Texi Plan #46 P1a inventory
• PR #742 (commit 4401d8d) — SPEC-163 v0.7 targeted Marconi-only sweep
• Plan #46 P1d — execution tracker

​

Supersession

Closes the gap left by SPEC-163 v0.7. Together with PR #742, this SPEC’s full execution retires Redis pubsub from signal-routing end to end.