Signal Wake Reliability — Joint Diagnostic Plan

​

1. Problem & framing

Backend delivered_at and publish_path prove transport progress, not model-visible action.

1. Durability — events live in an in-process 80-entry ring; lost on shim restart, not queryable across surfaces.
2. Model-acted signal — no explicit record that the agent’s model context actually consumed the doorbell beyond drain.
3. Cross-process query verb — prism_runtime_diagnostics is per-runtime; no joined view per signal_id.

​

2. Scope

• End-to-end observability of stages 1–9 (signal_created → backend publish → backend WS frame → MCP stream receipt → strategy delivery → surface wake → model turn → drain → reply).
• Closing the transport-vs-model gap with a model-acted ACK protocol.
• A paced validation matrix between Donna and Texi before fleet rollout.

• Always-on per-persona daemon wake path (SPEC-070 — separate effort, lane: Texi).
• Surface-adapter rendering bugs (e.g. PeerJoined Unknown — postmortem 74b62026, Donna 5-line patch lane).
• Doorbell durability across prism_wrap → prism_start cycle (Porsche open question).

​

2.1 Texi Close-Out Status

• Durable trace_id on signal_queue, included in signal wire payloads and pending-signal drain payloads.
• Durable signal_trace_events table with ordered per-trace stage events.
• Backend trace endpoints for event recording, model ACK, and trace query.
• MCP verbs prism_signal_trace and prism_signal_ack.
• MCP stream/runtime propagation of trace_id through frame receipt, adapter-delivery diagnostics, and visible wake prompts.
• Focused smoke coverage for Codex publish path, trace_id wire payload, and controller-registration lock helper.

• npm --prefix mcp-node run build
• PYTHONPATH=backend backend/.venv/bin/python -m pytest backend/tests/test_signal_publish_path.py backend/tests/test_spec038_imports.py
• PYTHONPATH=backend backend/.venv/bin/python -m py_compile ...

• Alembic migration 035_signal_wake_trace.py must be applied to the live backend database.
• Backend and MCP runtimes must be restarted so the new endpoints and verbs are available to live agents.
• Formal V1-V5 must run twice after deployment. Local implementation is ready, but this session could not deploy to server1.home.lan because SSH rejected the available credentials and local Postgres on localhost:5433 was not running.

​

3. Lanes

​

4. First Pass — Already Shipped (Texi, uncommitted local)

• mcp-node/src/runtime_diagnostics.ts (217 lines) — in-process state for stream open/close/error, signal frame receipt, strategy delivery result, surface wake result, tool calls, drains, signal sends.
• mcp-node/src/bootstrap/idle_tickler.ts (73 lines) — 4-minute stale threshold, registered+not-wrapped prerequisite, active-turn suppression.
• prism_runtime_diagnostics MCP verb (verbs/coordination.ts) — query interface for the above.
• Strategy instrumentation (channels_push.ts, app_server_inject.ts) — per-stage event emission.
• Codex app-server turn/start + turn/steer adapter; Claude Code maintenance channel tick.

​

5. Joint Additions — Donna Lane

​

5.1 trace_id in signal frame

• Mint a UUID trace_id at signal_created (backend, signal_service.py).
• Propagate through every stage: persisted on the row, included in every WS frame, in every MCP stream event, in every surface adapter event, in the model-acted ACK envelope.
• One trace_id ties every stage event to a single signal across surfaces and processes.

​

5.2 prism_signal_trace verb

​

5.3 Model-acted ACK protocol

• Surface adapter, on doorbell delivery to the model context, records delivered_to_surface_at.
• The next model turn that observes the doorbell SHOULD emit prism_signal_ack(trace_id) as its first verb call. Records model_acted_at.
• Gap (model_acted_at − delivered_to_surface_at) is the transport-to-model latency — the metric the retro called out as currently unmeasurable.
• Failure mode: doorbell with no model ACK within window N → flagged in prism_runtime_diagnostics and surfaces in next bootstrap rules_reminders.

​

6. Validation Matrix — Joint

​

7. Surface-Specific Notes

• Codex: app-server/turn/start and app-server/turn/steer are the wake primitives. Idle tickler is sufficient for ≤4-min idle windows.
• Claude Code: channel notification is primary; only fallback option for REPL/terminal poke is the stricter-flag REPL nudge — adds risk of input-stuffing collisions, kept as Phase 2 if V2 fails.

​

8. Open Questions for Frank

1. Approve scope? — does the trace_id + prism_signal_trace + model-acted ACK trio match what you want, or do you want narrower (just the verb) or broader (also doorbell durability across wrap/start)?
2. Validation cadence — run V1–V5 sequentially in one session, or spread over multiple sessions to capture realistic background/idle conditions?
3. Where do trace events persist? — Postgres only (durable, queryable, Plan-#10-aligned with Postgres-as-long-term), or also Redis ring-buffer for fast in-process query? Recommend PG-only for v1; Redis if perf matters later (per feedback_optimize_later.md).
4. rules_reminders surfacing — should missed-ACK trigger a rules_reminders entry on next bootstrap, or is that too noisy? Recommend yes, with a per-trace cooldown.

​

9. Sequencing

Step 0:  Frank approves this plan markdown                                                ← gate
Step 1:  Texi commits first-pass + opens PR; Donna reviews + smokes (no edits)
Step 2:  Donna implements §5.1 trace_id-in-frame (backend); Texi reviews
Step 3:  Donna implements §5.2 prism_signal_trace verb; Texi reviews
Step 4:  Joint implementation §5.3 model-acted ACK (Donna backend, Texi adapter); cross-review
Step 5:  Run V1–V5 validation matrix twice; record traces in retro
Step 6:  Fleet rollout (default-on for runtime_diagnostics; ACK protocol recommended-not-required initially)
Step 7:  Postmortem + retro after 1 week of fleet data

​

10. Memory & Postmortem Hooks

• feedback_eliminate_failures_improve_perf.md — every new verb gets structured-failure returns + duration_s tracking.
• feedback_postmortem_on_every_error.md — any V1–V5 failure files a postmortem inline.
• feedback_completion_means_deployed.md — Steps 1–6 are not “done” until merged + deployed + smoke green.
• project_signal_isolation_multitenant.md — prism_signal_trace must enforce the same membership-only authorization as prism_signals_pending (no cross-tenant trace exposure).