Stage 4: honest health, explicit error policy, event-bus integrity #4

r · 2026-07-10T11:02:20Z

r commented

2026-07-10 11:02:20 +00:00

Stage 4 of the architecture-review roadmap (stacked on #3). The theme: the system stops pretending — no silent mock fallback, no "healthy while doing nothing", no swallowed service errors, no silently evicted subscribers.

Explicit engine mode (F7)

USE_MOCK_SIP=true is now the only way to get the mock engine. An unconfigured/placeholder trunk host without it makes build_sip_engine raise and startup exit with guidance — no more silent degradation to mock.
Root cause found for "engine always ran mock": nested pydantic-settings built via default_factory only read os.environ, never .env — none of the 8 sub-settings classes declared env_file. All 8 now do (env_file=".env", extra="ignore"), so SIP_TRUNK_HOST etc. actually load.

Honest /health

Reports engine: sippy|mock, a live SELECT 1 DB ping, trunk registration state with a reason, and TTS/STT availability from their last real request. healthy now requires ready + db ok + sippy engine + registered trunk; anything less is degraded. Verified live: mock boot reports degraded / engine: mock / database: ok.

Error policy (F7)

Leaf services (tts, transcription, llm_client) raise on failure and track available; returning ""/False on error is gone.
Call-loop callers (hold_slayer, receptionist) catch, publish EventType.ERROR naming the failed service, and apply an explicit per-call fallback — a down STT now surfaces as an error event, not "the AI deciding badly".
Persistence writes (persist_call_on_end, _persist_recording) get one bounded 3× exponential retry, then an ERROR log naming what was lost.
_get_llm → shared get_llm() singleton (kills receptionist's private cross-service import).

Event-bus integrity (F6)

A full subscriber queue drops its oldest event (per-subscription dropped counter, escalating log) instead of silently evicting a live WS subscriber whose socket then stays open forever, mute.
subscribe(replay_last=N) delivers the docstring's advertised history replay, respecting type filters; /ws/events uses replay_last=25.

Receptionist correctness batch (F9)

A matched TAKE_MESSAGE rule now beats the LLM (previously the LLM could override an explicit rule).
Voicemail is no longer a flat 90 s sleep: 1 s polls detect caller hangup early, and stop/transcribe/publish/hangup all run in finally.
RecordingSession finally keeps its leg_ids argument, so stopping a recording actually detaches its audio taps.
Recording timeout task is cancelled on manual stop (with a guard against self-cancellation when the timeout itself fires).

Dead code removed

models/contact.py + the Contact ORM table, dtmf_buffer (captured, never read), transcribe_stream TODO stub, and the fully stubbed SMS path in notification.py.

Tests

New tests/test_health_policy.py: mock-only-when-asked, unconfigured-trunk-refuses, overflow-drops-oldest-keeps-subscription, replay_last seeding, replay-respects-filter. Updated receptionist/services/concurrency tests for the new semantics. 127 passed + the 2 pre-existing failures on main (test_complex_tone_as_music, test_trunk_status).

Live verification: booted with USE_MOCK_SIP=true — honest degraded health as above, 401 without token / 200 with, clean shutdown with no destroyed-task warnings.

🤖 Generated with Claude Code

Stage 4 of the architecture-review roadmap (stacked on #3). The theme: the system stops pretending — no silent mock fallback, no "healthy while doing nothing", no swallowed service errors, no silently evicted subscribers. ## Explicit engine mode (F7) - `USE_MOCK_SIP=true` is now the **only** way to get the mock engine. An unconfigured/placeholder trunk host without it makes `build_sip_engine` raise and startup exit with guidance — no more silent degradation to mock. - **Root cause found for "engine always ran mock"**: nested pydantic-settings built via `default_factory` only read `os.environ`, never `.env` — none of the 8 sub-settings classes declared `env_file`. All 8 now do (`env_file=".env", extra="ignore"`), so `SIP_TRUNK_HOST` etc. actually load. ## Honest /health Reports `engine: sippy|mock`, a live `SELECT 1` DB ping, trunk registration state with a reason, and TTS/STT availability from their last real request. `healthy` now requires ready + db ok + sippy engine + registered trunk; anything less is `degraded`. Verified live: mock boot reports `degraded` / `engine: mock` / `database: ok`. ## Error policy (F7) - Leaf services (`tts`, `transcription`, `llm_client`) **raise** on failure and track `available`; returning `""`/`False` on error is gone. - Call-loop callers (`hold_slayer`, `receptionist`) catch, publish `EventType.ERROR` naming the failed service, and apply an explicit per-call fallback — a down STT now surfaces as an error event, not "the AI deciding badly". - Persistence writes (`persist_call_on_end`, `_persist_recording`) get one bounded 3× exponential retry, then an ERROR log naming what was lost. - `_get_llm` → shared `get_llm()` singleton (kills receptionist's private cross-service import). ## Event-bus integrity (F6) - A full subscriber queue **drops its oldest event** (per-subscription `dropped` counter, escalating log) instead of silently evicting a live WS subscriber whose socket then stays open forever, mute. - `subscribe(replay_last=N)` delivers the docstring's advertised history replay, respecting type filters; `/ws/events` uses `replay_last=25`. ## Receptionist correctness batch (F9) - A matched `TAKE_MESSAGE` rule now beats the LLM (previously the LLM could override an explicit rule). - Voicemail is no longer a flat 90 s sleep: 1 s polls detect caller hangup early, and stop/transcribe/publish/hangup all run in `finally`. - `RecordingSession` finally keeps its `leg_ids` argument, so stopping a recording actually detaches its audio taps. - Recording timeout task is cancelled on manual stop (with a guard against self-cancellation when the timeout itself fires). ## Dead code removed `models/contact.py` + the Contact ORM table, `dtmf_buffer` (captured, never read), `transcribe_stream` TODO stub, and the fully stubbed SMS path in `notification.py`. ## Tests New `tests/test_health_policy.py`: mock-only-when-asked, unconfigured-trunk-refuses, overflow-drops-oldest-keeps-subscription, replay_last seeding, replay-respects-filter. Updated receptionist/services/concurrency tests for the new semantics. **127 passed** + the 2 pre-existing failures on main (`test_complex_tone_as_music`, `test_trunk_status`). Live verification: booted with `USE_MOCK_SIP=true` — honest degraded health as above, 401 without token / 200 with, clean shutdown with no destroyed-task warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

r added 1 commit 2026-07-10 11:02:21 +00:00

Stage 4: honest health, explicit error policy, event-bus integrity 4048ce1db6

Engine mode is now explicit: USE_MOCK_SIP=true is the only way to get
the mock engine; an unconfigured trunk fails startup with guidance
instead of silently degrading. Root-caused why the engine always ran
mock: nested pydantic-settings never read .env (no env_file on the
sub-settings classes) — all 8 now declare it.

/health stops lying: reports engine mode (sippy|mock), a live DB
SELECT 1, trunk registration state with reason, and TTS/STT
availability from their last real request; "healthy" now requires
ready + db + sippy + registered trunk.

Error policy: leaf services (tts/transcription/llm_client) raise and
track availability; call-loop callers catch, publish EventType.ERROR
naming the failed service, and apply an explicit fallback. Persistence
writes get one bounded 3x exponential retry, then an ERROR log — no
more silent data loss.

Event bus: a full subscriber queue drops its oldest event (counted)
instead of silently evicting the subscription; subscribe(replay_last=N)
delivers the advertised history replay, used by /ws/events (25).

Receptionist correctness: a matched TAKE_MESSAGE rule beats the LLM;
voicemail polls for early hangup and stops/transcribes/hangs up in
finally; RecordingSession finally keeps its leg_ids so taps detach.

Dead code removed: models/contact.py + Contact table, dtmf_buffer,
transcribe_stream stub, SMS stub in notification.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

r referenced this pull request

2026-07-10 11:43:24 +00:00

Stage 5: Alembic migrations, durable call rows, one transcript truth #5

r referenced this pull request

2026-07-10 17:45:58 +00:00

Stage 6: learn_call_flow rebuilt on the learner, docs truth sweep #6

r merged commit 20faffca50 into feature/stage3-structure

2026-07-10 17:48:52 +00:00

r referenced this issue from a commit

2026-07-10 17:48:54 +00:00

Merge pull request 'Stage 4: honest health, explicit error policy, event-bus integrity' (#4) from feature/stage4-honest-health into feature/stage3-structure

Sign in to join this conversation.