The speech fixture classified as LIVE_HUMAN on its first 3s window and drifted
to MUSIC for every window after. I had validated only the first window and
reported the fixture as verified, which overstated it: any lab result resting
on that fixture — the hold-slayer scenarios above all — was proving less than
it appeared to.
The cause was one modelling error, not a tuning problem. `_detect_tonality`
looks for an autocorrelation peak above 0.5 in the 50-1000 Hz lag range, and
each syllable used a *constant* f0, which is perfectly periodic there. That
scored is_tonal=True, handing the music score a free 0.3 that speech could not
outrun — and the decision requires speech_score to strictly exceed
music_score, so ties went to music.
Real voices glide and jitter, so the periodicity never locks. The fundamental
now follows a per-syllable pitch contour (rise or fall, plus ~2% cycle-to-cycle
jitter), with the frequency integrated to phase rather than multiplied by t —
`2*pi*f*t` is only a chirp when f is the instantaneous rate, which it is not
once f0 itself moves. is_tonal is now False in every window.
Two smaller fixes fell out of that:
- Aspiration noise is high-passed rather than broadband. Flat noise puts
energy in every Goertzel bin, so the strongest DTMF row and column both
clear the detector's `total_power * 0.1` threshold and each syllable reads
as a keypress. A first-difference filter leaves the 697-1633 Hz bands
comparatively empty. The level is set for margin — spectral flatness lands
at ~0.46, mid-way through the 0.1-0.5 band, not on an edge.
- The music fixture gained two more harmonics and a recording-style noise
floor. Windows straddling a chord change had a momentarily sparse spectrum
and fell *below* the music score's 0.05 flatness floor, scoring as speech.
All three fixtures now classify correctly in 100% of windows (music 27/27,
speech 5/5, silence 2/2), and remain correct when the window is stepped by
half a window — a fixture that only works on aligned boundaries would still
be a trap in a live call, where the analysis window has no relationship to
where the audio began.
Confirmed on a real call through the lab: scenario 1003 now shows the whole
hold-slayer arc, speech -> sustained music -> speech, matching the dialplan.
tests/test_lab_fixtures.py guards this: it sweeps every window rather than
sampling the first, which is exactly what the original validation missed, and
checks the generator is byte-for-byte deterministic. It skips when the
fixtures have not been generated, since they are gitignored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An Asterisk instance that answers calls, plays an IVR, holds with music and
connects a "human", so the gateway has something real to dial that is not the
PSTN: no charges, no strangers, no E911 exposure.
Asterisk rather than Kamailio because the unproven risks are media risks.
Kamailio is a proxy — it routes signalling and answers nothing, so it would
forward the INVITE and find nobody home. Asterisk is a B2BUA: it answers,
plays prompts and collects DTMF, which is the hold-slayer scenario itself.
Kamailio remains the better model for trunk registration/digest auth later.
No application changes are needed to use it. SIP_TRUNK_HOST is just an
address, so the production code path runs unmodified — there is no test-only
branch anywhere in the gateway. It also means safety is structural: while the
trunk points at the lab there is no route to the PSTN at all, an absence of
route rather than a policy that could be misconfigured.
Nine scenarios (1001-1008 plus an echo test) cover the baseline call, the
IVR/DTMF path, hold-then-human, long hold, busy, no-answer, remote hangup and
silence.
The image ships no sound files, so sounds/generate.py synthesises three
fixtures from fixed seeds — byte-identical on every run, which is what makes
a classifier regression distinguishable from noise. Verified against
AudioClassifier: music→MUSIC 0.85, speech→LIVE_HUMAN 0.75,
silence→SILENCE 1.00. The speech formants deliberately avoid the DTMF bands;
the first version landed on a valid pair and classified as a keypress.
Anonymous inbound calls are refused, and endpoint matching is by source
address — Asterisk's default matches the From-header domain, which Hold
Slayer populates from its SIP bind address (0.0.0.0 on a wildcard bind).
Generated audio and the rendered per-host configs are gitignored: the former
is reproducible from a fixed seed, the latter carry a host-specific IP and
the lab password.
Known limit, documented in the README: MediaPipeline.create_tap is a stub, so
the classifier receives no audio on a live call. RTP flows and Asterisk plays
audio, but the tap is never fed — the fixture results above were measured by
feeding the classifier directly. This blocks scenarios 1002/1003/1004.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add comprehensive deployment and validation plan documenting a staged
bring-up approach that gates each layer on its predecessor and defers
PSTN testing until everything else is proven.
Update .env.example to reflect current configuration:
- Replace static API_TOKEN with Casdoor SSO + owner-minted PAT auth
- Add Rhema TTS settings with port-collision warning
- Add AI Receptionist settings for inbound calls
- Reconcile GATEWAY_SIP_PORT to 5060 and default SIP domain
Add comprehensive rule documentation for AI-assisted development covering
authentication surfaces, outbound-call safety invariants, and other project
conventions to guide Claude's understanding of critical system behaviors.
The UI never sent the bearer token, so with API_TOKEN set every data
call 401'd and /ws/events was rejected pre-accept (the 403s in the
uvicorn log). The API client now keeps the token in localStorage,
attaches Authorization to every request, prompts once on a 401 and
retries, and appends ?token= to the WebSocket connect — the page's
reconnect loop picks the token up after the first prompt.
require_token also accepts a ?token= query parameter (same convention
as the WebSocket) because <audio> elements fetching recordings can't
set headers; recordingUrl() rides the token there.
The dashboard header's status call moved to a new authenticated
GET /api/v1/status — its old source was the JSON root endpoint that
the dashboard itself replaced at /.
Two new auth tests (query-param accepted / wrong query-param 401);
dashboard rebuilt. Verified live: WS rejected without token and
connected with ?token=, status 200, ?token=wrong 401.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The SvelteKit build was always made for the root (no base path); it
now mounts at / — registered last so /api/v1, /ws, /health, and /mcp
match first — and the JSON root endpoint is gone (its info lives in
/health and gateway_status). REST routers move from /api/* to
/api/v1/*; /ws and /health stay put; /mcp/ unchanged. Dashboard API
client, tests, README, and docs updated; dashboard rebuilt (build/ is
gitignored).
Verified live: / serves the UI, /api/v1 answers 200/401, the old
/api paths 404, MCP still lists 15 tools.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The call-flow learner finally gets fed: exploration mode records its
IVR discoveries on the call (ActiveCall.exploration_steps) instead of
throwing them away, persistence stores them in the call record's
metadata, and the rebuilt learn_call_flow MCP tool turns a completed
exploration call into a stored flow via CallFlowLearner — correct
constructor (llm_client from get_llm, heuristic labels when the LLM
is unavailable), build for a new number, merge/refine when a flow
already exists. save_learned_flow/update_flow_from_model keep the
CallFlow↔row mapping in call_persistence.
Test gaps closed: tests/test_learner.py (discoveries→linked steps,
exploration persistence, learn-then-refine through the in-memory MCP
client, no-data and unknown-call answers) and tests/test_websocket.py
(4401 without token, trunk-status-then-replay on connect, per-call
stream filtering).
Docs aligned to code: README (15 tools incl. learn_call_flow, HTTP
not SSE, Python 3.12+, PostgreSQL+Alembic — no SQLite fallback, media
pipeline marked stub-mode until pjsua2 installed, Alembic and honest
/health checked off); docs/mcp-server.md rewritten against the actual
tool surface (hangup not end_call, real params, 3 real resources,
/mcp/ streamable HTTP + bearer auth); architecture/development/
configuration drift fixed.
pyproject: pruned never-imported deps (websockets, librosa,
soundfile, python-multipart).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Alembic replaces create_all as the schema authority: async env.py
against Base.metadata (CLI and in-app entry paths share it via
config.attributes["connection"]), an autogenerated baseline of the
create_all-era schema, and init_db now runs upgrade head — stamping
the baseline first on a pre-Alembic database so existing deployments
adopt cleanly. create_all remains for tests only.
Calls are durable from the start: CallManager gains an
on_call_created hook (wired to persist_call_on_create) that inserts
an in_progress CallRecord the moment a call is created;
persist_call_on_end finalizes that same row. A SIGKILL mid-call now
leaves an in_progress row instead of erasing the call from history
(verified live against the dev database).
One transcript representation: ActiveCall.transcript_chunks holds
TranscriptEntry (t_offset_ms, speaker, text) — add_transcript stamps
real offsets from connect time, receptionist passes speaker instead
of encoding it into "caller: ..." strings, persisted chunks carry
real seek offsets, and the dead CallRecord.transcript Text column is
dropped by migration. Device.is_online migrates String → Boolean
(with a USING cast for existing rows).
Model de-triplication: CallResponse/CallStatusResponse build via
from_call classmethods (one ActiveCall→response mapping);
DeviceStatus deleted — can_receive_call is a computed field on
Device and the list endpoint returns the domain model; all row↔dict
and row↔domain mapping now lives in call_persistence.py
(record_summary/record_detail/chunk_to_dict + device row functions).
New tests/test_data_layer.py: upgrade-head-matches-models,
pre-Alembic adoption, durable in_progress rows, end-without-create
fallback, transcript offsets, consolidated response models.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Engine mode is now explicit: USE_MOCK_SIP=true is the only way to get
the mock engine; an unconfigured trunk fails startup with guidance
instead of silently degrading. Root-caused why the engine always ran
mock: nested pydantic-settings never read .env (no env_file on the
sub-settings classes) — all 8 now declare it.
/health stops lying: reports engine mode (sippy|mock), a live DB
SELECT 1, trunk registration state with reason, and TTS/STT
availability from their last real request; "healthy" now requires
ready + db + sippy + registered trunk.
Error policy: leaf services (tts/transcription/llm_client) raise and
track availability; call-loop callers catch, publish EventType.ERROR
naming the failed service, and apply an explicit fallback. Persistence
writes get one bounded 3x exponential retry, then an ERROR log — no
more silent data loss.
Event bus: a full subscriber queue drops its oldest event (counted)
instead of silently evicting the subscription; subscribe(replay_last=N)
delivers the advertised history replay, used by /ws/events (25).
Receptionist correctness: a matched TAKE_MESSAGE rule beats the LLM;
voicemail polls for early hangup and stops/transcribes/hangs up in
finally; RecordingSession finally keeps its leg_ids so taps detach.
Dead code removed: models/contact.py + Contact table, dtmf_buffer,
transcribe_stream stub, SMS stub in notification.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gateway was the composition root, device registry, inbound-call
policy, and call-operations service in one class, with core↔services
circular imports papered over by function-local imports, wiring done
by assigning private attributes, and MCP tools duplicating REST query
logic against their own sessions.
Composition:
- main.py's lifespan now builds every service and wires them by
constructor/registration. gateway.from_config() is gone; core/ no
longer imports services/ anywhere — the cycle is dead.
- Inbound-call policy moved to ReceptionistService.on_inbound_call
(routing evaluation, reject/answer, screening dispatch); wired as
the engine's on_incoming_call by the lifespan. Receptionist deps
(tts/transcription/recording/routing) are constructor-injected —
no more gateway._tts reach-through or importing hold_slayer's
private _get_llm (now services.llm_client.get_llm, shared).
- Hold-slayer launch goes through a mode-handler registry
(register_mode_handler); the gateway no longer knows the service's
type. CallManager takes on_call_ended in its constructor.
- build_sip_engine() is a pure function taking explicit callbacks.
- api/routing.py uses the routing service from app.state via a
proper dependency instead of gateway._routing.
Shared data layer:
- db.session_scope() is the one session convention (get_db wraps it).
- services/call_persistence.py gains the query/write functions and
the single StoredCallFlow→CallFlow mapper; api/call_flows.py,
api/call_history.py, and the six DB-touching MCP tools are thin
wrappers over them — the two surfaces can't drift.
- legs_for_call() replaces the three private _call_legs scans
(gateway transfer/hangup, REST dtmf, MCP dtmf).
7 new tests (mode-handler launch, on_call_ended hook, receptionist
inbound answer/reject, call-flow CRUD round-trip and history routes
against real SQLite through the shared layer). aiosqlite added to dev
deps for that.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three thread domains were mutating shared dicts with no locks: Sippy's
ED thread wrote _legs/_registered_devices directly from SIP handlers,
the asyncio loop wrote them from make_call/hangup, and
run_in_executor(None, ...) had default-pool threads driving sippy UA
objects. AudioTap.feed() pushed into an asyncio.Queue (not
thread-safe) from the PJSUA2 thread.
New ownership rule, enforced structurally:
- The asyncio loop owns all app-visible state; the only mutator is the
new _on_engine_event funnel. Sippy handlers extract plain strings on
the ED thread and post via run_coroutine_threadsafe.
- The ED thread owns sippy objects plus _ed_ua_to_leg/_ed_leg_to_ua;
loop-side commands (INVITE/BYE/DTMF/trunk register) hop over via
ED2.callFromThread. UA references no longer live on SipCallLeg.
- AudioTap captures its loop and feed() hops via call_soon_threadsafe.
- Fix ED import: installed sippy 2.x exposes ED2, not ED — the old
import could never start the event loop.
Also:
- Wire the never-connected on_leg_state_change callback: outbound
ringing/connected/terminated now reaches CallManager; a call ends
when its last leg terminates (transfers keep it alive). Adds
CallManager.unmap_leg/legs_for_call.
- AudioClassifier.classify(): async entry that runs the FFT work in
asyncio.to_thread and updates history on the loop — all four
hold_slayer call sites now route through it, fixing both the
loop-blocking and the 2-of-4 history gap. DTMF Goertzel loop
replaced by the equivalent vectorized DFT-bin power.
- Task hygiene: gateway.spawn() tracks hold-slayer/receptionist tasks
and stop() cancels them; recording safety-timeout task is retained
and cancelled on stop_recording; engine tracks incoming-call
dispatch tasks.
10 new tests: funnel events from a foreign thread, auto-answer
fallback, AudioTap cross-thread feed, classifier history, leg-state →
call status (including no stomping of ON_HOLD), stop() cancellation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The MCP server was created but never mounted — no client could reach
it. Mount it at /mcp/ over streamable HTTP with a combined lifespan,
resolving the gateway lazily so mounting happens at app construction.
Security and safety for the agent surface:
- One static API_TOKEN (SecretStr) enforced across REST (dependency),
WebSocket (query param/header before accept), and MCP
(StaticTokenVerifier). Startup refuses tokenless non-loopback binds.
- Emergency numbers (911/9911/112) always refused on make_call, plus a
MAX_CONCURRENT_CALLS cap; ValueError surfaces as 400/ToolError.
- Safe defaults: debug off, no credential in default DATABASE_URL,
SIP/LLM/TTS secrets as SecretStr.
Cleanups:
- Delete broken learn_call_flow tool (wrong ctor args, nonexistent
method) and the never-fed CallAnalytics service; keep
call_flow_learner for proper wiring later.
- Trim dial_plan to what is actually used (emergency guard, extension
allocation); delete the unreferenced matcher/normaliser.
- Register call_history before calls so /api/calls/history is no
longer shadowed by /api/calls/{call_id}.
- fastmcp pinned >=3.0 (http_app + StaticTokenVerifier).
New tests: MCP in-memory client (tool surface, lazy gateway, emergency
refusal, call cap) and API security (401 paths, route order, mount).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds read-only access to persisted call records for the dashboard
and implements a client for the Rhema text-to-speech service.
- api/call_history.py: New router providing paged call lists
and detailed call records with transcript metadata.
- services/tts.py: Async client for OpenAI-compatible TTS
endpoints (Rhema/Kokoro) used for call-flow steps.
Complete project scaffolding and core implementation of an AI-powered
telephony system that calls companies, navigates IVR menus, waits on
hold, and transfers to the user when a human answers.
Key components:
- FastAPI server with REST API, WebSocket, and MCP (SSE) interfaces
- SIP/VoIP call management via PJSUA2 with RTP audio streaming
- LLM-powered IVR navigation using OpenAI/Anthropic with tool calling
- Hold detection service combining audio analysis and silence detection
- Real-time STT (Whisper/Deepgram) and TTS (OpenAI/Piper) pipelines
- Call recording with per-channel and mixed audio capture
- Event bus (asyncio pub/sub) for real-time client updates
- Web dashboard with live call monitoring
- SQLite persistence via SQLAlchemy with call history and analytics
- Notification support (email, SMS, webhook, desktop)
- Docker Compose deployment with Opal VoIP and Opal Media containers
- Comprehensive test suite with unit, integration, and E2E tests
- Simplified .gitignore and full project documentation in README
