Robert Helewka
e2051f7486
Started as the SIP_ENGINE row flagged in the last commit. Cross-checking the tables against config.py mechanically (rather than by eye) turned up more, including two entries that were actively wrong. Corrections: - GATEWAY_RTP_PORT_MIN/MAX and GATEWAY_HOST are documented in configuration.md but do not exist — no code reads them and they are absent from .env.example. Setting them today does nothing. Replaced with the real GATEWAY_SIP_ fields (host/port/domain). - GATEWAY_SIP_PORT was documented as 5080 in two places; the code and .env.example both say 5060. - DATABASE_URL was documented with a SQLite default. There is none, and startup exits if it is unset. Additions — every env var the models accept is now documented somewhere (verified bidirectionally: nothing in the models undocumented, nothing documented that the models reject): - Server section: HOST, PORT, DEBUG, LOG_LEVEL, LOG_FORMAT - Safety section: MAX_CONCURRENT_CALLS, USE_MOCK_SIP, SIP_ENGINE - Receptionist section (configuration.md had none, though seven vars exist) Structural staleness, from the PR #8 media-plane work: - core/pjsua_engine.py was absent from the component list and file tree; so were dial_plan.py (the emergency guard) and sip_engine.py. - architecture.md's banner still read "media plane in transition". The engine landed; it is now two selectable engines with the audio consequence stated. - Tech Stack described "single-process async architecture" — the simplification CLAUDE.md explicitly calls out. Now points at the threading model, since there are three execution contexts. - The Asterisk lab shipped in PR #8 with its own README but nothing linked to it. Linked from the test section and both doc indexes. - CLAUDE.md's "no structured JSON logging" gap is closed; test count was 146 across 16 files, now 189 across 19. The other listed gaps (no /metrics, no rate limiting, no health-probe log filter) were re-verified and still hold. Deliberately not hardcoding a test count in the README — that is the same staleness this commit is clearing up. All internal links and anchors verified to resolve; 189 tests pass; lint unchanged at its 216 baseline. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
256 lines
14 KiB
Markdown
256 lines
14 KiB
Markdown
# Architecture
|
|
|
|
Hold Slayer is a single-process async Python application built on FastAPI. It
|
|
acts as an intelligent B2BUA (Back-to-Back User Agent) sitting between your SIP
|
|
trunk (PSTN access) and your desk phone/softphone.
|
|
|
|
> **Two SIP engines, selected by `SIP_ENGINE`.** `sippy` (the default) signals
|
|
> only — PJSUA2 will not surface an RTP stream for a dialog it does not own, so
|
|
> **no audio reaches the classifier** on that path. `pjsua2`
|
|
> (`core/pjsua_engine.py`) places the call itself and is the only mode where
|
|
> audio reaches the classifier; it is opt-in while being proven against the lab.
|
|
> Read [Media plane: why PJSUA2 places the call](#media-plane-why-pjsua2-places-the-call)
|
|
> before changing anything in `core/`.
|
|
|
|
## System Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ FastAPI Server │
|
|
│ │
|
|
│ ┌──────────┐ ┌──────────┐ ┌───────────┐ ┌──────────────┐ │
|
|
│ │ REST API │ │WebSocket │ │MCP Server │ │ Dashboard │ │
|
|
│ │ /api/v1/*│ │ /ws/* │ │ (HTTP) │ │ / │ │
|
|
│ └────┬─────┘ └────┬─────┘ └─────┬─────┘ └──────────────┘ │
|
|
│ │ │ │ │
|
|
│ ┌────┴──────────────┴──────────────┴────┐ │
|
|
│ │ Event Bus │ │
|
|
│ │ (asyncio Queue pub/sub per client) │ │
|
|
│ └────┬──────────────┬──────────────┬────┘ │
|
|
│ │ │ │ │
|
|
│ ┌────┴─────┐ ┌─────┴─────┐ ┌────┴──────────┐ │
|
|
│ │ Call │ │ Hold │ │ Services │ │
|
|
│ │ Manager │ │ Slayer │ │ (LLM, STT, │ │
|
|
│ │ │ │ │ │ Recording, │ │
|
|
│ │ │ │ │ │ Analytics, │ │
|
|
│ │ │ │ │ │ Notify) │ │
|
|
│ └────┬─────┘ └─────┬─────┘ └──────────────┘ │
|
|
│ │ │ │
|
|
│ ┌────┴──────────────┴───────────────────┐ │
|
|
│ │ SIP Engine │ │
|
|
│ │ signalling + call control │ │
|
|
│ └────┬──────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ┌────┴──────────────────────────────────┐ │
|
|
│ │ Media Pipeline (PJSUA2) │ │
|
|
│ │ RTP, conference bridge, taps, record │ │
|
|
│ └────┬──────────────────────────────────┘ │
|
|
│ │ │
|
|
└───────┼─────────────────────────────────────────────────────────┘
|
|
│
|
|
┌────┴────┐
|
|
│SIP Trunk│ ──→ PSTN
|
|
└─────────┘
|
|
```
|
|
|
|
## Component Overview
|
|
|
|
### Presentation Layer
|
|
|
|
| Component | File | Protocol | Purpose |
|
|
|-----------|------|----------|---------|
|
|
| REST API | `api/calls.py`, `api/call_flows.py`, `api/devices.py` | HTTP | Call management, CRUD, configuration |
|
|
| WebSocket | `api/websocket.py` | WS | Real-time event streaming to clients |
|
|
| MCP Server | `mcp_server/server.py` | Streamable HTTP at `/mcp/` | AI assistant tool integration |
|
|
|
|
### Orchestration Layer
|
|
|
|
| Component | File | Purpose |
|
|
|-----------|------|---------|
|
|
| Gateway | `core/gateway.py` | Top-level orchestrator — owns all services, routes calls |
|
|
| Call Manager | `core/call_manager.py` | Active call state, lifecycle, transcript tracking |
|
|
| Event Bus | `core/event_bus.py` | Async pub/sub connecting everything together |
|
|
|
|
### Intelligence Layer
|
|
|
|
| Component | File | Purpose |
|
|
|-----------|------|---------|
|
|
| Hold Slayer | `services/hold_slayer.py` | IVR navigation, hold monitoring, human detection |
|
|
| Audio Classifier | `services/audio_classifier.py` | Real-time waveform analysis (music/speech/DTMF/silence) |
|
|
| LLM Client | `services/llm_client.py` | OpenAI-compatible LLM for IVR menu decisions |
|
|
| Transcription | `services/transcription.py` | Speaches/Whisper STT for live audio |
|
|
| Call Flow Learner | `services/call_flow_learner.py` | Builds reusable IVR trees from exploration data |
|
|
|
|
### Infrastructure Layer
|
|
|
|
| Component | File | Purpose |
|
|
|-----------|------|---------|
|
|
| Sippy Engine | `core/sippy_engine.py` | SIP signalling (INVITE, BYE, REGISTER, DTMF) |
|
|
| Media Pipeline | `core/media_pipeline.py` | PJSUA2 RTP media, conference bridge, taps, recording |
|
|
| Recording | `services/recording.py` | WAV file management and storage |
|
|
| Analytics | `services/call_analytics.py` | Call metrics, hold time stats, trends |
|
|
| Notifications | `services/notification.py` | WebSocket + SMS alerts |
|
|
| Database | `db/database.py` | SQLAlchemy async (PostgreSQL, Alembic migrations) |
|
|
|
|
## Data Flow — Hold Slayer Call
|
|
|
|
```
|
|
1. User Request
|
|
POST /api/v1/calls/hold-slayer { number, intent, call_flow_id }
|
|
│
|
|
2. Gateway.make_call()
|
|
├── is_emergency_number() → REFUSE 911/112 (before anything else)
|
|
├── concurrency cap check → refuse past max_concurrent_calls
|
|
├── CallManager.create_call() → track state
|
|
└── sip_engine.make_call() → place the call, media follows
|
|
│
|
|
3. HoldSlayer.run_with_flow() or run_exploration()
|
|
├── AudioClassifier.classify() → analyze 3s audio windows
|
|
│ ├── silence? → wait
|
|
│ ├── ringing? → wait
|
|
│ ├── DTMF? → detect tones
|
|
│ ├── music? → HOLD_DETECTED event
|
|
│ └── speech? → transcribe + decide
|
|
│
|
|
├── TranscriptionService.transcribe() → STT on speech audio
|
|
│
|
|
├── LLMClient.analyze_ivr_menu() → pick menu option (fallback)
|
|
│ └── sip_engine.send_dtmf() → press the button
|
|
│
|
|
└── detect_hold_to_human_transition()
|
|
└── HUMAN_DETECTED! → transfer
|
|
│
|
|
4. Transfer
|
|
├── SippyEngine.bridge_calls() → join the two call legs
|
|
├── MediaPipeline.bridge_streams() → bridge RTP in the conf bridge
|
|
├── EventBus.publish(TRANSFER_STARTED)
|
|
└── NotificationService → "Pick up your phone!"
|
|
│
|
|
5. Real-Time Updates (throughout)
|
|
EventBus.publish() → WebSocket clients
|
|
→ MCP server resources
|
|
→ Notification service
|
|
→ Analytics tracking
|
|
```
|
|
|
|
The emergency guard and the concurrency cap are the first two steps of
|
|
`make_call` for a reason, and their order is load-bearing — see
|
|
[.claude/rules/call-safety.md](../.claude/rules/call-safety.md).
|
|
|
|
## Threading Model
|
|
|
|
The README's "single-process async" is a simplification. There are **three**
|
|
execution contexts, and the boundaries between them are the highest-leverage
|
|
invariant in the codebase.
|
|
|
|
```
|
|
asyncio loop (main thread) Sippy ED thread PJSUA2 worker threads
|
|
├── FastAPI (uvicorn) └── ED2 dispatcher └── media / RTP
|
|
├── EventBus ├── SIP signalling └── onFrameReceived
|
|
├── CallManager ├── UA objects
|
|
├── HoldSlayer └── DTMF relay
|
|
├── AudioClassifier
|
|
├── TranscriptionService
|
|
├── LLMClient
|
|
├── NotificationService
|
|
└── RecordingService
|
|
```
|
|
|
|
**Crossing the boundaries — one funnel each way:**
|
|
|
|
| Direction | Mechanism | Notes |
|
|
|---|---|---|
|
|
| Sippy ED → loop | `_post_from_ed` → `asyncio.run_coroutine_threadsafe` → `_on_engine_event` | The single funnel where Sippy-thread events mutate loop state |
|
|
| loop → Sippy ED | `_run_on_sippy` → `ED2.callFromThread` | Anything touching a Sippy UA object |
|
|
| PJSUA2 worker → loop | `AudioTap.feed` → `loop.call_soon_threadsafe` | The **only** thing a PJSUA2 callback may touch |
|
|
|
|
`onFrameReceived` runs on a PJSUA2 worker thread every 20 ms. It must call
|
|
nothing but `AudioTap.feed`; reaching into pipeline state, the event bus, or a
|
|
Sippy object from there is a data race. An exception escaping into PJSUA2's C++
|
|
callback tears down the worker thread and silently kills media for every call,
|
|
which is why the capture port catches and logs once rather than per frame.
|
|
|
|
Full detail: [.claude/rules/concurrency-threads.md](../.claude/rules/concurrency-threads.md).
|
|
|
|
## Design Decisions
|
|
|
|
### Media plane: why PJSUA2 places the call
|
|
|
|
The original split was *Sippy signals, PJSUA2 carries media*. It does not work,
|
|
for a reason that is not obvious until you try it:
|
|
|
|
**PJSUA2 exposes no standalone RTP media object.** Every `AudioMedia` subclass
|
|
in the Python bindings is a file player, recorder, tone generator, or capture
|
|
port. RTP is reachable only through `pj.Call.getAudioMedia()`, after
|
|
`onCallMediaState` fires on a dialog **PJSUA2 itself owns**. There is no
|
|
"give me an AudioMedia for this remote host:port" API to call.
|
|
|
|
So a design where Sippy owns the dialog can never obtain a media stream from
|
|
PJSUA2. `MediaPipeline.add_remote_stream()` is not unfinished work — it is a
|
|
function that cannot be written against this API. The consequence is that audio
|
|
never reaches the classifier: `create_tap` builds a valid capture port with
|
|
nothing to attach it to.
|
|
|
|
**The resolution: PJSUA2 places the call; Sippy keeps every other role.**
|
|
|
|
| Concern | Owner |
|
|
|---|---|
|
|
| Emergency guard, concurrency cap | `gateway.make_call` — unchanged, still first |
|
|
| Trunk registration | PJSUA2 `Account` |
|
|
| Outbound INVITE / answer / hangup | PJSUA2 `Call` |
|
|
| RTP, conference bridge, taps, recording | PJSUA2 media |
|
|
| DTMF | PJSUA2 `Call.dialDtmf` (RFC 2833) |
|
|
| Device registration, routing, leg bridging | Sippy / gateway |
|
|
| Inbound call dispatch | PJSUA2 `Account.onIncomingCall` |
|
|
|
|
Sippy remains the SBC-shaped layer — it is where device registrations, routing
|
|
decisions and B2BUA leg-joining live. What moves is the raw dialog for a trunk
|
|
call, because owning the dialog is the price of owning the media.
|
|
|
|
Alternatives considered and rejected:
|
|
|
|
- **Terminate RTP ourselves** (aiortc or raw sockets) and feed PCM into
|
|
`AudioTap` directly, keeping Sippy on the wire. Preserves the split, but
|
|
means owning jitter buffering, packet loss concealment and ulaw/alaw
|
|
transcoding — precisely the work PJSUA2 exists to do.
|
|
- **A loopback `pj.Call` mirroring each real leg**, so PJSUA2 has a dialog it
|
|
owns. Avoids touching call placement, but adds a phantom call per real call
|
|
and the SDP juggling is fragile.
|
|
|
|
> **Safety note for this refactor:** `is_emergency_number()` stays the first
|
|
> check in `gateway.make_call`, above the concurrency cap and above any SIP
|
|
> action, regardless of which library dials. A new outbound path that reaches
|
|
> the SIP layer without passing that guard is a serious regression even if
|
|
> every test passes.
|
|
|
|
### Why asyncio Queue-based EventBus?
|
|
|
|
- **Single process** — no need for Redis/RabbitMQ cross-process messaging
|
|
- **Zero dependencies** — pure asyncio, no external services to deploy
|
|
- **Per-subscriber queues** — slow consumers don't block fast publishers
|
|
- **Dead subscriber cleanup** — full queues are automatically removed
|
|
- **Event history** — late joiners can catch up on recent events
|
|
|
|
If scaling to multiple gateway processes becomes necessary, the EventBus
|
|
interface can be backed by Redis pub/sub without changing consumers.
|
|
|
|
### Why OpenAI-compatible LLM API?
|
|
|
|
The LLM client uses raw HTTP (httpx) against any OpenAI-compatible endpoint.
|
|
This means:
|
|
|
|
- **Ollama** (local, free) — `http://localhost:11434/v1`
|
|
- **LM Studio** (local, free) — `http://localhost:1234/v1`
|
|
- **vLLM** (local, fast) — `http://localhost:8000/v1`
|
|
- **OpenAI** (cloud) — `https://api.openai.com/v1`
|
|
|
|
No SDK dependency. No vendor lock-in. Switch models by changing one env var.
|
|
|
|
## Testing against a fake PSTN
|
|
|
|
`tests/lab/` runs an Asterisk instance that answers calls, plays an IVR, holds
|
|
with music and connects a "human" — so the gateway has something real to dial
|
|
that is not the PSTN. `SIP_TRUNK_HOST` is just an address, so the production
|
|
code path runs unmodified; while it points at the lab there is no route to the
|
|
PSTN at all. See [tests/lab/README.md](../tests/lab/README.md).
|