Files
hold-slayer/tests/lab
Robert Helewka 92c45e9c4d fix(lab): make the speech fixture actually classify as speech
The speech fixture classified as LIVE_HUMAN on its first 3s window and drifted
to MUSIC for every window after. I had validated only the first window and
reported the fixture as verified, which overstated it: any lab result resting
on that fixture — the hold-slayer scenarios above all — was proving less than
it appeared to.

The cause was one modelling error, not a tuning problem. `_detect_tonality`
looks for an autocorrelation peak above 0.5 in the 50-1000 Hz lag range, and
each syllable used a *constant* f0, which is perfectly periodic there. That
scored is_tonal=True, handing the music score a free 0.3 that speech could not
outrun — and the decision requires speech_score to strictly exceed
music_score, so ties went to music.

Real voices glide and jitter, so the periodicity never locks. The fundamental
now follows a per-syllable pitch contour (rise or fall, plus ~2% cycle-to-cycle
jitter), with the frequency integrated to phase rather than multiplied by t —
`2*pi*f*t` is only a chirp when f is the instantaneous rate, which it is not
once f0 itself moves. is_tonal is now False in every window.

Two smaller fixes fell out of that:

- Aspiration noise is high-passed rather than broadband. Flat noise puts
  energy in every Goertzel bin, so the strongest DTMF row and column both
  clear the detector's `total_power * 0.1` threshold and each syllable reads
  as a keypress. A first-difference filter leaves the 697-1633 Hz bands
  comparatively empty. The level is set for margin — spectral flatness lands
  at ~0.46, mid-way through the 0.1-0.5 band, not on an edge.

- The music fixture gained two more harmonics and a recording-style noise
  floor. Windows straddling a chord change had a momentarily sparse spectrum
  and fell *below* the music score's 0.05 flatness floor, scoring as speech.

All three fixtures now classify correctly in 100% of windows (music 27/27,
speech 5/5, silence 2/2), and remain correct when the window is stepped by
half a window — a fixture that only works on aligned boundaries would still
be a trap in a live call, where the analysis window has no relationship to
where the audio began.

Confirmed on a real call through the lab: scenario 1003 now shows the whole
hold-slayer arc, speech -> sustained music -> speech, matching the dialplan.

tests/test_lab_fixtures.py guards this: it sweeps every window rather than
sampling the first, which is exactly what the original validation missed, and
checks the generator is byte-for-byte deterministic. It skips when the
fixtures have not been generated, since they are gitignored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 07:56:58 -04:00
..

Asterisk lab — a fake PSTN

An Asterisk instance that answers calls, plays an IVR, holds you with music, and eventually connects a "human". It gives the gateway something real to dial that is not the PSTN: no charges, no strangers, no E911 exposure, and a deterministic script that makes classifier regressions reproducible.

Design rationale and the Virgo deployment plan: docs/asterisk-lab-design.md.

This lab found five bugs in SippyEngine on its first call — the engine had never successfully placed one. Everything below runs against the real SippyEngine, never MockSIPEngine, which is the entire point.


Run it

cd tests/lab

# 1. Generate the audio fixtures (the image ships with NO sound files).
python sounds/generate.py

# 2. Render the local configs. They carry a host-specific IP and the lab
#    password, so they are gitignored — regenerate them per machine.
cd dialplan
LOCALIP=$(ip route get 1.1.1.1 | grep -oP '(?<=src\s)\d+(\.\d+){3}')
sed -e "s/{{ asterisk_sip_port }}/21061/" \
    -e "s/{{ asterisk_external_ip }}/$LOCALIP/" \
    -e "s#{{ asterisk_local_net }}#10.10.0.0/24#" \
    -e "s/{{ asterisk_match_host }}/127.0.0.1/" \
    -e "s/{{ asterisk_sip_username }}/holdslayer/" \
    -e "s/{{ asterisk_sip_password }}/labpassword/" \
    pjsip.conf > pjsip.local.conf
sed -e "s/{{ asterisk_rtp_start }}/21100/" \
    -e "s/{{ asterisk_rtp_end }}/21149/" \
    rtp.conf > rtp.local.conf
cd ..

# 3. Start it.
docker compose -f docker-compose.lab.yml up -d

Point Hold Slayer at it — no code changes, no test-only branch. make_call builds sip:{number}@{trunk_host}:{trunk_port}, so the lab is just an address:

USE_MOCK_SIP=false
SIP_TRUNK_HOST=127.0.0.1
SIP_TRUNK_PORT=21061
SIP_TRUNK_USERNAME=holdslayer
SIP_TRUNK_PASSWORD=labpassword
SIP_TRUNK_DID=+15550000000
GATEWAY_SIP_PORT=21062     # must differ from the Asterisk port

The repo's own .env sets USE_MOCK_SIP=true and a placeholder trunk host, and pydantic-settings lets .env win over the process environment. If the engine reports MockSIPEngine despite the above, that is why.

AIPSTNGateway(settings=...) also defaults to MockSIPEngine unless an engine is assigned — main.py's lifespan calls build_sip_engine() after construction. A harness that skips that step silently tests the mock.

Useful commands

docker compose -f docker-compose.lab.yml exec asterisk asterisk -rvvv   # CLI
docker compose -f docker-compose.lab.yml logs -f asterisk               # logs
docker compose -f docker-compose.lab.yml exec asterisk \
    asterisk -rx "pjsip set logger on"                                  # SIP trace

Scenarios

Hold Slayer dials these as number.

Ext Scenario Proves
1001 Answers, speech, hangs up Baseline: INVITE→200→ACK→RTP→BYE, audio both ways
1002 IVR menu, branches on DTMF send_dtmf really emits RFC 2833 and Asterisk receives it
1003 Hold music, then a human The hold-slayer loop: music → wait → human → ring owner
1004 Long hold (~10 min) MAX_HOLD_TIME, HOLD_CHECK_INTERVAL
1005 Busy Failure path: call marked FAILED, no stuck leg
1006 Rings, never answers Timeout path
1007 Answers, hangs up after 5s Remote BYE, DB persistence on hangup
1008 Answers, then silence Classifier SILENCE vs. no-audio
1099 Echo test Debugging aid — confirm bidirectional RTP by ear

Audio fixtures

The Asterisk image ships no sound files, and the design calls for deterministic audio: real hold music varies per call, so a classifier regression on the PSTN is indistinguishable from noise. sounds/generate.py synthesises three fixtures from fixed seeds — byte-identical every run.

Verified against AudioClassifier (16 kHz):

Fixture Classifies as Confidence
lab-music.sln MUSIC 0.85
lab-speech.sln LIVE_HUMAN 0.75
lab-silence.sln SILENCE 1.00

Format is 8 kHz 16-bit mono signed-linear (.sln) — Asterisk's native telephony rate, played without transcoding.

The speech fixture's formants deliberately avoid the DTMF bands (rows 697941 Hz, columns 12091633 Hz). The first version landed on a valid DTMF pair and the whole utterance classified as a keypress.


Known limits

  • The classifier receives nothing on a live call. MediaPipeline.create_tap is a stub — it logs 🎤 Audio tap created and returns a tap that is never fed (core/media_pipeline.py, and the same at stream creation). RTP flows and Asterisk plays audio, but nothing reaches the classifier. The table above was measured by feeding the fixtures directly. This blocks the hold-slayer scenarios (1002/1003/1004).
  • Not real PSTN audio — no transcoding artefacts, packet loss, jitter, or carrier-side DTMF mangling. Asterisk is clean; the PSTN is not.
  • Not real IVR behaviour — this dialplan is what we imagine a bank sounds like. Real trees are longer, noisier, and interrupt.
  • Not trunk registration against a real ITSP_register_trunk() works against Asterisk, but carrier quirks are their own phase.

Security

pjsip.conf refuses anonymous inbound calls: every call must authenticate as the hold-slayer endpoint. Asterisk's stock examples allow anonymous calls and are a well-known toll-fraud target — there is no PSTN behind this instance, so an unauthorised call reaches only the dialplan, but the lock-down keeps this config safe to copy.

Endpoint matching is by source address (type=identify). Asterisk's default matches the From-header domain, which Hold Slayer populates from its SIP bind address — 0.0.0.0 on a wildcard bind, which never matches.

Separate, pre-existing: Hold Slayer's own SIP listener answers 200 OK to any REGISTER with no digest challenge. Fine on loopback; it must be resolved before the gateway binds a LAN interface, or any host on the network can register as a device and receive transferred calls.