From e9219f2d4a8d8aaeca497bdb674cc1377fcd5011 Mon Sep 17 00:00:00 2001 From: Robert Helewka Date: Tue, 28 Jul 2026 22:05:26 -0400 Subject: [PATCH] docs: mark PJSUA2 build complete and add build procedure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update deployment validation plan to reflect Phase 1b completion — pjsua2 built from pjproject 2.17 on caliban without sudo. Document the non-obvious RPATH/patchelf step, the status() method name correction, and two remaining caveats (not captured by pip install, Docker still runs stub media). Update README to point at the new docs/pjsua2-build.md and clarify stub-mode behavior. --- README.md | 7 +- docs/deployment-validation-plan.md | 54 +++++++++--- docs/pjsua2-build.md | 130 +++++++++++++++++++++++++++++ 3 files changed, 177 insertions(+), 14 deletions(-) create mode 100644 docs/pjsua2-build.md diff --git a/README.md b/README.md index 2eef06f..6dc2caf 100644 --- a/README.md +++ b/README.md @@ -165,7 +165,12 @@ pip install -e ".[dev]" > The PJSUA2 media pipeline needs the `pjsua2` Python bindings, which are > **not pip-installable** — they're built from pjproject (`./configure && > make && make install` with `--enable-shared` and the Python SWIG target). -> Without them the media layer runs in stub mode (signaling only). +> Without them the media layer runs in stub mode (signaling only): audio +> routing, recording and playback become no-ops that still return success. +> +> **See [docs/pjsua2-build.md](docs/pjsua2-build.md)** for the verified +> procedure (pjproject 2.17, no `sudo` required). It includes the `patchelf` +> RPATH step, without which the bindings compile and install but fail to import. ### 2. Configure diff --git a/docs/deployment-validation-plan.md b/docs/deployment-validation-plan.md index 0f961b1..efd7f6e 100644 --- a/docs/deployment-validation-plan.md +++ b/docs/deployment-validation-plan.md @@ -18,7 +18,7 @@ Verified on `caliban`, not assumed: | Fact | State | Consequence | |---|---|---| | `sippy` 2.3.0 | installed | SIP signaling is real | -| `pjsua2` | **not installed** | media pipeline is a **stub** — signaling works, audio does not | +| `pjsua2` | ✅ **built 2026-07-28** (pjproject 2.17) | real media pipeline; see [pjsua2-build.md](pjsua2-build.md) | | `USE_MOCK_SIP` | `true` | engine is `MockSIPEngine`; `/health` can never be `healthy` | | `SIP_TRUNK_*` | placeholders (`sip.yourprovider.com`) | no trunk configured | | `SIP_TRUNK_DID` | `+16472474242` | real DID already allocated | @@ -160,20 +160,48 @@ teardown is clean, no leaked legs after 10 call cycles, suite still green. --- -## Phase 1b — Gate: build PJSUA2 +## Phase 1b — Gate: build PJSUA2 ✅ COMPLETE -**Everything audio-shaped is blocked until this is done.** No TTS into a call, -no STT out of one, no recording, no hold-music classification on live audio. -`MediaPipeline` logs `⚠️ PJSUA2 not installed — media pipeline running in stub -mode` and every media call becomes a no-op that *returns successfully* — which -is the dangerous part, since it looks like it worked. +**Done 2026-07-28.** Built out of order (ahead of Phase 1a) because nothing +useful works without it. pjproject **2.17** — newer than expected, with Python +3.13 and modern-gcc support already upstream, so no patching was needed. -Build pjproject with Python bindings (`--enable-shared`, swig python2/3 -bindings) into the `/home/robert/env/hold-slayer` venv. Budget real time for -this; it is a from-source build, not a pip install. +`MediaPipeline.status()` now reports **`pjsua2_available: True`**; the stub-mode +warning is gone and the pipeline starts and stops cleanly at 16 kHz. Full +procedure recorded in **[pjsua2-build.md](pjsua2-build.md)**; README note now +points at it. -**Exit criteria:** `python -c "import pjsua2"` succeeds in the venv, and -`/health` / `media_pipeline.get_status()` reports `pjsua2_available: true`. +Notes worth carrying forward: + +- **No `sudo` was needed.** Both missing tools (`swig`, `patchelf`) have PyPI + wheels and installed into the venv, so nothing on the host changed outside + `~/src` and `~/.local`. +- **The RPATH step is the non-obvious part.** The bindings compile and install + cleanly and then fail at import with + `ImportError: libpjsua2.so.2: cannot open shared object file`, because + `~/.local/lib` isn't on the loader path. Fixed with `patchelf --set-rpath` on + both the extension **and** all 12 `libpj*.so.2` libraries — patching only the + extension just surfaces the transitive deps one layer down. Chosen over + `LD_LIBRARY_PATH` because that would have to be set for uvicorn, systemd, cron + and every subprocess, and it fails at call time rather than startup. Verified + under `env -i` from `/`, so it depends on no inherited environment. +- **Configure found OpenSSL, ALSA and Opus** — the full codec/TLS surface. +- **165 tests still pass.** All `pjsua2` imports in the codebase are lazy + (inside functions), so the suite still runs without touching real media. + +**Exit criteria:** `import pjsua2` succeeds in the venv ✅; `MediaPipeline` +reports `pjsua2_available: true` ✅ (note the method is `status()`, not +`get_status()` as this plan originally said). + +**Two caveats this build does not solve:** + +1. **Not captured by `pip install -e ".[dev]"`.** It lives outside Python + packaging metadata — a fresh venv, new host or rebuilt container needs it + repeated. Host provisioning, not a dependency. +2. **The Docker image still runs stub media** — the Dockerfile deliberately + skips this build. So the Phase 2 audio validation must run **outside the + container**, or the Dockerfile needs a pjproject build stage. Worth deciding + before Phase 2, since it determines where you test. --- @@ -301,7 +329,7 @@ docs. The README config table was already current — no drift there. | # | Blocker | Blocks | Severity | |---|---|---|---| -| 1 | PJSUA2 not installed | all audio: TTS/STT/recording/classifier in-call | **hard gate** | +| 1 | PJSUA2 not installed | all audio: TTS/STT/recording/classifier in-call | ✅ **fixed** — pjproject 2.17 built, `pjsua2_available: True` | | 2 | REGISTER accepts any credentials, no digest auth | safe exposure of port 5060 | **security** | | 3 | `TTS_BASE_URL` unset → defaults to app's own port | TTS entirely | config — ✅ fixed in template; **still set it in your real `.env`** | | 4 | STT/LLM endpoints unreachable as tested | Phase 2 | environment | diff --git a/docs/pjsua2-build.md b/docs/pjsua2-build.md new file mode 100644 index 0000000..43512de --- /dev/null +++ b/docs/pjsua2-build.md @@ -0,0 +1,130 @@ +# Building the PJSUA2 Python bindings + +The media pipeline ([core/media_pipeline.py](../core/media_pipeline.py)) needs +the `pjsua2` Python bindings. They are **not pip-installable** — they are SWIG +bindings compiled from pjproject. Without them `MediaPipeline.start()` catches +`ImportError` and runs in **stub mode**: signaling works, but audio routing, +recording, tapping and playback are all no-ops that *return successfully*. That +last part is the trap — a stub gateway looks healthy while being unable to speak +or listen. + +Verified on Ubuntu 25.10 / Python 3.13.7 / gcc 15.2, pjproject **2.17**, +2026-07-28. + +--- + +## Prerequisites + +`swig` and `patchelf` are both needed and both have PyPI wheels, so **no `sudo` +is required** — install them into the venv: + +```bash +pip install swig patchelf +``` + +System dev libraries (already present on caliban; `libsrtp2-dev` is *not* +needed — pjproject bundles its own SRTP): + +``` +libasound2-dev libssl-dev libopus-dev uuid-dev python3-dev +``` + +## Build + +```bash +mkdir -p ~/src && cd ~/src +git clone --depth 1 --branch 2.17 https://github.com/pjsip/pjproject.git +cd pjproject + +# Minimal config_site.h — enable TLS transport support +echo '#define PJ_HAS_SSL_SOCK 1' > pjlib/include/pj/config_site.h + +# -fPIC is REQUIRED: the Python extension links these into a shared object. +# --enable-shared builds the .so files the bindings load at runtime. +CFLAGS="-fPIC -O2" CXXFLAGS="-fPIC -O2" ./configure \ + --enable-shared \ + --disable-video --disable-libyuv --disable-libwebrtc \ + --prefix=$HOME/.local + +make dep && make -j$(nproc) && make install +``` + +Confirm configure found what the gateway needs (all should say yes/enabled): +OpenSSL, ALSA (`alsa/version.h`), OPUS. + +```bash +# Python bindings +cd pjsip-apps/src/swig +make python +cd python && python setup.py install +``` + +## The RPATH step — do not skip this + +The shared libraries install to `~/.local/lib`, which is **not** on the default +loader path, and neither the extension nor pjproject's own libraries carry an +RPATH. Straight after `setup.py install` the import fails with: + +``` +ImportError: libpjsua2.so.2: cannot open shared object file +``` + +Rather than requiring `LD_LIBRARY_PATH` everywhere (it would have to be set for +`uvicorn`, systemd, cron and any subprocess — easy to miss, and it fails at call +time, not startup), bake the path into the binaries: + +```bash +# The extension module … +patchelf --set-rpath $HOME/.local/lib \ + $VIRTUAL_ENV/lib/python3.13/site-packages/_pjsua2.cpython-313-x86_64-linux-gnu.so + +# … and pjproject's libraries, which must also find each other. +cd ~/.local/lib && for f in libpj*.so.2; do + patchelf --set-rpath $HOME/.local/lib "$f" +done +``` + +Patching only the extension is not enough: it resolves `libpjsua2`, which then +fails on its own transitive deps (`libpjsua`, `libpjsip`, `libpjmedia`, `libpj`, +…). Patch the whole set. + +## Verify + +```bash +# 1. Loads with a completely empty environment (proves RPATH, not inherited vars) +cd / && env -i $VIRTUAL_ENV/bin/python -c \ + "import pjsua2 as pj; ep=pj.Endpoint(); ep.libCreate(); print(ep.libVersion().full); ep.libDestroy()" + +# 2. No unresolved libraries +ldd $VIRTUAL_ENV/lib/python3.13/site-packages/_pjsua2*.so | grep "not found" + +# 3. The real gate — Hold Slayer's own pipeline reports it +python -c " +import asyncio +from core.media_pipeline import MediaPipeline +async def m(): + p = MediaPipeline(); await p.start() + assert p.status()['pjsua2_available'] is True + print('pjsua2_available: True'); await p.stop() +asyncio.run(m())" +``` + +`ImportError` in step 1 or `pjsua2_available: False` in step 3 means you are +still in stub mode. + +## Notes + +- **Not captured by `pip install -e ".[dev]"`.** This build lives outside the + Python packaging metadata, so a fresh venv, a rebuilt container, or another + host needs it repeated. Treat it as host provisioning. +- **The Docker image deliberately does not build this** (see the comment at the + top of the [Dockerfile](../Dockerfile)) — the container therefore runs stub + media. Anything validating audio must run outside the image, or the Dockerfile + needs a build stage adding. +- **Threading:** PJSUA2 starts its own worker threads, in addition to the Sippy + ED thread. Per the concurrency rule, PJSUA2 objects belong to the media + pipeline and must not be touched from the Sippy thread or mutated directly + from the asyncio loop. +- The extension compiles against system headers (`/usr/include/python3.13`) + rather than the venv's. Harmless while both are the same 3.13.7 with matching + SOABI — worth re-checking if the venv's Python is ever upgraded independently.