2 Commits

Author SHA1 Message Date
d8703a1ad6 🐾 fix(mantle): stop installing the unproven 20k max_tokens clamp
The fine-grained-tool-streaming opt-out is what actually fixes the
"Streaming completed but tool call never finished" crash loop: under
that beta an output cutoff mid-tool_use ends the stream without
content_block_stop, fast-agent's tool tracker leaves the block open,
and _raise_for_incomplete_anthropic_tools raises a RuntimeError that
bypasses the graceful stop_reason=max_tokens path and burns the retry
ladder. That shim costs no output length and is kept.

The companion max_tokens clamp is not. Its 20 000 ceiling was an
empirical observation, never a documented Mantle limit, and
re-investigation could not establish what enforces it: fast-agent
carries no 20 000 default anywhere (the matching TASK_BUDGET_MIN_TOKENS
is a validation floor for a different, unconfigured feature), no model
overlay is configured, and ModelDatabase reports
max_output_tokens=128000 for opus-4-8. Installing it would cement a
ceiling we cannot prove and silently truncate turns that might
otherwise complete.

install_max_tokens_clamp() is left in place, unwired, so it can be
re-enabled if the limit is ever confirmed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 21:55:10 -04:00
537f3c7963 Merge pull request '🐾 fix(mantle): survive Mantle's 20k output ceiling on tool-heavy turns' (#8) from feature/mantle-max-tokens-shim into main
Reviewed-on: #8
2026-08-04 20:54:12 +00:00
2 changed files with 19 additions and 3 deletions

View File

@@ -224,11 +224,25 @@ def install_max_tokens_clamp() -> None:
# ── Orchestrator ─────────────────────────────────────────────────────────────
def install_all() -> None:
"""Install all Mantle shims. Call once at process startup."""
"""Install all Mantle shims. Call once at process startup.
``install_max_tokens_clamp`` is deliberately NOT installed. Its 20 000
ceiling was an empirical observation, never a documented Mantle limit,
and re-investigation could not establish what enforces it: fast-agent
carries no 20 000 default anywhere, no model overlay is configured, and
``ModelDatabase`` reports ``max_output_tokens=128000`` for opus-4-8.
Hardcoding the constant would cement a ceiling we cannot prove and would
silently truncate turns that might otherwise complete. The shim is kept
below so it can be re-enabled if the limit is ever confirmed.
The fine-grained-tool-streaming opt-out is what actually fixes the
``Streaming completed but tool call never finished`` crash loop: it costs
no output length, it only lets a cutoff close its blocks properly and
land in fast-agent's graceful ``stop_reason=max_tokens`` handling.
"""
install_wire_name_prefix()
install_tool_use_caller_strip()
install_fine_grained_tool_streaming_opt_out()
install_max_tokens_clamp()
def maybe_install(anthropic_base_url: str | None) -> bool:

View File

@@ -200,7 +200,9 @@ def test_maybe_install_installs_when_mantle(monkeypatch: pytest.MonkeyPatch) ->
installed = mantle_shims.maybe_install("https://bedrock-mantle.us-east-1.api.aws/anthropic")
assert installed is True
assert calls == ["wire", "tool_use", "beta_opt_out", "max_tokens"]
# "max_tokens" is intentionally absent: the 20 000 clamp is not installed
# by default because the ceiling was never confirmed. See install_all().
assert calls == ["wire", "tool_use", "beta_opt_out"]
def test_maybe_install_noop_for_non_mantle(monkeypatch: pytest.MonkeyPatch) -> None: