🐾 fix(mantle): stop installing the unproven 20k max_tokens clamp #9

r · 2026-08-06T01:59:40Z

r commented

2026-08-06 01:59:40 +00:00

What

install_all() now wires three Mantle shims, not four. install_max_tokens_clamp() is left in the file, unwired, so it can be re-enabled if the ceiling is ever confirmed.

Why

Alan has been failing in Taurus with:

Error calling tool 'alan': Streaming completed but tool call never finished:
daedalus__revise_workspace_file:toolu_bdrk_01VDtxjSnkbDQujQmGnPbgvG

29 occurrences in 14 days on proteus, each burning a full retry ladder (~3 min/attempt) against the same wall. Also seen on publish_generated_document, and on Quentin.

The beta opt-out (shim 3) is what fixes it, and it is kept. Under fine-grained-tool-streaming-2025-05-14, an output cutoff mid-tool_use ends the stream without content_block_stop; fast-agent's tool tracker leaves the block open (tool_tracking.py:128), and _raise_for_incomplete_anthropic_tools (llm_anthropic.py:1790) raises a RuntimeError that bypasses the graceful stop_reason=max_tokens path. fast-agent 0.7.22 sends that beta unconditionally on every tool-bearing request — the base gate is hardcoded return True, no config knob. (Upstream's own Vertex subclass already overrides this method with an allowlist; this is the same move for Mantle.) The shim costs no output length — it only lets a cutoff close its blocks properly.

The companion max_tokens clamp (shim 4) is not kept. Its 20 000 ceiling was an empirical observation, never a documented Mantle limit, and re-investigation could not establish what enforces it:

fast-agent carries no 20 000 default anywhere — greppedterminal defaults, overlays, shared resources.
The matching TASK_BUDGET_MIN_TOKENS = 20_000 is a validation floor for a different feature, and task_budget_tokens is None for Mentor.
No model overlay is configured; Alan just sets model: anthropic.claude-opus-4-8.
ModelDatabase reports max_output_tokens=128000 for opus-4-8, and that is what fast-agent requests.
No AWS documentation of a 20 000 limit for Bedrock or Mantle. Real Bedrock ceilings are powers of two (4096/8192/65536/131072); a round decimal 20 000 looks software-chosen.

Installing it would cement a ceiling we cannot prove and silently truncate turns that might otherwise complete.

What the evidence does show

Across 14 days of proteus logs, no stream exceeded 20 000 output tokens, and two landed on exactly 20 000 with fast-agent's Maximum output tokens reached. That pattern is real and unexplained — but "observed ceiling" is not "documented product limit", and this PR declines to hardcode the latter from the former.

Verification

pytest was unavailable in both the local env (no fast_agent) and the service venv (no pytest), so behaviour was verified directly against the real fast-agent 0.7.22 on proteus:

install_all wires : ['wire', 'tool_use', 'beta_opt_out']
PASS: clamp is NOT installed

before opt-out: fine_grained_tool_streaming = True
after  opt-out: fine_grained_tool_streaming = False
after  opt-out: interleaved_thinking        = True (must stay True)
PASS: beta disabled, other betas untouched

⚠️ The full pytest suite was not run — please run it where pytest is available before merging.

Deploy note

Prod is running Pallas from before the shim-3/4 commit entirely: the deployed mantle_shims.py is the 5 819-byte two-shim version, and the running process (started Aug 3 20:51 UTC, the day before b5a3aa2) reports has beta opt-out: False. Taurus needs mentor/stage.yml + mentor/deploy.yml from fornax to pick this up.

Scope

This stops the crash/retry loop. It does not make large single-turn documents land — if something is truncating at 20 000 they will now truncate cleanly rather than crash. The durable fix is steering agents toward edit_workspace_file (changed passage only) over revise_workspace_file / publish_generated_document (whole body competes with thinking + prose for one budget).

## What `install_all()` now wires **three** Mantle shims, not four. `install_max_tokens_clamp()` is left in the file, unwired, so it can be re-enabled if the ceiling is ever confirmed. ## Why Alan has been failing in Taurus with: ``` Error calling tool 'alan': Streaming completed but tool call never finished: daedalus__revise_workspace_file:toolu_bdrk_01VDtxjSnkbDQujQmGnPbgvG ``` 29 occurrences in 14 days on `proteus`, each burning a full retry ladder (~3 min/attempt) against the same wall. Also seen on `publish_generated_document`, and on Quentin. **The beta opt-out (shim 3) is what fixes it, and it is kept.** Under `fine-grained-tool-streaming-2025-05-14`, an output cutoff mid-`tool_use` ends the stream without `content_block_stop`; fast-agent's tool tracker leaves the block open (`tool_tracking.py:128`), and `_raise_for_incomplete_anthropic_tools` (`llm_anthropic.py:1790`) raises a `RuntimeError` that bypasses the graceful `stop_reason=max_tokens` path. fast-agent 0.7.22 sends that beta unconditionally on every tool-bearing request — the base gate is hardcoded `return True`, no config knob. (Upstream's own Vertex subclass already overrides this method with an allowlist; this is the same move for Mantle.) The shim costs **no output length** — it only lets a cutoff close its blocks properly. **The companion `max_tokens` clamp (shim 4) is not kept.** Its 20 000 ceiling was an empirical observation, never a documented Mantle limit, and re-investigation could not establish what enforces it: - fast-agent carries no 20 000 default anywhere — greppedterminal defaults, overlays, shared resources. - The matching `TASK_BUDGET_MIN_TOKENS = 20_000` is a *validation floor* for a different feature, and `task_budget_tokens` is `None` for Mentor. - No model overlay is configured; Alan just sets `model: anthropic.claude-opus-4-8`. - `ModelDatabase` reports `max_output_tokens=128000` for opus-4-8, and that is what fast-agent requests. - No AWS documentation of a 20 000 limit for Bedrock or Mantle. Real Bedrock ceilings are powers of two (4096/8192/65536/131072); a round decimal 20 000 looks software-chosen. Installing it would cement a ceiling we cannot prove and silently truncate turns that might otherwise complete. ### What the evidence does show Across 14 days of `proteus` logs, no stream exceeded 20 000 output tokens, and two landed on exactly 20 000 with fast-agent's `Maximum output tokens reached`. That pattern is real and unexplained — but "observed ceiling" is not "documented product limit", and this PR declines to hardcode the latter from the former. ## Verification `pytest` was unavailable in both the local env (no `fast_agent`) and the service venv (no `pytest`), so behaviour was verified **directly against the real fast-agent 0.7.22 on proteus**: ``` install_all wires : ['wire', 'tool_use', 'beta_opt_out'] PASS: clamp is NOT installed before opt-out: fine_grained_tool_streaming = True after opt-out: fine_grained_tool_streaming = False after opt-out: interleaved_thinking = True (must stay True) PASS: beta disabled, other betas untouched ``` ⚠️ The **full pytest suite was not run** — please run it where pytest is available before merging. ## Deploy note Prod is running Pallas from **before** the shim-3/4 commit entirely: the deployed `mantle_shims.py` is the 5 819-byte two-shim version, and the running process (started Aug 3 20:51 UTC, the day *before* `b5a3aa2`) reports `has beta opt-out: False`. Taurus needs `mentor/stage.yml` + `mentor/deploy.yml` from fornax to pick this up. ## Scope This stops the crash/retry loop. It does **not** make large single-turn documents land — if something is truncating at 20 000 they will now truncate cleanly rather than crash. The durable fix is steering agents toward `edit_workspace_file` (changed passage only) over `revise_workspace_file` / `publish_generated_document` (whole body competes with thinking + prose for one budget).

r added 1 commit 2026-08-06 01:59:40 +00:00

🐾 fix(mantle): stop installing the unproven 20k max_tokens clamp d8703a1ad6

The fine-grained-tool-streaming opt-out is what actually fixes the
"Streaming completed but tool call never finished" crash loop: under
that beta an output cutoff mid-tool_use ends the stream without
content_block_stop, fast-agent's tool tracker leaves the block open,
and _raise_for_incomplete_anthropic_tools raises a RuntimeError that
bypasses the graceful stop_reason=max_tokens path and burns the retry
ladder. That shim costs no output length and is kept.

The companion max_tokens clamp is not. Its 20 000 ceiling was an
empirical observation, never a documented Mantle limit, and
re-investigation could not establish what enforces it: fast-agent
carries no 20 000 default anywhere (the matching TASK_BUDGET_MIN_TOKENS
is a validation floor for a different, unconfigured feature), no model
overlay is configured, and ModelDatabase reports
max_output_tokens=128000 for opus-4-8. Installing it would cement a
ceiling we cannot prove and silently truncate turns that might
otherwise complete.

install_max_tokens_clamp() is left in place, unwired, so it can be
re-enabled if the limit is ever confirmed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

r merged commit 84e026c2e7 into main

2026-08-06 02:00:15 +00:00

r referenced this issue from a commit

2026-08-06 02:00:17 +00:00

Merge pull request '🐾 fix(mantle): stop installing the unproven 20k max_tokens clamp' (#9) from fix/drop-unproven-max-tokens-clamp into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: r/pallas#9