🐾 fix(mantle): survive Mantle's 20k output ceiling on tool-heavy turns
Two new Mantle shims, auto-installed alongside the existing pair: - Opt out of the fine-grained-tool-streaming beta. Under it, an output-token cutoff mid-tool_use ends the stream without content_block_stop; fast-agent raises "Streaming completed but tool call never finished" and burns its retry ladder against the same wall (the observed ~700s Alan revise_workspace_file failures on Taurus). - Clamp default maxTokens to Mantle's observed 20 000-token server ceiling, so the model stops gracefully (proper block close + stop_reason=max_tokens) instead of being cut off by the gateway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -218,7 +218,7 @@ anthropic:
|
||||
|
||||
That's the whole configuration. Pallas auto-detects the
|
||||
`bedrock-mantle` hostname in `anthropic.base_url` at startup and installs
|
||||
two compatibility shims so fast-agent's default request shape matches
|
||||
four compatibility shims so fast-agent's default request shape matches
|
||||
what Mantle expects (see `pallas/mantle_shims.py`):
|
||||
|
||||
1. **Wire-name prefix** — re-adds the `anthropic.` prefix that fast-agent's
|
||||
@@ -233,6 +233,25 @@ what Mantle expects (see `pallas/mantle_shims.py`):
|
||||
Input should be a valid dictionary or object"`, which would otherwise
|
||||
break the MCP tool-use loop on the second turn.
|
||||
|
||||
3. **Fine-grained tool streaming opt-out** — stops fast-agent sending the
|
||||
`fine-grained-tool-streaming-2025-05-14` beta. Under that beta an
|
||||
output-token cutoff mid-`tool_use` block ends the stream without
|
||||
`content_block_stop`, which fast-agent surfaces as
|
||||
`Streaming completed but tool call never finished` and then retries
|
||||
into the same wall (~700 s agent failures on large tool bodies).
|
||||
Without the beta a cutoff closes blocks properly and lands in
|
||||
fast-agent's graceful `stop_reason=max_tokens` handling.
|
||||
|
||||
4. **`max_tokens` clamp** — Mantle enforces a server-side output ceiling
|
||||
of 20 000 tokens per response regardless of the requested
|
||||
`max_tokens` (fast-agent asks for the model's full 128 000). The shim
|
||||
clamps default `maxTokens` to `MANTLE_MAX_OUTPUT_TOKENS` (20 000) so
|
||||
the model stops gracefully at the limit instead of the gateway
|
||||
cutting the stream. A single agent turn — thinking, prose, and tool
|
||||
input combined — cannot exceed this on Mantle; agents that must emit
|
||||
more output in one turn need to split the work (e.g. patch-style
|
||||
edits instead of full-document rewrites).
|
||||
|
||||
The Anthropic SDK appends `/v1/messages` to `base_url` automatically.
|
||||
|
||||
**Feature support.** Mantle accepts the same Messages API request shape
|
||||
|
||||
Reference in New Issue
Block a user