docs: update Mantle setup to reflect automatic shim detection

This commit is contained in:
2026-05-12 11:16:22 -04:00
parent fe94f6a9a8
commit 75d529cf16
6 changed files with 380 additions and 105 deletions

View File

@@ -89,30 +89,35 @@ model_capabilities:
vision: false
context_window: 200000
max_output_tokens: 32000
mantle: false # optional — see "Mantle override" below
```
Capabilities are published in the registry and used to register unknown models
with fast-agent's `ModelDatabase`.
### Mantle override (`model_capabilities.mantle: true`)
### AWS Bedrock Mantle — automatic shims
Set this when the `anthropic.base_url` points at the AWS Bedrock **Mantle**
endpoint (`https://bedrock-mantle.{region}.api.aws/anthropic`). Pallas then
installs a provider-specific override for `(Provider.ANTHROPIC, model_name)`
in fast-agent's `ModelDatabase._PROVIDER_MODEL_OVERRIDES` that clones the
model's base parameters but strips the features Mantle rejects:
When `anthropic.base_url` points at a Bedrock Mantle endpoint
(`https://bedrock-mantle.{region}.api.aws/anthropic`), Pallas auto-detects it
at startup and installs two compatibility shims via `pallas.mantle_shims`.
No config flag is required.
- `anthropic_required_betas` — no `anthropic-beta: ...` header
- `reasoning` / `reasoning_effort_spec` — no extended-thinking request
- `anthropic_task_budget_supported` — no task budget
- `anthropic_web_fetch_version` / `anthropic_web_search_version` — no web tools
- `cache_ttl` — prompt caching disabled
**Shim 1 — wire-name prefix.** Mantle requires the full `anthropic.<name>`
wire id (e.g. `anthropic.claude-opus-4-7`). Fast-agent's model-spec parser
would otherwise strip the `anthropic.` prefix, causing a misleading
`404 "The model '...' does not exist"`. The shim registers the prefixed
forms in `ModelDatabase._PROVIDER_WIRE_MODEL_NAMES`.
Without this flag, fast-agent sends its default beta headers and `thinking`
parameters for modern Claude models (e.g. Opus 4.7, Sonnet 4.6) which Mantle
rejects with a misleading `404 "The model '...' does not exist"`. See
`docs/bedrock.md` for the full configuration walkthrough.
**Shim 2 — strip `caller: null` from replayed `tool_use` blocks.** Anthropic
SDK 0.100.x leaks `caller: null` onto serialised `BetaToolUseBlock` params
([upstream issue #1454](https://github.com/anthropics/anthropic-sdk-python/issues/1454)).
`api.anthropic.com` silently tolerates the extra field; Mantle rejects it
with `tool_use.caller: Input should be a valid dictionary or object`, which
breaks the MCP tool-use loop on the second turn. The shim monkeypatches
`AnthropicConverter._deserialize_assistant_raw_blocks` and
`_append_server_tool_channel_blocks` to pop the field before history is
re-sent.
See `docs/bedrock.md` for the full configuration walkthrough.
---