docs: update Mantle setup to reflect automatic shim detection
This commit is contained in:
37
README.md
37
README.md
@@ -89,30 +89,35 @@ model_capabilities:
|
||||
vision: false
|
||||
context_window: 200000
|
||||
max_output_tokens: 32000
|
||||
mantle: false # optional — see "Mantle override" below
|
||||
```
|
||||
|
||||
Capabilities are published in the registry and used to register unknown models
|
||||
with fast-agent's `ModelDatabase`.
|
||||
|
||||
### Mantle override (`model_capabilities.mantle: true`)
|
||||
### AWS Bedrock Mantle — automatic shims
|
||||
|
||||
Set this when the `anthropic.base_url` points at the AWS Bedrock **Mantle**
|
||||
endpoint (`https://bedrock-mantle.{region}.api.aws/anthropic`). Pallas then
|
||||
installs a provider-specific override for `(Provider.ANTHROPIC, model_name)`
|
||||
in fast-agent's `ModelDatabase._PROVIDER_MODEL_OVERRIDES` that clones the
|
||||
model's base parameters but strips the features Mantle rejects:
|
||||
When `anthropic.base_url` points at a Bedrock Mantle endpoint
|
||||
(`https://bedrock-mantle.{region}.api.aws/anthropic`), Pallas auto-detects it
|
||||
at startup and installs two compatibility shims via `pallas.mantle_shims`.
|
||||
No config flag is required.
|
||||
|
||||
- `anthropic_required_betas` — no `anthropic-beta: ...` header
|
||||
- `reasoning` / `reasoning_effort_spec` — no extended-thinking request
|
||||
- `anthropic_task_budget_supported` — no task budget
|
||||
- `anthropic_web_fetch_version` / `anthropic_web_search_version` — no web tools
|
||||
- `cache_ttl` — prompt caching disabled
|
||||
**Shim 1 — wire-name prefix.** Mantle requires the full `anthropic.<name>`
|
||||
wire id (e.g. `anthropic.claude-opus-4-7`). Fast-agent's model-spec parser
|
||||
would otherwise strip the `anthropic.` prefix, causing a misleading
|
||||
`404 "The model '...' does not exist"`. The shim registers the prefixed
|
||||
forms in `ModelDatabase._PROVIDER_WIRE_MODEL_NAMES`.
|
||||
|
||||
Without this flag, fast-agent sends its default beta headers and `thinking`
|
||||
parameters for modern Claude models (e.g. Opus 4.7, Sonnet 4.6) which Mantle
|
||||
rejects with a misleading `404 "The model '...' does not exist"`. See
|
||||
`docs/bedrock.md` for the full configuration walkthrough.
|
||||
**Shim 2 — strip `caller: null` from replayed `tool_use` blocks.** Anthropic
|
||||
SDK 0.100.x leaks `caller: null` onto serialised `BetaToolUseBlock` params
|
||||
([upstream issue #1454](https://github.com/anthropics/anthropic-sdk-python/issues/1454)).
|
||||
`api.anthropic.com` silently tolerates the extra field; Mantle rejects it
|
||||
with `tool_use.caller: Input should be a valid dictionary or object`, which
|
||||
breaks the MCP tool-use loop on the second turn. The shim monkeypatches
|
||||
`AnthropicConverter._deserialize_assistant_raw_blocks` and
|
||||
`_append_server_tool_channel_blocks` to pop the field before history is
|
||||
re-sent.
|
||||
|
||||
See `docs/bedrock.md` for the full configuration walkthrough.
|
||||
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user