chore(compose): add shared json-file logging config and component labels
Introduce x-logging anchor with json-file driver, size/file caps, and container name tagging so Alloy on puck can reliably tail every service through the Docker socket. Apply to all services and inject MNEMOSYNE_COMPONENT env vars (init/app/mcp/worker) for consistent log attribution both
This commit is contained in:
@@ -9,7 +9,7 @@ This document describes Mnemosyne's role in the Daedalus + Pallas architecture a
|
||||
Mnemosyne exposes two interfaces for the wider Ouranos ecosystem:
|
||||
|
||||
1. **REST API** (`/library/api/*`) — consumed by the Daedalus backend (HTTP Basic auth, service account `daedalus-service`) for workspace lifecycle and asynchronous file ingestion. Phase 1, **implemented**.
|
||||
2. **MCP Server** (port 22091 internal, `/mcp/` via nginx on 23090) — exposes search, browse, and retrieval tools. Phase 5 of Mnemosyne's own roadmap, **implemented** with workspace_id scoping and per-turn JWT access control. Consumed by Pallas FastAgents in production (Daedalus integration Phase 2, **implemented** — see [Phase 3 of this doc](#3-phase-3-per-turn-token-access-control-for-daedalus-integration)).
|
||||
2. **MCP Server** (port 22091 internal, `/mcp/` via nginx on 23090) — exposes search, browse, and retrieval tools. Phase 5 of Mnemosyne's own roadmap, **implemented** with workspace-scoped access control via long-lived team JWTs. Consumed by Pallas FastAgents in production (Daedalus integration Phase 2, **implemented** — see [Phase 3 of this doc](#3-phase-3-long-lived-team-jwt-access-control-for-pallas-instances)).
|
||||
|
||||
### Phase status
|
||||
|
||||
@@ -17,7 +17,7 @@ Mnemosyne exposes two interfaces for the wider Ouranos ecosystem:
|
||||
|-------|------|--------|
|
||||
| 1. REST workspace + ingest API for Daedalus | `POST /workspaces/`, `DELETE /workspaces/{id}/`, `POST /ingest/`, `GET /jobs/{id}/` | **Implemented** |
|
||||
| 2. MCP Server (Mnemosyne roadmap Phase 5) | `search`, `get_chunk`, `list_libraries`, `list_collections`, `list_items`, `get_health` | **Implemented** (workspace_id scoping enforced in Cypher) |
|
||||
| 3. Per-turn signed-token access control for Daedalus integration | Daedalus mints HS256 JWTs carrying `{ws, libs}` claims; Mnemosyne validates via `MCPSigningKey` and scopes search via `_scope_from_claims` | **Implemented** |
|
||||
| 3. Long-lived team JWT access control for Pallas instances | Mnemosyne mints a 10-year HS256 JWT per Pallas instance (Team); Daedalus stores it encrypted and the operator pastes the plaintext into `fastagent.secrets.yaml`. Mnemosyne scopes search to the team's assigned workspaces via `TeamWorkspaceAssignment`. | **Implemented** |
|
||||
|
||||
---
|
||||
|
||||
@@ -367,40 +367,65 @@ mnemosyne_s3_operations_total{operation,status} counter
|
||||
- [x] ASGI mount + uvicorn deployment on port 22091; nginx proxies via `/mcp/` on 23090
|
||||
- [x] Prometheus metrics (`mnemosyne_mcp_*`)
|
||||
|
||||
### Phase 3 — Per-turn token access control for Daedalus integration ✅ Implemented
|
||||
### Phase 3 — Long-lived team JWT access control for Pallas instances ✅ Implemented
|
||||
|
||||
Daedalus mints a short-lived HS256 JWT per chat turn and sends it as `Authorization: Bearer` to Pallas. Pallas forwards the token to outgoing Mnemosyne MCP calls (via `pallas/_fastagent_patch`). Mnemosyne validates the JWT and scopes every search to the workspace indicated by the `ws` claim.
|
||||
Each Pallas instance registered in Daedalus is mirrored as a Mnemosyne **Team**. Mnemosyne mints a long-lived (10-year) HS256 JWT for the team; the operator pastes the plaintext into the Pallas instance's `fastagent.secrets.yaml`. Every MCP call from that Pallas instance carries the team JWT as a static `Authorization: Bearer` header. Mnemosyne validates the JWT and scopes search to the workspaces assigned to that team.
|
||||
|
||||
**Mnemosyne-side components:**
|
||||
|
||||
- [x] `MCPSigningKey` model — stores active HS256 secrets keyed by `kid`. Managed via `manage.py seed_signing_key --kid <kid>`.
|
||||
- [x] `resolve_mcp_jwt(token_string)` in `mcp_server/auth.py` — validates signature, `exp`, `iss`, `jti` replay; returns claims dict.
|
||||
- [x] `MCPAuthMiddleware.on_call_tool` — detects JWT shape (three dot-separated segments), routes to `resolve_mcp_jwt`, stores claims in FastMCP context state via `STATE_KEY_CLAIMS`.
|
||||
- [x] `_scope_from_claims(claims, arg_workspace_id)` — claims trump tool args; returns `(ws, allowed_libraries)`.
|
||||
- [x] `allowed_libraries` on `SearchRequest` — extends `_WORKSPACE_SCOPE_CLAUSE` to include user-managed libraries in addition to the workspace's own.
|
||||
- [x] `MCPSigningKey` model — stores active HS256 secrets keyed by `kid`. Managed via `manage.py seed_signing_key --kid <kid>`. The hex stays in Mnemosyne's DB; Daedalus never sees it.
|
||||
- [x] `Team` model — one row per Pallas instance. `id` = `PallasInstance.id` on the Daedalus side (stable UUID). `active_jti` identifies the single currently-valid JWT; rotation changes this field, immediately invalidating the old token.
|
||||
- [x] `TeamWorkspaceAssignment` model — maps a `Team` to a set of Daedalus workspace UUIDs. Updated by Daedalus via `PUT /mcp_server/api/teams/{id}/workspaces/` whenever workspace attachments change.
|
||||
- [x] `resolve_mcp_jwt(token_string)` in `mcp_server/auth.py` — validates signature, `exp`, `iss`. For team JWTs (`iss=mnemosyne`, `typ=team`): parses `sub=team:<uuid>` → `claims["team_id"]`; bypasses the per-turn JTI replay cache (team tokens are intentionally reused).
|
||||
- [x] `_libraries_for_team(team_id, jti)` — looks up the `Team` row, verifies `active=True` and `active_jti == jti`, then translates `TeamWorkspaceAssignment` rows into Library UIDs via a single Cypher query.
|
||||
- [x] `MCPAuthMiddleware.on_call_tool` — routes team JWTs through `_libraries_for_team`; routes legacy per-turn JWTs through `_scope_from_claims` (backward-compatible).
|
||||
- [x] REST control plane at `/mcp_server/api/teams/`:
|
||||
- `POST /` — create team by UUID; mints JWT, returns plaintext once.
|
||||
- `GET /{id}/` — team state (workspace_ids, active status).
|
||||
- `DELETE /{id}/` — soft-delete (`active=False`); all JWTs immediately invalid.
|
||||
- `PUT /{id}/workspaces/` — replace workspace assignment list (idempotent).
|
||||
- `POST /{id}/rotate/` — mint new JWT with new `active_jti`; returns plaintext once.
|
||||
|
||||
**Token format (HS256):**
|
||||
**Team JWT format (HS256):**
|
||||
|
||||
```json
|
||||
{
|
||||
"iss": "daedalus",
|
||||
"sub": "chat",
|
||||
"ws": "<workspace_uuid>",
|
||||
"libs": [],
|
||||
"iss": "mnemosyne",
|
||||
"aud": "mnemosyne",
|
||||
"typ": "team",
|
||||
"sub": "team:<pallas-instance-uuid>",
|
||||
"iat": 1746000000,
|
||||
"exp": 1746000600,
|
||||
"jti": "<uuid4>"
|
||||
"exp": 2061360000,
|
||||
"jti": "<active_jti uuid>"
|
||||
}
|
||||
```
|
||||
|
||||
The `libs` claim is reserved for future user-managed library assignment (deferred). Currently always `[]`; the workspace's own library is always included via the `ws` claim.
|
||||
|
||||
**Provisioning:**
|
||||
**Provisioning (once per Pallas instance):**
|
||||
|
||||
```bash
|
||||
# On Mnemosyne host, once:
|
||||
docker compose exec app python manage.py seed_signing_key --kid daedalus-1
|
||||
# Copy the printed hex → DAEDALUS_MNEMOSYNE_SIGNING_SECRET in Daedalus .env
|
||||
# 1. Seed the MCPSigningKey on Mnemosyne (once per deployment, not per instance):
|
||||
docker compose exec app python manage.py seed_signing_key --kid daedalus-1 --retire-other
|
||||
# The hex stays in Mnemosyne's DB — no operator action required.
|
||||
|
||||
# 2. Register the Pallas instance in Daedalus admin UI (/admin/pallas/).
|
||||
# Daedalus calls POST /mcp_server/api/teams/ automatically.
|
||||
# The team JWT is minted and stored encrypted in Daedalus.
|
||||
|
||||
# 3. Reveal the JWT via Daedalus admin UI (one-shot):
|
||||
# GET /api/v1/pallas/{id}/team-jwt
|
||||
# Copy the returned JWT string.
|
||||
|
||||
# 4. Paste into fastagent.secrets.yaml on the Pallas host:
|
||||
# mcp:
|
||||
# servers:
|
||||
# mnemosyne:
|
||||
# headers:
|
||||
# Authorization: "Bearer <JWT>"
|
||||
|
||||
# 5. Restart the Pallas agent processes.
|
||||
|
||||
# 6. Attach workspaces in Daedalus workspace settings UI.
|
||||
# Daedalus calls PUT /mcp_server/api/teams/{id}/workspaces/ automatically.
|
||||
```
|
||||
|
||||
See the Daedalus-side spec [§9](../../daedalus/docs/mnemosyne_integration.md#9-phase-2--workspace-scoped-mcp-search-implemented) for the full integration architecture.
|
||||
See the Daedalus-side spec [§9](../../daedalus/docs/mnemosyne_integration.md#9-phase-2--workspace-scoped-mcp-search-implemented) for the full operator walkthrough including JWT rotation and disaster recovery.
|
||||
|
||||
Reference in New Issue
Block a user