chore(compose): add shared json-file logging config and component labels
All checks were successful
CVE Scan & Docker Build / security-scan (push) Successful in 52s
CVE Scan & Docker Build / build-and-push (push) Successful in 3m21s

Introduce x-logging anchor with json-file driver, size/file caps, and
container name tagging so Alloy on puck can reliably tail every service
through the Docker socket. Apply to all services and inject
MNEMOSYNE_COMPONENT env vars (init/app/mcp/worker) for consistent log
attribution both
This commit is contained in:
2026-05-11 13:52:00 -04:00
parent 8ddbcf4612
commit 551c641e90
5 changed files with 225 additions and 37 deletions

View File

@@ -9,7 +9,7 @@ This document describes Mnemosyne's role in the Daedalus + Pallas architecture a
Mnemosyne exposes two interfaces for the wider Ouranos ecosystem:
1. **REST API** (`/library/api/*`) — consumed by the Daedalus backend (HTTP Basic auth, service account `daedalus-service`) for workspace lifecycle and asynchronous file ingestion. Phase 1, **implemented**.
2. **MCP Server** (port 22091 internal, `/mcp/` via nginx on 23090) — exposes search, browse, and retrieval tools. Phase 5 of Mnemosyne's own roadmap, **implemented** with workspace_id scoping and per-turn JWT access control. Consumed by Pallas FastAgents in production (Daedalus integration Phase 2, **implemented** — see [Phase 3 of this doc](#3-phase-3-per-turn-token-access-control-for-daedalus-integration)).
2. **MCP Server** (port 22091 internal, `/mcp/` via nginx on 23090) — exposes search, browse, and retrieval tools. Phase 5 of Mnemosyne's own roadmap, **implemented** with workspace-scoped access control via long-lived team JWTs. Consumed by Pallas FastAgents in production (Daedalus integration Phase 2, **implemented** — see [Phase 3 of this doc](#3-phase-3-long-lived-team-jwt-access-control-for-pallas-instances)).
### Phase status
@@ -17,7 +17,7 @@ Mnemosyne exposes two interfaces for the wider Ouranos ecosystem:
|-------|------|--------|
| 1. REST workspace + ingest API for Daedalus | `POST /workspaces/`, `DELETE /workspaces/{id}/`, `POST /ingest/`, `GET /jobs/{id}/` | **Implemented** |
| 2. MCP Server (Mnemosyne roadmap Phase 5) | `search`, `get_chunk`, `list_libraries`, `list_collections`, `list_items`, `get_health` | **Implemented** (workspace_id scoping enforced in Cypher) |
| 3. Per-turn signed-token access control for Daedalus integration | Daedalus mints HS256 JWTs carrying `{ws, libs}` claims; Mnemosyne validates via `MCPSigningKey` and scopes search via `_scope_from_claims` | **Implemented** |
| 3. Long-lived team JWT access control for Pallas instances | Mnemosyne mints a 10-year HS256 JWT per Pallas instance (Team); Daedalus stores it encrypted and the operator pastes the plaintext into `fastagent.secrets.yaml`. Mnemosyne scopes search to the team's assigned workspaces via `TeamWorkspaceAssignment`. | **Implemented** |
---
@@ -367,40 +367,65 @@ mnemosyne_s3_operations_total{operation,status} counter
- [x] ASGI mount + uvicorn deployment on port 22091; nginx proxies via `/mcp/` on 23090
- [x] Prometheus metrics (`mnemosyne_mcp_*`)
### Phase 3 — Per-turn token access control for Daedalus integration ✅ Implemented
### Phase 3 — Long-lived team JWT access control for Pallas instances ✅ Implemented
Daedalus mints a short-lived HS256 JWT per chat turn and sends it as `Authorization: Bearer` to Pallas. Pallas forwards the token to outgoing Mnemosyne MCP calls (via `pallas/_fastagent_patch`). Mnemosyne validates the JWT and scopes every search to the workspace indicated by the `ws` claim.
Each Pallas instance registered in Daedalus is mirrored as a Mnemosyne **Team**. Mnemosyne mints a long-lived (10-year) HS256 JWT for the team; the operator pastes the plaintext into the Pallas instance's `fastagent.secrets.yaml`. Every MCP call from that Pallas instance carries the team JWT as a static `Authorization: Bearer` header. Mnemosyne validates the JWT and scopes search to the workspaces assigned to that team.
**Mnemosyne-side components:**
- [x] `MCPSigningKey` model — stores active HS256 secrets keyed by `kid`. Managed via `manage.py seed_signing_key --kid <kid>`.
- [x] `resolve_mcp_jwt(token_string)` in `mcp_server/auth.py` — validates signature, `exp`, `iss`, `jti` replay; returns claims dict.
- [x] `MCPAuthMiddleware.on_call_tool` — detects JWT shape (three dot-separated segments), routes to `resolve_mcp_jwt`, stores claims in FastMCP context state via `STATE_KEY_CLAIMS`.
- [x] `_scope_from_claims(claims, arg_workspace_id)` claims trump tool args; returns `(ws, allowed_libraries)`.
- [x] `allowed_libraries` on `SearchRequest` — extends `_WORKSPACE_SCOPE_CLAUSE` to include user-managed libraries in addition to the workspace's own.
- [x] `MCPSigningKey` model — stores active HS256 secrets keyed by `kid`. Managed via `manage.py seed_signing_key --kid <kid>`. The hex stays in Mnemosyne's DB; Daedalus never sees it.
- [x] `Team` model — one row per Pallas instance. `id` = `PallasInstance.id` on the Daedalus side (stable UUID). `active_jti` identifies the single currently-valid JWT; rotation changes this field, immediately invalidating the old token.
- [x] `TeamWorkspaceAssignment` model — maps a `Team` to a set of Daedalus workspace UUIDs. Updated by Daedalus via `PUT /mcp_server/api/teams/{id}/workspaces/` whenever workspace attachments change.
- [x] `resolve_mcp_jwt(token_string)` in `mcp_server/auth.py` — validates signature, `exp`, `iss`. For team JWTs (`iss=mnemosyne`, `typ=team`): parses `sub=team:<uuid>` `claims["team_id"]`; bypasses the per-turn JTI replay cache (team tokens are intentionally reused).
- [x] `_libraries_for_team(team_id, jti)` — looks up the `Team` row, verifies `active=True` and `active_jti == jti`, then translates `TeamWorkspaceAssignment` rows into Library UIDs via a single Cypher query.
- [x] `MCPAuthMiddleware.on_call_tool` — routes team JWTs through `_libraries_for_team`; routes legacy per-turn JWTs through `_scope_from_claims` (backward-compatible).
- [x] REST control plane at `/mcp_server/api/teams/`:
- `POST /` — create team by UUID; mints JWT, returns plaintext once.
- `GET /{id}/` — team state (workspace_ids, active status).
- `DELETE /{id}/` — soft-delete (`active=False`); all JWTs immediately invalid.
- `PUT /{id}/workspaces/` — replace workspace assignment list (idempotent).
- `POST /{id}/rotate/` — mint new JWT with new `active_jti`; returns plaintext once.
**Token format (HS256):**
**Team JWT format (HS256):**
```json
{
"iss": "daedalus",
"sub": "chat",
"ws": "<workspace_uuid>",
"libs": [],
"iss": "mnemosyne",
"aud": "mnemosyne",
"typ": "team",
"sub": "team:<pallas-instance-uuid>",
"iat": 1746000000,
"exp": 1746000600,
"jti": "<uuid4>"
"exp": 2061360000,
"jti": "<active_jti uuid>"
}
```
The `libs` claim is reserved for future user-managed library assignment (deferred). Currently always `[]`; the workspace's own library is always included via the `ws` claim.
**Provisioning:**
**Provisioning (once per Pallas instance):**
```bash
# On Mnemosyne host, once:
docker compose exec app python manage.py seed_signing_key --kid daedalus-1
# Copy the printed hex → DAEDALUS_MNEMOSYNE_SIGNING_SECRET in Daedalus .env
# 1. Seed the MCPSigningKey on Mnemosyne (once per deployment, not per instance):
docker compose exec app python manage.py seed_signing_key --kid daedalus-1 --retire-other
# The hex stays in Mnemosyne's DB — no operator action required.
# 2. Register the Pallas instance in Daedalus admin UI (/admin/pallas/).
# Daedalus calls POST /mcp_server/api/teams/ automatically.
# The team JWT is minted and stored encrypted in Daedalus.
# 3. Reveal the JWT via Daedalus admin UI (one-shot):
# GET /api/v1/pallas/{id}/team-jwt
# Copy the returned JWT string.
# 4. Paste into fastagent.secrets.yaml on the Pallas host:
# mcp:
# servers:
# mnemosyne:
# headers:
# Authorization: "Bearer <JWT>"
# 5. Restart the Pallas agent processes.
# 6. Attach workspaces in Daedalus workspace settings UI.
# Daedalus calls PUT /mcp_server/api/teams/{id}/workspaces/ automatically.
```
See the Daedalus-side spec [§9](../../daedalus/docs/mnemosyne_integration.md#9-phase-2--workspace-scoped-mcp-search-implemented) for the full integration architecture.
See the Daedalus-side spec [§9](../../daedalus/docs/mnemosyne_integration.md#9-phase-2--workspace-scoped-mcp-search-implemented) for the full operator walkthrough including JWT rotation and disaster recovery.