docs: add Daedalus/Pallas/Mnemosyne integration design v1
Document the end-state auth/authz model unifying the three services around a bearer → resolved library set abstraction. Replaces the per-turn JWT forwarding scheme with static team JWTs held by Pallas deployments, eliminating custom transport code and the monkey-patch chain that caused opaque failures in agent teams. Also records the UX shift where Daedalus workspaces attach Teams (Pallas instances) rather than individual agents.
This commit is contained in:
743
docs/DAEDALUS_PALLAS_INTEGRATION_v1.md
Normal file
743
docs/DAEDALUS_PALLAS_INTEGRATION_v1.md
Normal file
@@ -0,0 +1,743 @@
|
|||||||
|
# Daedalus ↔ Pallas ↔ Mnemosyne Integration — v1
|
||||||
|
|
||||||
|
**Status:** Draft / approved design
|
||||||
|
**Authoritative home:** `mnemosyne/docs/DAEDALUS_PALLAS_INTEGRATION_v1.md`
|
||||||
|
**Versioning:** subsequent major revisions ship as `..._v2.md`, `..._v3.md`
|
||||||
|
alongside this file rather than overwriting it. Cross-service docs
|
||||||
|
(Daedalus, Pallas) link here rather than duplicating.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Summary
|
||||||
|
|
||||||
|
This document describes the end-state authentication / authorization
|
||||||
|
model connecting three services:
|
||||||
|
|
||||||
|
* **Mnemosyne** — knowledge platform. Owns Libraries, users, and the
|
||||||
|
MCP surface third-party clients query.
|
||||||
|
* **Daedalus** — workspace + file-lifecycle UI. Single-user per
|
||||||
|
instance. Registers Pallas instances, syncs file content to
|
||||||
|
Mnemosyne, drives chat.
|
||||||
|
* **Pallas** — FastAgent-backed MCP host that exposes agent teams
|
||||||
|
(Kottos, Mentor, Iolaus, Daedalus-chat, …) as HTTP MCP servers.
|
||||||
|
|
||||||
|
The model replaces the per-turn JWT *forwarding* scheme with a unified
|
||||||
|
**bearer → resolved library set** abstraction. Every authenticated
|
||||||
|
Mnemosyne request resolves to a set of Library UIDs the caller may
|
||||||
|
read; the principal type (opaque `MCPToken`, Daedalus per-turn JWT,
|
||||||
|
team JWT) only determines how that set is derived.
|
||||||
|
|
||||||
|
It also records the UX shift in Daedalus: **workspaces attach Teams
|
||||||
|
(Pallas instances), not individual agents**; the agent picker in chat
|
||||||
|
is filtered by the workspace's attached teams.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Motivation
|
||||||
|
|
||||||
|
The previous design forwarded a Daedalus-minted per-turn JWT through
|
||||||
|
Pallas to Mnemosyne via a custom `_DynamicBearerAuth`, a `ContextVar`,
|
||||||
|
a YAML scanner (`_refresh_forward_servers`), a header-mutation
|
||||||
|
monkey-patch, and three trace wrappers in
|
||||||
|
`pallas/pallas/_fastagent_patch.py`. When something broke on this
|
||||||
|
path, tracebacks landed nowhere visible because fast-agent's
|
||||||
|
`MCPAggregator._execute_on_server.try_execute` swallowed exceptions
|
||||||
|
(`except Exception as e: logger.error(…str(e)…); return error_factory(...)`),
|
||||||
|
and we were monkey-patching under it.
|
||||||
|
|
||||||
|
The failure mode (agent teams like Harper going into infinite
|
||||||
|
token-burning loops when Mnemosyne was unhappy) combined with the
|
||||||
|
diagnostic opacity made this architecture unsustainable. Per-turn
|
||||||
|
forwarding was also the wrong shape for non-interactive agent teams
|
||||||
|
that have no user session per call.
|
||||||
|
|
||||||
|
This design eliminates forwarding. Each Pallas deployment carries a
|
||||||
|
static, long-lived team JWT in its own `fastagent.secrets.yaml`. No
|
||||||
|
custom transport code in Pallas. Authorization happens server-side in
|
||||||
|
Mnemosyne against live DB rows.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Architecture
|
||||||
|
|
||||||
|
### 3.1 Services and responsibilities
|
||||||
|
|
||||||
|
| Service | Role in auth model |
|
||||||
|
|---|---|
|
||||||
|
| **Mnemosyne** | Owns Libraries, Library memberships, MCPTokens, Teams, TeamWorkspaceAssignments, signing keys. Validates bearers. Resolves every authenticated request to a Library set. |
|
||||||
|
| **Daedalus** | Control plane. Registers Pallas instances as Teams in Mnemosyne. Manages workspace ↔ team attachments. Stores team JWTs for copying into Pallas deployment configs. |
|
||||||
|
| **Pallas** | Stateless MCP host. Holds a static team JWT in `fastagent.secrets.yaml`. No custom auth-forwarding code. |
|
||||||
|
|
||||||
|
### 3.2 Three credential types
|
||||||
|
|
||||||
|
Every Mnemosyne MCP call presents a Bearer token that falls into one
|
||||||
|
of three categories:
|
||||||
|
|
||||||
|
| # | Credential | `iss` | Issuer | Lifetime | Library scope source |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| 1 | **Opaque `MCPToken`** | n/a | Mnemosyne admin | Until revoked | `MCPToken.allowed_libraries` (M2M, set at mint) |
|
||||||
|
| 2 | **Per-turn JWT** | `daedalus` | Daedalus | Minutes | `libs` claim (baked in at mint) |
|
||||||
|
| 3 | **Team JWT** | `mnemosyne` | Mnemosyne | 10 years | Live DB lookup via `Team.workspaces → Library` |
|
||||||
|
|
||||||
|
Category 2 is used only by Daedalus chat. Once Daedalus-chat itself
|
||||||
|
registers as a Pallas Team in Phase 4, category 2 retires entirely
|
||||||
|
and the design collapses to two credential types.
|
||||||
|
|
||||||
|
### 3.3 Resolved-library abstraction
|
||||||
|
|
||||||
|
Mnemosyne's auth middleware populates a single
|
||||||
|
`resolved_libraries: set[str]` per request. Downstream code (search,
|
||||||
|
get_document, list_libraries, etc.) only reads that set; it does not
|
||||||
|
care where the set came from.
|
||||||
|
|
||||||
|
```
|
||||||
|
Bearer → classify → dispatch
|
||||||
|
├─ Opaque MCPToken → allowed_libraries M2M
|
||||||
|
├─ per-turn JWT → claims["libs"]
|
||||||
|
└─ team JWT (typ=team) → live DB: team.workspaces → libraries
|
||||||
|
(filtered by Library.workspace_id)
|
||||||
|
↓
|
||||||
|
resolved_libraries: set[str]
|
||||||
|
↓
|
||||||
|
downstream tools
|
||||||
|
```
|
||||||
|
|
||||||
|
Fail-closed: if the resolution produces an empty set, the request sees
|
||||||
|
no Libraries. There is no "empty means everything" path.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Data model
|
||||||
|
|
||||||
|
### 4.1 Mnemosyne additions
|
||||||
|
|
||||||
|
#### `LibraryMembership` (new)
|
||||||
|
```python
|
||||||
|
class LibraryMembership(models.Model):
|
||||||
|
user = FK(User, related_name="library_memberships")
|
||||||
|
library_uid = CharField(max_length=64, db_index=True) # neo4j Library.uid
|
||||||
|
role = CharField(choices=[("owner","owner"),
|
||||||
|
("manager","manager"),
|
||||||
|
("reader","reader")])
|
||||||
|
created_at = DateTimeField(auto_now_add=True)
|
||||||
|
class Meta:
|
||||||
|
unique_together = ("user", "library_uid", "role")
|
||||||
|
```
|
||||||
|
- **owner** — full control: rename, delete, reassign ownership,
|
||||||
|
grant/revoke any role.
|
||||||
|
- **manager** — can grant `reader` and scope Library into MCPTokens
|
||||||
|
they own, but cannot delete the library or remove other owners.
|
||||||
|
- **reader** — can read via their own MCPTokens; cannot grant to other
|
||||||
|
users.
|
||||||
|
|
||||||
|
User can scope a Library into `MCPToken.allowed_libraries` iff they
|
||||||
|
have `owner` or `manager` role on it.
|
||||||
|
|
||||||
|
#### `MCPToken.allowed_libraries` (new M2M on existing model)
|
||||||
|
```python
|
||||||
|
allowed_libraries = models.ManyToManyField(Library, blank=True)
|
||||||
|
```
|
||||||
|
Fail-closed: empty → token grants access to zero libraries.
|
||||||
|
Admin form filters the picker by the current user's owned/managed
|
||||||
|
memberships.
|
||||||
|
|
||||||
|
#### `Team` (new)
|
||||||
|
```python
|
||||||
|
class Team(models.Model):
|
||||||
|
id = UUIDField(primary_key=True, editable=False)
|
||||||
|
# = Daedalus PallasInstance.id. Stays stable across redeploy /
|
||||||
|
# rehost of a given Pallas instance.
|
||||||
|
name = CharField(max_length=200) # display; mirrored from Daedalus
|
||||||
|
active = BooleanField(default=True)
|
||||||
|
active_jti = UUIDField(null=True) # current valid jti
|
||||||
|
created_at = DateTimeField(auto_now_add=True)
|
||||||
|
updated_at = DateTimeField(auto_now=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `TeamWorkspaceAssignment` (new)
|
||||||
|
```python
|
||||||
|
class TeamWorkspaceAssignment(models.Model):
|
||||||
|
team = FK(Team, related_name="workspace_assignments",
|
||||||
|
on_delete=CASCADE)
|
||||||
|
workspace_id = CharField(max_length=64) # matches Library.workspace_id
|
||||||
|
created_at = DateTimeField(auto_now_add=True)
|
||||||
|
class Meta:
|
||||||
|
unique_together = ("team", "workspace_id")
|
||||||
|
```
|
||||||
|
|
||||||
|
No library-level assignment for teams. Teams gain access to *all*
|
||||||
|
libraries of their assigned workspaces. If finer control is ever
|
||||||
|
needed later, it layers on without disturbing this model.
|
||||||
|
|
||||||
|
#### `MCPSigningKey` (existing, unchanged)
|
||||||
|
Re-used to sign team JWTs. The same signing key can back both
|
||||||
|
per-turn tokens (pre-retirement) and team tokens (long-lived).
|
||||||
|
|
||||||
|
### 4.2 Daedalus additions
|
||||||
|
|
||||||
|
#### `PallasInstance.team_jwt_encrypted` (new column, text, Fernet)
|
||||||
|
Stores the team JWT received from Mnemosyne at registration time.
|
||||||
|
Fernet-encrypted at rest using the same pattern as
|
||||||
|
`daedalus/llm_manager/encryption.py`. Displayed plaintext exactly once
|
||||||
|
in the admin detail page immediately after provisioning, so the
|
||||||
|
operator can copy it into `fastagent.secrets.yaml` on the Pallas
|
||||||
|
deployment.
|
||||||
|
|
||||||
|
#### `PallasInstance.pallas_team_mnemosyne_status` (new column)
|
||||||
|
`NULL | "pending" | "provisioned" | "failed"`. Drives the reconciler;
|
||||||
|
analogous to the existing `WorkspaceFile.mnemosyne_status`.
|
||||||
|
|
||||||
|
#### `workspace_pallas_assignments` (new M2M table)
|
||||||
|
```sql
|
||||||
|
CREATE TABLE workspace_pallas_assignments (
|
||||||
|
workspace_id TEXT NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
|
||||||
|
pallas_instance_id TEXT NOT NULL REFERENCES pallas_instances(id) ON DELETE CASCADE,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
PRIMARY KEY (workspace_id, pallas_instance_id)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
Starts empty on workspace create. Operator explicitly attaches Pallas
|
||||||
|
instances (Teams) before any Mnemosyne-backed agent — including
|
||||||
|
Daedalus chat — can search that workspace.
|
||||||
|
|
||||||
|
### 4.3 Nothing changes in Pallas
|
||||||
|
After cleanup, `pallas/pallas/_fastagent_patch.py` either becomes an
|
||||||
|
empty placeholder or is removed entirely. `pallas/__init__.py` no
|
||||||
|
longer invokes `install()`. Pallas deployments configure stock
|
||||||
|
fast-agent with a static `Authorization: Bearer <team-jwt>`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. JWT claim shapes
|
||||||
|
|
||||||
|
### 5.1 Per-turn JWT (category 2 — legacy, retires in Phase 4)
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"iss": "daedalus",
|
||||||
|
"aud": "mnemosyne", // optional, not enforced
|
||||||
|
"sub": "daedalus-chat",
|
||||||
|
"iat": 1715000000,
|
||||||
|
"exp": 1715000600, // ≤ 10 minutes
|
||||||
|
"jti": "uuid4",
|
||||||
|
"ws": "ws_abc", // Daedalus workspace id
|
||||||
|
"libs": ["lib_xxx", "lib_yyy"] // user-managed libraries
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Kept unchanged during Phase 2–3 so Daedalus chat continues to work
|
||||||
|
while we ship the team infrastructure.
|
||||||
|
|
||||||
|
### 5.2 Team JWT (category 3 — new)
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"iss": "mnemosyne",
|
||||||
|
"aud": "mnemosyne",
|
||||||
|
"sub": "team:<pallas_instance_uuid>", // UUID; Daedalus id
|
||||||
|
"typ": "team", // distinguishes from per-turn
|
||||||
|
"iat": 1715000000,
|
||||||
|
"exp": 1976000000, // +10 years
|
||||||
|
"jti": "uuid4"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
No `ws`, no `libs`. Authorization is evaluated live against
|
||||||
|
`TeamWorkspaceAssignment` rows on every request.
|
||||||
|
|
||||||
|
### 5.3 Validator changes in `mcp_server/auth.py`
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Accept both issuers; distinguish paths by typ.
|
||||||
|
_JWT_ISS = {"daedalus", "mnemosyne"}
|
||||||
|
|
||||||
|
def resolve_mcp_jwt(token_string: str) -> dict:
|
||||||
|
... # validate signature, iat/exp, required claims including sub
|
||||||
|
|
||||||
|
typ = claims.get("typ")
|
||||||
|
if typ == "team":
|
||||||
|
# No replay cache — team tokens are reused on every request.
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
if _remember_jti(jti, float(exp)):
|
||||||
|
raise MCPAuthError("Token replay detected.")
|
||||||
|
|
||||||
|
return claims
|
||||||
|
```
|
||||||
|
|
||||||
|
Downstream, the middleware branches:
|
||||||
|
```python
|
||||||
|
if claims.get("typ") == "team":
|
||||||
|
team = Team.objects.get(id=uuid_from_sub(claims["sub"]),
|
||||||
|
active=True,
|
||||||
|
active_jti=claims["jti"])
|
||||||
|
resolved_libraries = _libraries_for_team(team)
|
||||||
|
else:
|
||||||
|
resolved_libraries = claims["libs"]
|
||||||
|
```
|
||||||
|
|
||||||
|
`_libraries_for_team(team)` = all `Library` UIDs whose `workspace_id`
|
||||||
|
is in the team's `TeamWorkspaceAssignment` set.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Auth flow
|
||||||
|
|
||||||
|
### 6.1 Third-party MCP client with opaque `MCPToken`
|
||||||
|
1. Client sends `Authorization: Bearer <plaintext>`.
|
||||||
|
2. Middleware hashes → looks up `MCPToken` → validates active/expired.
|
||||||
|
3. `resolved_libraries = token.allowed_libraries.values_list("uid")`.
|
||||||
|
4. Fails closed if empty.
|
||||||
|
|
||||||
|
### 6.2 Daedalus chat per-turn JWT (legacy, retires Phase 4)
|
||||||
|
Unchanged from today. `iss=daedalus`, `typ` absent, `libs` carries the
|
||||||
|
workspace's user-managed libraries, `ws` carries the workspace id.
|
||||||
|
Mnemosyne validates against `MCPSigningKey` keyed by `kid`.
|
||||||
|
|
||||||
|
### 6.3 Agent team (Kottos / Mentor / Iolaus / post-migration Daedalus-chat)
|
||||||
|
1. Pallas sends `Authorization: Bearer <team-jwt>` (static, read from
|
||||||
|
`fastagent.secrets.yaml`).
|
||||||
|
2. Middleware validates signature → detects `typ=team`.
|
||||||
|
3. Reads `Team` row by UUID from `sub`. Verifies `active=True` and
|
||||||
|
`jti == active_jti`. Rejects otherwise.
|
||||||
|
4. Expands to `resolved_libraries` via
|
||||||
|
`TeamWorkspaceAssignment` → `Library.workspace_id`.
|
||||||
|
5. Fails closed if the team has no workspaces attached.
|
||||||
|
|
||||||
|
### 6.4 Failure modes
|
||||||
|
|
||||||
|
| Condition | Response |
|
||||||
|
|---|---|
|
||||||
|
| JWT signature invalid | `PermissionError("Invalid MCP token.")` |
|
||||||
|
| `exp` past (+30s leeway) | `PermissionError("Token has expired.")` |
|
||||||
|
| `iss` not in `_JWT_ISS` | `PermissionError("Invalid token issuer.")` |
|
||||||
|
| `typ=team`, team not found | `PermissionError("Invalid MCP token.")` |
|
||||||
|
| `typ=team`, team not active | `PermissionError("Token has been deactivated.")` |
|
||||||
|
| `typ=team`, `jti` stale | `PermissionError("Invalid MCP token.")` |
|
||||||
|
| Opaque token not found | `PermissionError("Invalid MCP token.")` |
|
||||||
|
| Opaque token, inactive user | `PermissionError("User account is disabled.")` |
|
||||||
|
| Resolved library set empty | Tool call proceeds but returns empty — this is *authorization*, not *authentication*, and the caller is legitimately scoped to nothing. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. REST API — Mnemosyne team lifecycle
|
||||||
|
|
||||||
|
All endpoints live under `/mcp_server/api/teams/` and are protected
|
||||||
|
by the existing `daedalus-service` HTTP Basic account (same auth as
|
||||||
|
`/library/api/workspaces/` and `/library/api/ingest/`).
|
||||||
|
|
||||||
|
### 7.1 `POST /mcp_server/api/teams/`
|
||||||
|
Create a team.
|
||||||
|
|
||||||
|
**Request**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "a3f1…", // UUID; mirrors Daedalus PallasInstance.id
|
||||||
|
"name": "Kottos"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response 201**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "a3f1…",
|
||||||
|
"name": "Kottos",
|
||||||
|
"jwt": "eyJhbGci…" // shown once; not recoverable later
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
On `id` collision: idempotent — returns existing team **without** the
|
||||||
|
JWT. Caller must call `/rotate` to get a new one.
|
||||||
|
|
||||||
|
### 7.2 `DELETE /mcp_server/api/teams/{id}/`
|
||||||
|
Soft-delete. Sets `active=False`. Old JWT invalid on next call.
|
||||||
|
|
||||||
|
### 7.3 `PUT /mcp_server/api/teams/{id}/workspaces/`
|
||||||
|
Replace the team's workspace assignment set. Idempotent.
|
||||||
|
|
||||||
|
**Request**
|
||||||
|
```json
|
||||||
|
{ "workspace_ids": ["ws_abc", "ws_def"] }
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response 200**
|
||||||
|
```json
|
||||||
|
{ "workspace_ids": ["ws_abc", "ws_def"] }
|
||||||
|
```
|
||||||
|
|
||||||
|
Non-existent workspaces silently accepted (they become active if/when
|
||||||
|
a `Library` with that `workspace_id` is later created). Mirrors the
|
||||||
|
Daedalus source of truth.
|
||||||
|
|
||||||
|
### 7.4 `POST /mcp_server/api/teams/{id}/rotate/`
|
||||||
|
Generate a fresh `jti` and JWT, replace `active_jti`. Old JWT invalid
|
||||||
|
immediately.
|
||||||
|
|
||||||
|
**Response 200**
|
||||||
|
```json
|
||||||
|
{ "jwt": "eyJhbGci…" }
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7.5 `GET /mcp_server/api/teams/{id}/`
|
||||||
|
Read-only team detail (no JWT). Used by Daedalus reconciler to
|
||||||
|
confirm state.
|
||||||
|
|
||||||
|
**Response 200**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "a3f1…",
|
||||||
|
"name": "Kottos",
|
||||||
|
"active": true,
|
||||||
|
"active_jti": "…", // for diagnostics, not a credential
|
||||||
|
"workspace_ids": ["ws_abc"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Daedalus lifecycle hooks
|
||||||
|
|
||||||
|
Mirrors the pattern in `daedalus/backend/daedalus/mnemosyne/lifecycle.py`:
|
||||||
|
every hook is best-effort, logs errors without blocking the local
|
||||||
|
operation, and is retried by the reconciler.
|
||||||
|
|
||||||
|
### 8.1 `on_pallas_registered(instance)`
|
||||||
|
1. `POST /mcp_server/api/teams/` with `id=instance.id`, `name=instance.name`.
|
||||||
|
2. Encrypt JWT via Fernet; store on `instance.team_jwt_encrypted`;
|
||||||
|
set `pallas_team_mnemosyne_status="provisioned"`.
|
||||||
|
3. Log `pallas_team_provisioned`.
|
||||||
|
|
||||||
|
On failure: status `"failed"`, reconciler retries.
|
||||||
|
|
||||||
|
### 8.2 `on_pallas_deleted(instance_id)`
|
||||||
|
`DELETE /mcp_server/api/teams/{id}/`. Row cascade locally; Mnemosyne
|
||||||
|
soft-deletes. Best-effort.
|
||||||
|
|
||||||
|
### 8.3 `on_workspace_pallas_attached(workspace_id, instance_id)`
|
||||||
|
1. Read all current `workspace_pallas_assignments` where
|
||||||
|
`pallas_instance_id=instance_id`.
|
||||||
|
2. `PUT /mcp_server/api/teams/{id}/workspaces/` with the resulting
|
||||||
|
workspace_id list.
|
||||||
|
|
||||||
|
### 8.4 `on_workspace_pallas_detached(workspace_id, instance_id)`
|
||||||
|
Symmetric to 8.3.
|
||||||
|
|
||||||
|
### 8.5 `on_workspace_deleted(workspace_id)`
|
||||||
|
For every attached Pallas instance, recompute and push updated
|
||||||
|
workspace list (so teams lose the deleted workspace).
|
||||||
|
|
||||||
|
### 8.6 Reconciler extension
|
||||||
|
Extends
|
||||||
|
`daedalus/backend/daedalus/mnemosyne/reconciler.py`:
|
||||||
|
- Re-runs `POST /teams/` for instances with status NULL or `"failed"`.
|
||||||
|
- Re-syncs workspace assignments for all instances on every cycle
|
||||||
|
(cheap idempotent PUT; guards against silent drift).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Operator workflows
|
||||||
|
|
||||||
|
### 9.1 Register a new Pallas deployment
|
||||||
|
1. Operator adds entry in Daedalus admin: `POST /api/v1/pallas`
|
||||||
|
with the Pallas registry URL. Daedalus fetches the registry,
|
||||||
|
creates the `PallasInstance` row, then calls Mnemosyne
|
||||||
|
`POST /mcp_server/api/teams/`. The JWT lands in
|
||||||
|
`instance.team_jwt_encrypted`.
|
||||||
|
2. Daedalus admin detail page surfaces the JWT plaintext **once**
|
||||||
|
(decrypted client-side or via a one-shot "reveal" endpoint that
|
||||||
|
logs the access). Operator copies it.
|
||||||
|
3. On the Pallas deployment machine, operator pastes the JWT into
|
||||||
|
`fastagent.secrets.yaml`:
|
||||||
|
```yaml
|
||||||
|
mcp:
|
||||||
|
servers:
|
||||||
|
mnemosyne:
|
||||||
|
transport: http
|
||||||
|
url: https://mnemosyne.example.helu.ca/mcp/
|
||||||
|
headers:
|
||||||
|
Authorization: Bearer eyJhbGci…
|
||||||
|
```
|
||||||
|
Operator removes any stale `forward_inbound_auth: true` from the
|
||||||
|
corresponding entry in `fastagent.config.yaml`. Restart Pallas.
|
||||||
|
|
||||||
|
### 9.2 Attach a Pallas team to a workspace
|
||||||
|
1. Daedalus workspace settings → "Attached Teams" → multi-select
|
||||||
|
across registered Pallas instances → save.
|
||||||
|
2. Daedalus fires `on_workspace_pallas_attached`, Mnemosyne's
|
||||||
|
`TeamWorkspaceAssignment` updates.
|
||||||
|
3. Agent picker in chat immediately shows agents from that team for
|
||||||
|
this workspace.
|
||||||
|
|
||||||
|
### 9.3 Retire a Pallas deployment
|
||||||
|
1. Daedalus admin → delete PallasInstance.
|
||||||
|
2. Daedalus calls `DELETE /mcp_server/api/teams/{id}/`; row marked
|
||||||
|
inactive in Mnemosyne. JWT rejected on next call.
|
||||||
|
3. Operator shuts down the Pallas deployment.
|
||||||
|
|
||||||
|
### 9.4 Rotate a compromised team JWT
|
||||||
|
1. Daedalus admin → "Rotate team JWT" action on the PallasInstance.
|
||||||
|
2. Daedalus calls `POST /mcp_server/api/teams/{id}/rotate/`,
|
||||||
|
re-encrypts and stores the new JWT.
|
||||||
|
3. Operator copies the new JWT into the Pallas deployment's
|
||||||
|
`fastagent.secrets.yaml`, restarts.
|
||||||
|
|
||||||
|
### 9.5 Provision existing Pallas instances (one-time migration)
|
||||||
|
After Mnemosyne phase 2 deploys:
|
||||||
|
```
|
||||||
|
$ daedalus manage.py provision_teams
|
||||||
|
```
|
||||||
|
Walks all existing `PallasInstance` rows, calls
|
||||||
|
`POST /mcp_server/api/teams/` for each, stores + prints JWTs in a
|
||||||
|
table for the operator to distribute. Idempotent: rows already
|
||||||
|
`provisioned` are skipped.
|
||||||
|
|
||||||
|
### 9.6 Issue an MCPToken for a third-party MCP client
|
||||||
|
1. Mnemosyne admin → MCPTokens → add. Pick user. Library picker is
|
||||||
|
filtered to libraries where that user has `owner` or `manager`
|
||||||
|
membership.
|
||||||
|
2. Submit. Plaintext shown once on the response page.
|
||||||
|
3. Operator pastes the plaintext into the third-party client's
|
||||||
|
config (Claude Desktop, Cline, etc.).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. UX changes in Daedalus
|
||||||
|
|
||||||
|
### 10.1 Workspace → attached Teams
|
||||||
|
|
||||||
|
Today: workspaces accumulate `AgentConnection` rows across every
|
||||||
|
registered Pallas instance; the agent picker is long, sub-agents
|
||||||
|
share names with parents (e.g. two distinct "Harper" agents), and it
|
||||||
|
is unclear which workspace grants which agents Mnemosyne access.
|
||||||
|
|
||||||
|
New:
|
||||||
|
- Workspace settings has a new section **Attached Teams**, a
|
||||||
|
multi-select over registered `PallasInstance` rows. Starts empty on
|
||||||
|
workspace create.
|
||||||
|
- Attaching/detaching a team triggers the lifecycle hook that updates
|
||||||
|
Mnemosyne's `TeamWorkspaceAssignment` for that team.
|
||||||
|
|
||||||
|
### 10.2 Agent picker in chat
|
||||||
|
|
||||||
|
New behavior:
|
||||||
|
- Lists only agents belonging to teams attached to the current
|
||||||
|
workspace.
|
||||||
|
- Displays each agent as **`TeamName › AgentTitle`** so sub-agent
|
||||||
|
name collisions resolve visually.
|
||||||
|
- Empty state: "No teams attached to this workspace. Go to workspace
|
||||||
|
settings → Attached Teams." (links inline).
|
||||||
|
|
||||||
|
### 10.3 Agent switching during chat
|
||||||
|
|
||||||
|
Unchanged mechanically: each chat turn may target a different
|
||||||
|
`agent_server_id`. New constraint: the target agent must belong to a
|
||||||
|
team currently attached to the workspace. The REST endpoint validates
|
||||||
|
this and returns 403 otherwise. The frontend picker already enforces
|
||||||
|
it as a UX affordance.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Migration
|
||||||
|
|
||||||
|
### 11.1 Current state (pre-migration)
|
||||||
|
|
||||||
|
* Mnemosyne is currently not in a released/working deployment; a
|
||||||
|
fresh rollout is possible.
|
||||||
|
* Daedalus has existing `PallasInstance` rows (registered via
|
||||||
|
`POST /api/v1/pallas`) but none have a team JWT.
|
||||||
|
* Kottos / Mentor / Iolaus each carry `forward_inbound_auth: true`
|
||||||
|
in `fastagent.config.yaml` and currently rely on the Pallas
|
||||||
|
forwarding patch to pass Daedalus's per-turn JWT to Mnemosyne.
|
||||||
|
|
||||||
|
### 11.2 Order of operations (must follow)
|
||||||
|
|
||||||
|
1. **Mnemosyne phase 2 deploys.** REST `/mcp_server/api/teams/` is
|
||||||
|
live; old per-turn JWT path still works. No consumers yet.
|
||||||
|
2. **Daedalus phase 4 deploys.** New columns + lifecycle hooks +
|
||||||
|
`provision_teams` command. On upgrade, migration creates columns
|
||||||
|
with default NULL status; existing PallasInstances remain
|
||||||
|
functional on the legacy (non-Mnemosyne) path.
|
||||||
|
3. **Operator runs `provision_teams`.** Every existing PallasInstance
|
||||||
|
gets a team in Mnemosyne and a stored JWT.
|
||||||
|
4. **Operator distributes JWTs** to each Pallas deployment
|
||||||
|
(Kottos / Mentor / Iolaus / Daedalus-chat). Each deployment
|
||||||
|
updates `fastagent.secrets.yaml`, removes
|
||||||
|
`forward_inbound_auth: true`, restarts.
|
||||||
|
5. **Pallas phase 3 cleanup deploys.** Forwarding infrastructure
|
||||||
|
removed from Pallas codebase. Safe only after all deployments
|
||||||
|
have switched to static JWTs.
|
||||||
|
6. **Daedalus per-turn token path retires.** `mnemosyne/tokens.py`
|
||||||
|
and its config (`MNEMOSYNE_SIGNING_SECRET`,
|
||||||
|
`MNEMOSYNE_SIGNING_KID`, `MNEMOSYNE_TOKEN_TTL_SECONDS`) are
|
||||||
|
removed after Daedalus chat's own team JWT is in place.
|
||||||
|
|
||||||
|
### 11.3 Rollback story
|
||||||
|
|
||||||
|
* **Mnemosyne phase 2**: safe to roll back — old per-turn JWT path
|
||||||
|
untouched; new endpoints simply disappear.
|
||||||
|
* **Daedalus phase 4**: safe to roll back until `provision_teams`
|
||||||
|
has run. After that, the JWTs are already distributed; rolling
|
||||||
|
back means the JWTs go unused but nothing breaks.
|
||||||
|
* **Pallas phase 3**: *not* safe to roll back independently — if any
|
||||||
|
deployment still has `forward_inbound_auth: true` pointing at code
|
||||||
|
that no longer exists, that deployment fails to start. Sequence
|
||||||
|
correctly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Deprecated / removed
|
||||||
|
|
||||||
|
At end-of-migration (after Phase 6):
|
||||||
|
|
||||||
|
### Pallas (`pallas/pallas/_fastagent_patch.py`)
|
||||||
|
- `_DynamicBearerAuth` (httpx Auth subclass)
|
||||||
|
- `_CurrentBearer` ContextVar plumbing
|
||||||
|
- `_refresh_forward_servers()` YAML scanner
|
||||||
|
- `_prepare_headers_and_auth_with_forward` (the header-mutation monkey-patch)
|
||||||
|
- `_send_request_with_trace`, `_session_call_tool_with_trace`,
|
||||||
|
`_execute_on_server_with_trace` (diagnostic wrappers installed
|
||||||
|
because the forwarding path was opaque)
|
||||||
|
- `install()` function in `_fastagent_patch`; `pallas/__init__.py` no
|
||||||
|
longer auto-installs
|
||||||
|
|
||||||
|
### Agent team configs (`kottos/`, `mentor/`, `iolaus/`)
|
||||||
|
- `forward_inbound_auth: true` under any server stanza in
|
||||||
|
`fastagent.config.yaml`
|
||||||
|
|
||||||
|
### Daedalus (`daedalus/backend/daedalus/`)
|
||||||
|
- `mnemosyne/tokens.py` (per-turn JWT mint)
|
||||||
|
- Config settings: `MNEMOSYNE_SIGNING_SECRET`, `MNEMOSYNE_SIGNING_KID`,
|
||||||
|
`MNEMOSYNE_TOKEN_TTL_SECONDS`, `MNEMOSYNE_MCP_URL`
|
||||||
|
|
||||||
|
### Mnemosyne (`mnemosyne/mcp_server/`)
|
||||||
|
- `_JWT_ISS` constant-string (replaced by set containing `daedalus`
|
||||||
|
and `mnemosyne`; `daedalus` entry removed in a later version once
|
||||||
|
per-turn path is deleted)
|
||||||
|
- The `_JTI_CACHE` replay logic continues to exist for the per-turn
|
||||||
|
path until that path retires; team JWTs bypass it entirely
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 13. Security
|
||||||
|
|
||||||
|
### 13.1 Token lifetimes
|
||||||
|
* **Opaque MCPToken**: until revoked (admin). Rotation is manual.
|
||||||
|
* **Per-turn JWT**: ≤ 10 minutes (existing `MNEMOSYNE_TOKEN_TTL_SECONDS`).
|
||||||
|
Retires with Phase 4 completion.
|
||||||
|
* **Team JWT**: 10 years. Rationale: operator cannot tolerate a
|
||||||
|
silent expiry-induced outage in a year. Revocation is explicit via
|
||||||
|
`Team.active`, `Team.active_jti`, or key rotation.
|
||||||
|
|
||||||
|
### 13.2 Revocation levers, in order of granularity
|
||||||
|
1. `PUT /teams/{id}/workspaces/` with `[]` — team sees nothing, JWT
|
||||||
|
still validates. Useful for pausing a deployment without
|
||||||
|
redistributing tokens.
|
||||||
|
2. `DELETE /teams/{id}/` — team marked inactive. All its tokens
|
||||||
|
rejected. Restoring requires re-POST (new id) or admin DB edit.
|
||||||
|
3. `POST /teams/{id}/rotate/` — `active_jti` changes; the token that
|
||||||
|
leaked stops working; the new JWT must be distributed.
|
||||||
|
4. `MCPSigningKey.retire()` — nuclear option. All JWTs signed with
|
||||||
|
that kid stop validating. Re-key + re-issue every team token.
|
||||||
|
|
||||||
|
### 13.3 At-rest protection
|
||||||
|
* `MCPToken.token_hash`: SHA-256 of plaintext; plaintext never stored.
|
||||||
|
* `MCPSigningKey.secret_hex`: 256-bit hex secret stored in Mnemosyne
|
||||||
|
DB only (not distributed).
|
||||||
|
* `PallasInstance.team_jwt_encrypted`: Fernet-encrypted by Daedalus's
|
||||||
|
`SECRET_KEY` (or `MNEMOSYNE_FERNET_KEY` if configured);
|
||||||
|
ciphertext at rest.
|
||||||
|
|
||||||
|
### 13.4 Audit points
|
||||||
|
* Every auth failure increments `mcp_auth_failures_total{reason=…}`.
|
||||||
|
* Every team lifecycle action logs `pallas_team_{provisioned,
|
||||||
|
deleted, rotated, workspaces_updated}` in Daedalus with full
|
||||||
|
correlation IDs.
|
||||||
|
* Every bearer resolution logs the principal type + resolved
|
||||||
|
library count at DEBUG (INFO until shakedown stabilizes).
|
||||||
|
|
||||||
|
### 13.5 Isolation model
|
||||||
|
|
||||||
|
Separation of "work" vs "personal" agents composes from three
|
||||||
|
independent mechanisms:
|
||||||
|
|
||||||
|
1. **Per-token scope (Mnemosyne)**: each MCPToken carries its own
|
||||||
|
`allowed_libraries`. A personal token and a work token may
|
||||||
|
belong to the same user yet see disjoint Library sets.
|
||||||
|
2. **Per-workspace attachment (Daedalus)**: a Pallas instance only
|
||||||
|
sees workspaces explicitly attached to it. Work Pallas and
|
||||||
|
personal Pallas attach to disjoint workspaces.
|
||||||
|
3. **Per-Daedalus-instance (deployment)**: the strongest isolation
|
||||||
|
is two Daedalus deployments pointing at distinct Mnemosyne
|
||||||
|
accounts. Nothing the operator does in deployment A reaches any
|
||||||
|
data accessible to deployment B.
|
||||||
|
|
||||||
|
For typical operator isolation, (1) + (2) suffice. (3) is the
|
||||||
|
escape hatch for hard compartmentalization.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 14. Testing
|
||||||
|
|
||||||
|
### 14.1 Mnemosyne test surface
|
||||||
|
* `resolve_mcp_jwt` accepts `iss in {daedalus, mnemosyne}`.
|
||||||
|
* `typ=team` branch: bypasses replay cache; resolves team + active_jti.
|
||||||
|
* `typ=team`: rejects if team missing, inactive, or `jti` stale.
|
||||||
|
* `LibraryMembership`: owner can grant; reader cannot; grant form
|
||||||
|
filters correctly.
|
||||||
|
* `MCPToken.allowed_libraries` empty → resolved library set empty.
|
||||||
|
* `TeamWorkspaceAssignment` PUT is idempotent and replaces, not
|
||||||
|
unions.
|
||||||
|
* `/mcp_server/api/teams/` endpoints: create, delete, rotate,
|
||||||
|
workspaces PUT, all authenticated as `daedalus-service`.
|
||||||
|
|
||||||
|
### 14.2 Daedalus test surface
|
||||||
|
* `on_pallas_registered` populates `team_jwt_encrypted` and transitions
|
||||||
|
status to `provisioned`.
|
||||||
|
* `on_workspace_pallas_attached` triggers the correct PUT payload.
|
||||||
|
* Agent-picker endpoint filters `AgentConnection` by attached
|
||||||
|
Pallas instances.
|
||||||
|
* `provision_teams` is idempotent.
|
||||||
|
|
||||||
|
### 14.3 Integration
|
||||||
|
* End-to-end: third-party MCP client with MCPToken → Mnemosyne
|
||||||
|
search scoped to `allowed_libraries`.
|
||||||
|
* End-to-end: Pallas agent with team JWT → Mnemosyne search scoped
|
||||||
|
to team's attached workspaces.
|
||||||
|
* End-to-end: workspace detached from team → agent no longer sees
|
||||||
|
that workspace's libraries (on next request, not stale-cached).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 15. Phased delivery
|
||||||
|
|
||||||
|
| # | Phase | Surface | Deployable independently? |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | Design doc | This file | Yes — this document |
|
||||||
|
| 2 | Mnemosyne | `LibraryMembership`, `MCPToken.allowed_libraries`, `Team`, `TeamWorkspaceAssignment`, unified `auth.py` resolver, `/mcp_server/api/teams/` REST, admin UIs, backfill, tests | Yes — old per-turn JWT path untouched |
|
||||||
|
| 3 | Pallas cleanup | Remove `_fastagent_patch.py` internals, docs | No — must wait until all deployments use static JWTs |
|
||||||
|
| 4 | Daedalus integration | `workspace_pallas_assignments`, `team_jwt_encrypted`, `pallas_team_mnemosyne_status`, lifecycle hooks, reconciler, `provision_teams`, admin API, agent-picker filter, register chat as team | Yes — new columns nullable, legacy path still works |
|
||||||
|
| 5 | Daedalus frontend | Workspace settings attached-teams picker, agent picker namespacing | Yes — backwards-compatible once phase 4 ships |
|
||||||
|
| 6 | Agent team cutovers | Kottos / Mentor / Iolaus paste JWT, remove `forward_inbound_auth`, restart | Yes — one at a time |
|
||||||
|
| 7 | Documentation | Mnemosyne README, Pallas README + `docs/auth.md`, Daedalus operator docs, cross-references to this file | Yes |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 16. Open items (v1)
|
||||||
|
|
||||||
|
None — all decisions are closed in this revision. Future revisions
|
||||||
|
that add scope (e.g., team-level library scoping finer than
|
||||||
|
workspace granularity; OAuth 2.1 for external MCP clients;
|
||||||
|
per-library audit logs) will ship as `_v2.md` alongside this file.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 17. Cross-references
|
||||||
|
|
||||||
|
* Existing Mnemosyne per-turn JWT implementation:
|
||||||
|
`mnemosyne/mnemosyne/mcp_server/auth.py`,
|
||||||
|
`mnemosyne/mnemosyne/mcp_server/models.py` (`MCPToken`,
|
||||||
|
`MCPSigningKey`).
|
||||||
|
* Existing Daedalus Mnemosyne integration:
|
||||||
|
`daedalus/backend/daedalus/mnemosyne/` (`client.py`,
|
||||||
|
`tokens.py`, `lifecycle.py`, `reconciler.py`),
|
||||||
|
`daedalus/backend/daedalus/api/v1/pallas.py`.
|
||||||
|
* Existing Pallas auth-forwarding patch (to be removed):
|
||||||
|
`pallas/pallas/_fastagent_patch.py`,
|
||||||
|
`pallas/pallas/__init__.py`.
|
||||||
Reference in New Issue
Block a user