r/ouranos
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(alloy): add journal relabeling and kottos integration on puck

Browse Source 
Introduce structured journal relabel rules on puck to tag Pallas-managed
units with {service, project, component} labels matching the Mnemosyne
and Daedalus schema. Add kottos release variable and vault secrets
example entries for the new Pallas FastAgent runtime.

Remove the defunct mnemosyne syslog listener now that Mnemosyne ships
JSON logs via the docker-socket pipeline.
This commit is contained in:
Robert Helewka
2026-05-11 13:54:14 -04:00
parent e92ab80bbf
commit 8c95173705
19 changed files with 1336 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

90
docs/kottos.md
View File

@@ -163,6 +163,96 @@ The registry includes model capabilities on each agent entry:
}
```
## Deployment
Kottos runs two ways:
1. **Locally on caliban**, hand-started for iteration (`kottos` from the repo root). This is the flow documented above in *Quickstart*.
2. **In Ouranos / Virgo / Taurus via Ansible**, as a `systemd`-managed `pallas` process on the puck.incus container. This is the pipeline that feeds the Puck Services dashboard in Grafana.
### Ansible role
Lives in `ouranos/ansible/kottos/`:
| File | Purpose |
|---|---|
| `deploy.yml` | Main playbook — user/group, venv, systemd unit, config templating, registry probe. |
| `stage.yml` | Clones `git.helu.ca/r/kottos` at `{{ kottos_rel }}` and creates the release tarball. |
| `kottos.service.j2` | systemd unit. `SyslogIdentifier=kottos`, `StandardOutput=journal`, `PALLAS_LOG_STDOUT=1` via the env file. |
| `.env.j2` | Runtime environment for `pallas` — logging config, `PALLAS_AGENTS_CONFIG`. |
| `agents.yaml.j2` | Deployment topology with host/ports pulled from inventory. |
| `fastagent.config.yaml.j2` | LLM provider + MCP server URLs, parametric per environment. |
| `fastagent.secrets.yaml.j2` | API keys and auth tokens, rendered from Ansible Vault. |
### Inventory
Host variables live in `inventory/host_vars/puck.incus.yml` under **Kottos Configuration**:
```yaml
kottos_user: kottos
kottos_group: kottos
kottos_directory: /srv/kottos
kottos_host: "puck.incus"
kottos_registry_port: 24100
kottos_harper_port: 24101
kottos_scotty_port: 24102
kottos_research_port: 24150
kottos_tech_research_port: 24151
pallas_log_level: INFO
kottos_default_model: "openai.Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf"
kottos_openai_base_url: "http://nyx.helu.ca:22079/v1"
# ...plus one entry per downstream MCP URL so each environment overrides freely
```
Every host variable is parametric — Virgo's `puck.virgo.yml` (or wherever the Pallas host lives) can override any value without touching the templates.
### Vault
Four vault keys required — all documented in `inventory/group_vars/all/vault.yml.example`:
| Key | Used for |
|---|---|
| `vault_kottos_openai_api_key` | OpenAI-compatible LLM endpoint (nyx Qwen in Ouranos). |
| `vault_kottos_github_pat` | `GITHUB_PERSONAL_ACCESS_TOKEN` for the local GitHub MCP Docker container. |
| `vault_kottos_angelia_bearer` | Bearer token accepted by the Angelia MCP server. |
| `vault_kottos_mnemosyne_jwt` | Long-lived team JWT from Daedalus admin UI — Mnemosyne validates it on every `search_memory` call and scopes results to this team's workspaces. |
### Deploying
Wired into `site.yml`:
```bash
cd ansible
ansible-playbook kottos/stage.yml # clone repo + build tarball (local)
ansible-playbook kottos/deploy.yml # deploy + template + start
```
Or run the full site (`ansible-playbook site.yml`) — kottos's stage + deploy steps are the last block in the sequence.
### Logs
Journal identifier `kottos`, so on the host:
```bash
sudo journalctl -u kottos -f --output=cat | jq .
```
Alloy on puck's journal source relabels `__journal_syslog_identifier=kottos` to `{service="pallas", project="kottos"}`, then into Loki. Everything shows up in Grafana's *Puck Services — Logs & Health* dashboard under the **Pallas** row, with per-agent colouring driven by the `component` JSON field (`harper`, `scotty`, `research`, `tech_research`).
For per-agent follow-along:
```logql
{service="pallas", project="kottos", component="harper"} | json
```
For the opaque-MCP-transport-failure trace stream (see Pallas's bearer-forwarding incident history):
```logql
{service="pallas", project="kottos"} |= "pallas.forward.trace" | json
```
See [logging.md](logging.md) for the full label schema + level policy + add-a-new-service guide.
## Downstream MCP Servers
| Server | Host | URL |