diff --git a/.claude/rules/cache-api.md b/.claude/rules/cache-api.md new file mode 100644 index 0000000..8b74d20 --- /dev/null +++ b/.claude/rules/cache-api.md @@ -0,0 +1,35 @@ +--- +description: TheSportsDB client + read-through cache — permanent vs volatile, TTL + PG layers, V1/V2 split +paths: + - "nike/sportsdb.py" +--- + +# TheSportsDB client & caching + +- **Two cache layers.** In-memory TTL cache in `sportsdb.py` (`_CACHE`, 5-min TTL, + keyed by `v1|path|params` / `v2|path`) fronts every HTTP call; the PostgreSQL + cache in `db.py` stores permanent data. `clear_cache()` flushes the in-memory + layer; the dashboard's Clear Cache / `POST /api/cache/invalidate` flushes both. + +- **Permanent vs volatile is the core distinction.** Permanent data — teams, + players, leagues, events — is cached to PostgreSQL (read-through: DB first, API on + miss, then upsert). **Volatile data is fetched live every time and never written + to PG**: standings (`v1_standings`), livescores, current fixtures. Don't add + volatile endpoints to the PG cache; don't bypass the cache for permanent lookups. + +- **V2 is primary, V1 fills gaps.** V2 (`X-API-KEY` header, premium) covers search/ + lookup/list/schedule/livescore. V1 (key in URL path, free key `3` works) covers + standings (`lookuptable.php`), events-by-date, H2H, and the team/player search + the resolver uses. Keep new endpoints in the right versioned helper; both go + through `_get_v1`/`_get_v2` so they inherit the TTL cache. + +- **One function per endpoint, returning parsed dicts.** Client functions + `raise_for_status()` and return the JSON dict — they don't format and don't + swallow errors (the *tools* decide how to handle failure). Keep them thin. + +- **The free key is `3`.** `SPORTSDB_V1` embeds it in the URL; V2 needs a real + premium key. `check_connection()` probes a lightweight V1 endpoint and returns a + `{connected, latency_ms, backend}` dict for the dashboard — keep that shape. + +- **Timeouts are explicit** (default 15s, 8s for the health probe). Keep timeouts + on every outbound call; an upstream hang must not wedge a request. diff --git a/.claude/rules/config-deploy.md b/.claude/rules/config-deploy.md new file mode 100644 index 0000000..d4b24e1 --- /dev/null +++ b/.claude/rules/config-deploy.md @@ -0,0 +1,45 @@ +--- +description: os.getenv + dotenv config, NIKE_ prefix, systemd + docker, fresh-env build order +paths: + - "nike/config.py" + - ".env*" + - "nike.service" + - "Dockerfile" + - "docker-compose.yml" + - "run.py" + - "scripts/**" +--- + +# Config & deployment + +- **Config is `os.getenv` + `python-dotenv`, NOT pydantic-settings.** `config.py` + loads `.env` from the repo root (path-anchored, so it works regardless of CWD) and + reads `NIKE_*` vars into module constants. A new setting = a new + `os.getenv("NIKE_...", default)` **plus** a line in `.env.example`. Do not add a + `Settings` model. + +- **Every env var uses the `NIKE_` prefix** (`red_panda_standards.md`). DB config is + **individual parts** (`NIKE_DB_HOST/PORT/USER/PASSWORD/NAME`) — never a single + `DATABASE_URL`. `SERVER_HOST`/`SERVER_PORT` have no default and will raise if + unset (deliberate — a missing port should fail loudly, not bind somewhere random). + +- **`.env` is gitignored; only `.env.example` is tracked.** Never commit real + secrets. Keep `.env.example` in sync with the `NIKE_*` vars `config.py` reads. + +- **`NIKE_TRUSTED_PROXY` gates `X-Forwarded-*` trust.** `'*'` is safe only because + Nike's port is firewalled to HAProxy; keep that constraint in mind if the topology + changes. + +- **Fresh-environment build order:** `pip install -e .` → `scripts/apply_schema.py` + (provision the PG cache from `schema.sql`) → `cd dashboard && npm install && npm + run build` → `python run.py`. The dashboard build is a hard prerequisite for + serving `/`. Any change to this flow must keep the fresh path working. + +- **Deploy is systemd (`nike.service`) with a Docker option** (`Dockerfile` + + `docker-compose.yml`). Note the checked-in `nike.service` has a **stale + `WorkingDirectory`/`ExecStart`** (`/home/robert/gitea/nike`) — flag it if you + touch the unit; the repo lives at `~/git/nike`. Per the estate venv convention the + runtime venv is `~/env/nike`. + +- **`run.py` is the entry point** (`python run.py` → `nike.server:main` → uvicorn). + Keep it a thin shim. diff --git a/.claude/rules/db.md b/.claude/rules/db.md new file mode 100644 index 0000000..8a01403 --- /dev/null +++ b/.claude/rules/db.md @@ -0,0 +1,41 @@ +--- +description: psycopg2 pool, get_conn contextmanager, cache-table upserts, cache_meta TTL, schema.sql +paths: + - "nike/db.py" + - "schema.sql" + - "scripts/apply_schema.py" +--- + +# PostgreSQL cache layer + +- **`psycopg2` with a `ThreadedConnectionPool`, not SQLAlchemy, not async.** The + pool is created in the FastAPI lifespan (`create_pool`) and closed on shutdown. + New DB work checks out a connection via the **`get_conn()` contextmanager** + (commit-on-success / rollback-on-exception / return-to-pool). Never open a bare + connection for query work. + +- **Exception: `check_connection()` deliberately connects outside the pool** (a + fresh `psycopg2.connect`) so it can report health even when the pool is + unhealthy. Note this is called on every `/ready`, `/metrics`, and `/api/status` — + it's a known hot-path cost (flag if you're optimising), not a pattern to copy for + normal queries. + +- **Cache writes are upserts** (`cache_team`, `cache_league`, `cache_player`, + `cache_event`, and the event sub-tables). They tolerate the raw TheSportsDB dict + shape. Reads (`query_team`, `query_player_by_id`, `query_roster`) return the + DB-row shape (`name`/`id`/`league_name`). Keep both shapes consistent with what + the tools expect (see cache-api / mcp-tools). + +- **`cache_meta` drives TTL freshness** for the volatile-ish rows and powers the + dashboard's "last cache" display. `invalidate_cache("%")` clears it (the Clear + Cache path). Keep meta updates alongside the data upserts they describe. + +- **Schema lives in `schema.sql`**, applied by `scripts/apply_schema.py`. There is + **no migration framework** — a schema change is an edit to `schema.sql` plus a + thought about existing cache DBs (the cache is disposable, so a rebuild is usually + fine, but say so). `get_table_counts` lists the canonical tables; keep it in sync + when you add one. + +- **Parameterise all SQL.** Never f-string user/API values into a query. The one + f-string on a table name in `get_table_counts` is over a hardcoded allowlist — + keep it that way. diff --git a/.claude/rules/fastapi-dashboard.md b/.claude/rules/fastapi-dashboard.md new file mode 100644 index 0000000..09321d8 --- /dev/null +++ b/.claude/rules/fastapi-dashboard.md @@ -0,0 +1,46 @@ +--- +description: dashboard /api routes, tool-runner, metrics middleware, health routes, HAProxy edge, mount order +paths: + - "nike/server.py" +--- + +# FastAPI dashboard, routing & edge + +- **Mount order matters and is load-bearing.** `dashboard.mount("/mcp", _mcp_app)` + goes **before** the SPA `StaticFiles` mount at `/` (which is greedy with + `html=True` SPA fallback). MCP must be mounted first or `/` swallows `/mcp`. The + MCP ASGI app is built early (`mcp.http_app(path="/")`) so its lifespan can be + nested inside the FastAPI lifespan — keep that wiring. + +- **`/api/*` routes are thin façades** over `db`, `sportsdb`, and `mcp`. `/api/status` + (health cards), `/api/tools` (derived from `mcp.list_tools()` — the single source + the dashboard reads), `/api/logs` (the in-memory request log), `/api/cache/invalidate` + (flush both caches), `/api/run` (tool-runner: `mcp.get_tool` → `tool.fn(**args)`), + `/api/v1/telemetry` (browser error sink). Keep `/api/tools` derived, never + hand-maintained. + +- **Prometheus middleware skips health/MCP paths.** `_SKIP_METRICS_PREFIXES = + ("/live","/ready","/metrics","/mcp")` — probes and MCP transport must not inflate + `nike_http_requests_total`. It also normalises UUIDs in paths to `{id}` to bound + label cardinality. Keep both behaviours; add new health-ish prefixes to the skip + list. + +- **`/metrics` is Prometheus exposition, IP-gated in Python.** `_METRICS_ALLOWED_NETS` + = the standard four ranges (`10.10.0.0/24`, `172.16.0.0/12`, `127.0.0.0/8`, + `::1/128`), no auth — HAProxy/Prometheus can't authenticate (`red_panda_standards.md`). + Don't widen it or add app-side auth. + +- **Health routes register both slash forms** (`/live` + `/live/`, `/ready` + + `/ready/`). `/ready` returns 503 when `db.check_connection()` reports down. `/live` + is a bare `{"status": "ok"}`. + +- **HAProxy is the edge.** `uvicorn.run(..., proxy_headers=True, + forwarded_allow_ips=config.TRUSTED_PROXY_IPS, ws="wsproto")`. The write routes + (`/api/run`, `/api/cache/invalidate`) are **not individually authenticated** — + access control is at HAProxy; the code comment says so. If you add a write route, + keep that assumption explicit; if Nike could be exposed directly, it needs auth. + `/api/v1/telemetry` stays unprotected on purpose (`sendBeacon` can't set headers). + +- **Structured logging is re-applied in the lifespan** (`configure_logging` after + `create_pool`) because uvicorn overwrites handlers on `config.load()`. Don't drop + the second call, or JSON logging silently reverts to uvicorn's default. diff --git a/.claude/rules/mcp-tools.md b/.claude/rules/mcp-tools.md new file mode 100644 index 0000000..116f687 --- /dev/null +++ b/.claude/rules/mcp-tools.md @@ -0,0 +1,43 @@ +--- +description: FastMCP tool rules — sync def, string-table returns, premium gating, _log, dual dict shapes +paths: + - "nike/server.py" +--- + +# MCP tools + +- **Tools return plain formatted `str`, not JSON and not Pydantic models.** They + build human-readable ASCII blocks/tables for the LLM (`"=== {name} ==="`, aligned + standings columns, grouped rosters). This is Nike's convention — don't convert a + tool to return JSON or a model to match other estate servers. Match the existing + formatting. + +- **Tools are synchronous `def`.** `sportsdb` (`requests`) and `db` (`psycopg2`) + are blocking; FastMCP runs sync tools in a threadpool. Don't make them `async`. + Keep them plain module-level callables — `/api/run` invokes `tool.fn(**args)` + directly. + +- **Standard tool shape:** `@mcp.tool(annotations=ToolAnnotations(readOnlyHint=True))` + (every tool here is read-only), a docstring written for Claude, resolve the team + via `_resolve_team` if needed, call the `sportsdb` client, format text, and + `_log(tool, args, duration_ms)` before returning. Not-found returns a friendly + string (`"Team '…' not found."`), never an exception. + +- **Premium gating is two things kept in sync:** the runtime check + `config.SPORTSDB_KEY in ('3', '')` (→ return a clear "requires a premium key" + message, or degrade to cached data like `get_roster` does) **and** the + `tags={"premium"}` on the decorator (so `/api/tools` and the dashboard can flag + it). Add both when a new tool needs premium data. + +- **Rows come in two shapes — handle both.** A team/player dict is either a raw + TheSportsDB dict (`strTeam`, `idTeam`, `strLeague`) or a cached DB row (`name`, + `id`, `league_name`). Read with `x.get("strTeam") or x.get("name")`. A cache hit + and a cache miss must render identically — don't assume one shape. + +- **Per-external-call `try/except … pass` is intentional** so a partial API + failure yields partial results. Keep each API call individually guarded; don't + merge them into one try or turn them into hard failures. + +- **When you add/rename a tool, update the hand-written strings too:** the `mcp` + `instructions=` and the `football_analyst()` prompt enumerate tools manually and + do **not** auto-update (unlike `/api/tools`, which derives from `mcp.list_tools()`). diff --git a/.claude/rules/svelte.md b/.claude/rules/svelte.md new file mode 100644 index 0000000..2636b96 --- /dev/null +++ b/.claude/rules/svelte.md @@ -0,0 +1,42 @@ +--- +description: SvelteKit static SPA — derived tool list, telemetry, DaisyUI, no server side +paths: + - "dashboard/src/**" +--- + +# Dashboard — static SPA (SvelteKit 2 / Svelte 5 / Tailwind 4 + DaisyUI 5) + +`npm run check` (svelte-check) is the gate — run it after editing any `.svelte` or +`.ts` and report the result. + +**Static SPA (`adapter-static`) served by FastAPI from `dashboard/build` — no Node +at runtime, no server side:** + +- **No `+page.server.ts`, no `$lib/server`, no form actions, no private `$env`.** + The dashboard only talks to `/api/*` (and `/mcp` is for external MCP clients, not + the dashboard). Secrets live in the backend. +- **`npm run build` is required before FastAPI can serve `/`** — an unbuilt + dashboard makes the index route return a 503 with build instructions. In dev, the + Vite dev server (`:5173`) proxies `/api` and `/mcp` to `:8000`. +- **The tool list is derived, not hardcoded.** `/tools` fetches `/api/tools` (which + the backend builds from `mcp.list_tools()`), so it stays in sync with the + registered MCP tools automatically. Don't hand-maintain a tool list in the + frontend — render whatever `/api/tools` returns, including the `premium` flag. +- **Two routes:** `/` (status: cache/API/MCP health cards, followed teams, tools + list, request log) and `/tools` (interactive runner → `POST /api/run`). Shared + fetch/types live in `src/lib/api.ts` / `src/lib/types.ts` — update types alongside + backend response shape changes. + +**Runes & types:** + +- Runes mode (`$state`, `$derived`, `$props`, `$effect`); `$derived` for computed, + `$effect` only for genuine side effects. Props via `let { … } = $props()` (typed), + no `export let`. +- `onMount` cleanup must be synchronous. + +**Browser telemetry:** report JS errors to `POST /api/v1/telemetry` (the +unprotected sink) so browser failures surface in Loki at WARNING — don't let them +be silently lost (`red_panda_standards.md`). + +Styling: Tailwind 4 + DaisyUI 5 components. No new CSS frameworks. No emojis in +the UI — use an icon set. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..65604fa --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,166 @@ +# CLAUDE.md — Nike + +🐾 A Heluca repo (project #205). Orientation before action; verify before +asserting. The universal standards in `~/.claude/CLAUDE.md` apply; this file +records what is **specific to Nike** and where it diverges from the other +FastAPI/MCP apps in the estate (Hecate, Periplus, Argos). + +## What Nike is + +Nike is a **football (soccer) data platform** — a read-through cache and MCP +front-end over the external **TheSportsDB** API, with a SvelteKit dashboard. One +FastAPI process on `0.0.0.0:{PORT}` hosts three surfaces: + +- **`/`** — the SvelteKit dashboard (`dashboard/build`, static SPA). +- **`/api/*`** — JSON API the dashboard consumes (status, tools, logs, tool-runner, telemetry). +- **`/mcp`** — FastMCP streamable-HTTP endpoint; the eight football tools. + +Data flow: an MCP tool calls `sportsdb.py` (the TheSportsDB client), which is +fronted by a **two-layer cache** — a short-lived in-memory TTL cache in +`sportsdb.py` and a **PostgreSQL cache** in `db.py` for permanent data (teams, +players, leagues, events). The tool formats the result as text for the LLM. + +Shape-wise Nike is a cousin of Hecate/Periplus (FastAPI + FastMCP + SvelteKit + +PostgreSQL + HAProxy + systemd), but three things set it apart and drive most of +the rules below: it is **a cache in front of a third-party API**, its **tools are +synchronous and return plain strings**, and its **config/DB/HTTP stack is the +sync `os.getenv` + `psycopg2` + `requests` generation**, not +pydantic-settings/SQLAlchemy/httpx. + +Modules: `config.py` (settings), `sportsdb.py` (external client + TTL cache), +`db.py` (PG pool + cache helpers), `logging_config.py`, `server.py` (FastAPI + +all MCP tools + routes). `schema.sql` is the cache DDL. Keep it this flat. + +## Red Panda Approval™ + +Canonical rubric: [docs/red_panda_standards.md](docs/red_panda_standards.md). +The README's emoji "under 500 lines 🚨" box is not the standard. Nike is already +one of the more estate-conformant apps — hold that bar: + +1. **Fresh Environment Test** — from a clean checkout: `pip install -e .`, + `scripts/apply_schema.py` provisions the cache DB, `cd dashboard && npm run + build`, then `python run.py` serves dashboard + MCP. The dashboard **must be + built** before FastAPI can serve it (unbuilt → the index route returns a 503 + with build instructions, by design). A schema change must apply cleanly on a + fresh DB. +2. **Elegant Simplicity** — one client function per TheSportsDB endpoint; tools + compose them. Don't inline a second `requests` call or a second cache in a tool. +3. **Observable & Debuggable** — structured JSON logging (`logging_config.py`), + `nike_*` Prometheus metrics, an in-memory request log (`/api/logs`), and the + dashboard status cards. A tool logs its name + args + duration via `_log(...)`. +4. **Consistent Patterns** — new tools look like the eight existing: `@mcp.tool( + annotations=ToolAnnotations(readOnlyHint=True))`, synchronous `def`, resolve + team → call `sportsdb` → format a text table, `_log(...)` before returning. +5. **Actually Works** — against the real TheSportsDB (free key `3` for basic, + premium for the ★ tools). The free key returns limited data; verify premium-gated + tools with an actual premium key or acknowledge you couldn't. + +## The landmines (what breaks if you don't know it) + +- **Tools return plain formatted `str`, not JSON and not Pydantic models.** Nike's + tools build human-readable ASCII tables/blocks for the LLM + (`"=== {team} ==="`, aligned standings columns). This is a *third* return + convention in the estate — don't "fix" a tool to return JSON or a model. Match + the existing formatting style. See [.claude/rules/mcp-tools.md](.claude/rules/mcp-tools.md). + +- **Tools are synchronous `def`** even though the app is async — `sportsdb` + (`requests`) and `db` (`psycopg2`) are blocking. Don't convert tools to `async`. + (FastMCP runs them in a threadpool.) The `/api/run` tool-runner calls + `tool.fn(**args)` directly — keep tools plain callables. + +- **Read-through cache: permanent vs volatile.** Permanent data (teams, players, + leagues, events) is cached in PostgreSQL — `_resolve_team` checks the DB first, + hits the API on a miss, then caches. **Volatile data is always fetched live and + never cached to PG**: standings, livescores, current fixtures. Don't add + standings/livescores to the PG cache, and don't skip the cache for team/player + lookups. See [.claude/rules/cache-api.md](.claude/rules/cache-api.md). + +- **Rows come in two shapes.** A team/player dict may be a **raw TheSportsDB dict** + (`strTeam`, `idTeam`, `strLeague`) or a **cached DB row** (`name`, `id`, + `league_name`). The tools read both with `x.get("strTeam") or x.get("name")`. + Preserve that dual-shape handling when you touch formatting — a cache hit and a + cache miss must render identically. + +- **Free vs premium key gate.** `config.SPORTSDB_KEY in ('3', '')` means "free + key." Premium-only tools (`get_match_detail`, `get_livescores`, V2 squad lists) + check this and return a clear "requires a premium key" message, or degrade to + cached data (`get_roster` falls back to the DB roster). Premium tools are also + `tags={"premium"}` so the dashboard can flag them. Keep both the runtime check + and the tag in sync when adding a premium tool. + +- **Per-API-call `try/except … pass` is deliberate.** Each external call inside a + tool is individually guarded so a partial failure yields partial results, not a + dead tool. Don't "clean this up" into one big try or into hard failures — a + missing fixtures endpoint shouldn't blank the whole team page. (It's the one + place broad `except` is correct; elsewhere, catch narrowly and log.) + +- **The tool catalogue is derived, single-source.** `/api/tools` and the MCP + `instructions`/`football_analyst` prompt all enumerate the tools; the dashboard + reads `/api/tools` (from `mcp.list_tools()`). When you add/rename a tool, the + catalogue updates itself — but **update the human-written `instructions` string + and the `football_analyst` prompt by hand**, they don't. + +## Observability & edge (the estate-conformant parts — keep them) + +- **`/metrics` is Prometheus exposition, IP-restricted in Python.** + `_METRICS_ALLOWED_NETS` = `10.10.0.0/24`, `172.16.0.0/12`, `127.0.0.0/8`, + `::1/128` (the standard four ranges) — no auth. The middleware **skips** + `/live`, `/ready`, `/metrics`, `/mcp` from request metrics (health probes must + not inflate counters — `red_panda_standards.md`). Keep both. +- **Health routes register both slash forms** (`/live` + `/live/`, `/ready` + + `/ready/`). `/ready` checks the DB and returns 503 if it's down. +- **Structured logging is re-applied inside the lifespan** (`configure_logging` + again after `db.create_pool`) because uvicorn's `config.load()` overwrites + handlers — the estate FastAPI logging landmine. Don't remove the second call. +- **HAProxy is the edge.** `proxy_headers=True` + `forwarded_allow_ips= + TRUSTED_PROXY_IPS`. The `/api/*` write routes are **not individually + authenticated** — access control is at HAProxy. If Nike is ever exposed directly, + those write endpoints (`/api/run`, `/api/cache/invalidate`) need protection. The + telemetry endpoint (`/api/v1/telemetry`) is intentionally unprotected + (`sendBeacon` can't set headers). + +## Config & deploy + +- **Config is `os.getenv` + `python-dotenv`, NOT pydantic-settings.** `config.py` + loads `.env` from the repo root and reads `NIKE_*` vars into module constants. + A new knob = a new `os.getenv("NIKE_...", default)` **plus** a line in + `.env.example`. Do **not** introduce a `Settings` model. `.env` is gitignored + (correctly — only `.env.example` is tracked); never commit real secrets. +- **DB access is `psycopg2` with a `ThreadedConnectionPool`**, checked out via the + `get_conn()` contextmanager (commit-on-success / rollback-on-exception). No + SQLAlchemy, no async DB. New queries use `get_conn()` and parameterised SQL. +- **Ships as systemd** (`nike.service`) and also has a `Dockerfile` + + `docker-compose.yml`. See [.claude/rules/config-deploy.md](.claude/rules/config-deploy.md) + for the drift in the checked-in unit. + +## README & code drift — trust the code + +- The health-check prose is Django/K8s boilerplate: `/ready/` does not "validate + cache connectivity" and `/live/` has nothing to do with "Django responding." + `/ready` checks the PG cache DB; `/live` is a bare 200. +- The emoji file-size rubric is not the real standard. + +When README and code disagree, the code wins. Fixing the README is worthwhile — +flag it, don't fold it in. + +## Known liabilities (flag, don't silently fix) + +Pre-existing; surface them, let Robert decide: + +- **`db.check_connection()` opens a fresh `psycopg2.connect` outside the pool** on + every call — and `/ready`, `/metrics`, and `/api/status` all call it. The pool's + hottest paths bypass the pool. A pooled `SELECT 1` would be cheaper. +- **`nike.service` has a stale `WorkingDirectory`/`ExecStart`** pointing at + `/home/robert/gitea/nike` (the repo lives at `~/git/nike`) — the checked-in unit + wouldn't start as-is. +- **`.DS_Store` is tracked** at the repo root and in `nike/`. macOS cruft — should + be gitignored and removed. + +## Path-scoped rules (`.claude/rules/`) + +- `mcp-tools.md` — sync tools, string-table returns, premium gating, `_log`, dual dict shapes. +- `cache-api.md` — read-through pattern, permanent-vs-volatile, TTL + PG layers, per-call try/except. +- `db.md` — psycopg2 pool, get_conn contextmanager, cache-table upserts, schema.sql. +- `fastapi-dashboard.md` — /api routes, tool-runner, metrics middleware, health routes, HAProxy edge, mount order. +- `svelte.md` — static SPA, derived tool list, telemetry, DaisyUI, no server side. +- `config-deploy.md` — os.getenv config, NIKE_ prefix, systemd + docker, fresh-env build order.