palladium/CLAUDE.md

# CLAUDE.md — Palladium (library of master notebooks)

🐾 Palladium is a **library of master notebooks** for consulting delivery: Mercury-served
Jupyter deliverables a client watches on a live call. "Library" means the copies in this
repo are **masters** — client-clean, reusable, maintained — it does **not** mean drafts.
A master becomes a client deliverable only as an **engagement copy made outside this
repo**, where it acquires client data and becomes confidential. The notebook a client
can open and trust is the whole product.

## 🐾 Red Panda Approval™ — for a consulting master library

Don't satisfy a checklist — satisfy the red pandas. Ask of each change: *does this
earn approval?*

1. **Rebuildable From Nothing** — inside any master: `python -m venv .venv` →
   `pip install -e ".[dev]"` → `pytest` → headless gate → `mercury --working-dir .`
   brings up the deliverable with no manual step. And a master **copied out of the
   repo runs standalone** — self-contained engine, own `staging.py`, no import back
   into Palladium. The ability to stand a workshop up on short notice is the product.
2. **Elegant Simplicity** — these are workshop tools, not enterprise platforms; the
   obvious solution, done well. A content cell should read like the survey it came
   from, not like a framework.
3. **Observable & Debuggable** — a failing gate names the number that drifted;
   diagnostics flow through `backstage()`, never onto the client-facing stage; the
   exports show the whole session state so a failure is diagnosable after the call.
4. **Consistent Patterns** — the three-layer contract, widget-pairs, tagged cells,
   per-master engine packages. Match the pattern docs, not personal taste.
5. **Actually Works** — "pytest was green" is not "the board renders on the stage
   with the engine's numbers." The gate under headless `nbconvert --execute` plus an
   actual Mercury render are the proof.

Criteria 1 and 5 are **externally verifiable** — the rebuild works or it doesn't; the
stage renders or it doesn't. Verify them, don't assert them. Criteria 2–4 are
judgement calls: when in doubt, match what the repo already does rather than grading
your own elegance.

> If a paw print isn't leading the response, the rest of this file probably isn't
> being honoured either. Lead with one. 🐾

---

## The three-layer contract

Every master separates three concerns; the split IS the architecture:

| Layer | Audience | Holds | Never holds |
|---|---|---|---|
| **Mercury** (the stage) | the client | the polished board/case, sidebar controls | diagnostics, question scripts, internals |
| **Notebook cells** (backstage) | the consultant | **content** (topic banks, prompts) and **client data** (engagement facts, spend, headcount) in **tagged cells**; presentation code | math |
| **Python modules** (the engine) | tests + the notebook | reusable logic, schema, calculations — typed, `mypy --strict`, pinned | **content — NEVER** |

The rule that has teeth: **content never lives in `.py`** — not as constants, not as a
notebook generator's string blocks. The notebook is the document the consultant edits;
hiding its content in importable modules defeats the point of a notebook. Mechanics:
[docs/Mercury_Notebook_Pattern_V1-00.md](docs/Mercury_Notebook_Pattern_V1-00.md);
per-type contracts: [docs/Assessment_Pattern_V1-00.md](docs/Assessment_Pattern_V1-00.md)
and [docs/Study_Pattern_V1-00.md](docs/Study_Pattern_V1-00.md).

## Taxonomy

- **Study** (`studies/YYYYMM_…`, dated) — reproduction of a dated base document
  (Forrester TEI or similar); verbatim anchor + client overlay; the date is the
  publication's, so it stays.
- **Assessment** (`assessments/Instrument_Name`, undated) — reusable workshop
  instrument; a living master. Reference implementation:
  `assessments/CX_Discovery_Workshop/`.
- **Engagement copy** (`YYYYMM_Client_Instrument`, stamped at copy-time) — a master
  copied OUT of this repo for a client; confidential; never merges back.
- **`template/MercuryNotebook/`** — copy-me scaffold for new studies (py-engine model
  only — see Known liabilities).
- **`core/`** — the shared Athena toolkit; masters do NOT import it.

## Before working in a master: check where you are

**Look at the path.** Which master's root are you under? Each master has its **own
venv** — `.venv` inside the master directory, provisioned by `pip install -e ".[dev]"`
there; the repo-root `.venv` (from `make setup`) serves only `core/` and the notebook
structural suite. Running a master's pytest from the wrong venv is the classic
"missing module" ghost.

- `MERCURY_CONFIG_DIR` present in the environment = **the stage is live** (a client
  may be looking). Its presence is the stage signal `staging.py` keys off — never set
  it manually except to simulate the stage in a test.
- Only masters live here. **If a notebook in this repo contains a real client's name,
  something is wrong** — stop and flag it (see Confidentiality).

## Risk tier: CONSULTING MASTERS — what you may run

**Free** — run without asking:
`pytest`, `mypy`, `ruff`, `make test`, `make check-notebooks`,
`jupyter nbconvert --execute` on a master, `mercury --working-dir .` locally,
`python scripts/export_report.py`, and any read-only git.

**Show first** — produce the diff/output, present it, wait for a human "go":
content edits to `topic-bank`/`engagement-data` cells or anchor-adjacent wording;
re-pinning a gate or test after a content/engine change (show the pin diff and the
KPI moves honestly); moving or renaming a master; `.gitignore` changes; `git commit`.

**Forbidden without explicit go-ahead:**
`git push`; editing `*_VERBATIM` anchors; committing any client document (SOW, quote,
NDA'd vendor deck); putting a client's name or data into a master; history rewrites;
deleting a study or assessment.

> The one clause that always applies: *"the user asked me to update the workshop" is
> not explicit go-ahead for a Show-first change.* Explicit go-ahead is the user seeing
> the specific diff and saying yes to **that**.

---

## Conventions (always-on)

### Done means the stage shows the engine's numbers, not that the file saved

The cell is a wish; the executed notebook is the fact. An edit that is syntactically
perfect has changed **nothing** until the notebook re-runs — and the committed outputs
now lie about the deliverable.

1. **Pin it** — `pytest` + `mypy` in the master's venv.
2. **Execute it** — `jupyter nbconvert --to notebook --execute --inplace
   notebooks/*.ipynb`; the gate passing headless is the study's smoke test.
3. **Export it** — `python scripts/export_report.py`; the `.md` must carry the
   appendix and its final JSON block.
4. **Read it back from the real system** — serve with Mercury (or open the exported
   HTML) and look at the board; then check what depends on the change: test pins,
   gate pins, README counts, pattern docs, the export JSON. A master is a *chain* —
   a renamed content key breaks widgets, notes, and the export two hops downstream.

### Confidentiality — masters stay client-clean

There is no Palladium without this rule; it is what makes the library shareable.

- Masters carry **placeholder** engagement data, published or synthetic numbers, and
  no client statements. Client personalization happens in an **engagement copy** made
  outside the repo (copy → rename `YYYYMM_Client_…` → fresh venv → fill
  `engagement-data` → verify; canonical checklist in the Assessment Pattern) —
  **before** any client data is entered, and the copy never merges back.
- **Never commit a client document** — SOWs, quotes, pricing decks, NDA'd vendor
  material. `.gitignore` blocks binary documents under every master's `docs/` and
  anything matching `*SOW*`; that is a guardrail, not permission — the rule is the
  rule even where the pattern has a hole. The ONE exception: a Study's **published,
  public** source PDF, added deliberately with `git add -f` and named in the commit
  message.
- Improvements discovered on an engagement come back to the master as **clean edits**
  (content/logic only, client facts stripped).

### Notebooks

- Widget-pair rule, gate cell, stage/backstage, data appendix, packaging: the
  [Mercury pattern](docs/Mercury_Notebook_Pattern_V1-00.md) is the contract — read it
  before editing any notebook.
- Tagged cells are the consultant's surface: `topic-bank` (content),
  `engagement-data` (client facts; placeholders in masters), `presentation`, `gate`,
  `data-appendix`. Tags are load-bearing — tests, the structural suite, and the
  export pipeline all find cells by tag.
- Content-cell keys are **stable identities** (widgets, notes, exports key off them);
  a title edit renames its sidebar widget label, which resets that widget's state
  mid-session.
- `assessments/CX_AI_Diagnostic/notebooks/diagnostic.ipynb` is **generated** by its
  `scripts/build_notebook.py` — never hand-edit it (and don't "fix" it to
  notebook-first casually; that's a recorded redesign).

### Python

- One venv per master; `pip install -e ".[dev]"` provisions everything (whole
  toolchain as core deps — never a `requirements.txt` in a master).
- Engines are `mypy --strict` clean; new logic lands as engine code + pins **before**
  the notebook section that renders it.
- `staging.py` is copied per master (self-containment beats DRY here — an engagement
  copy must run alone). The mypy-strict variant in
  `assessments/CX_Discovery_Workshop/discoverylib/staging.py` (with `backstage_md`)
  is the canonical form for new masters.

## Always-on anti-patterns

- **Content:** never move workshop content into `.py` — including generators that
  hold cell sources as strings.
- **Client data:** never in a master; never in a commit; a client name in
  `assessments/` or `template/` means the copy-out step was skipped.
- **Anchors:** never edit `*_VERBATIM` — overlay corrections, don't rewrite the
  record.
- **Widgets:** never read `.value` in the defining cell; widget cells emit no output.
- **Surfaces:** never build a parallel UI (Streamlit twin, second app) — the notebook
  is the surface.
- **Exports:** never commit generated exports; `exports/` is gitignored on purpose.
- **Emptiness pins:** never assert that engagement placeholders are empty — the
  filled engagement copy must stay green.

---

## Environment

- Repo root: `/home/robert/notebook/git/palladium` · remote:
  `ssh://git@git.helu.ca:22022/r/palladium.git` (Robert's Gitea) · branch `main`.
- Root venv: `make setup` (core/Athena layer + the notebook structural suite;
  `make test`, `make check-notebooks`). One venv **per master** besides it.
- Python ≥3.10 per master (`CX_AI_Diagnostic` pins ≥3.11; root core is ≥3.11);
  Mercury 3.2.x; per-master mypy is strict; repo-wide lint is ruff (root
  `pyproject.toml`).
- Athena onboarding: `00_setup.ipynb` (writes `.env`; sandbox
  `https://athena.ouranos.helu.ca`).

## Known liabilities (flag, don't silently fix)

If you find a known non-compliant choice, raise it rather than quietly fixing it or
quietly leaving it. Live ones worth knowing:

- **`studies/202607_CTM_GenesysCX/` is a real-client engagement study inside the
  library** — committed confidential vendor/client PDFs in `docs/` (also in git
  history), plus a signed SOW PDF sitting on disk untracked-and-ignored (the `*SOW*`
  ignore hides it from `git status` — it is still there). Grandfathered: do not add
  more material, do not use it as precedent; extraction to an engagement archive is
  a recorded follow-up. Two of its notebooks are dirty in the working tree —
  pre-existing, leave them.
- **`assessments/CX_AI_Diagnostic/` pre-dates the notebook-first model** — generated
  notebook, content in YAML configs. Redesign pending; until then its shape is
  intentional. Its `mypy` is also not clean in a fresh venv (11 pre-existing errors:
  missing pandas/PyYAML/plotly stubs, unannotated functions) even though its
  `pyproject.toml` declares strict mode — pytest and the headless execute are green;
  fix the typing with the redesign, not piecemeal.
- **TEI twins + CTM pre-date the tagged-cell taxonomy** — the structural suite
  grandfathers them by name in `tests/nbcheck.py`, each with its reason.
- **`template/MercuryNotebook/` encodes only the py-engine model** (and its
  `staging.py` lacks the mypy-strict `backstage_md` variant); rework is a recorded
  follow-up. Its `exports/*.{html,md}` are tracked — predates the exports rule.
- **`docs/brand.md` references a `brand_dark.md` that does not exist.**
- **`core.bootstrap.init(study=…)` imports `studies.<slug>.config`** — vestigial
  (no study ships a `config.py`; `studies/__init__.py` exists to serve it). Don't
  remove the `__init__.py` without retiring that path.

## Identified opportunities (recorded, not built)

- A private installable Palladium package on the Gitea (CI exists there) — would
  replace per-master `staging.py` copies; revisit when the duplication bites.
- A copy-out helper script (`scripts/new_engagement.py`) — after the manual checklist
  has been exercised a few times.
- CTM extraction to a confidential engagement archive.
- TEI + AI Diagnostic redesigns to the notebook-first model (each moves its notebook
  from GRANDFATHERED to NOTEBOOK_FIRST in `tests/nbcheck.py`).
- Template rework: an Assessment template derived from the Discovery reference;
  staging.py mypy-strict retrofit across masters.

## Reference

**Read the pattern for your master type before working in it — they hold the detail
this file summarises:**

- [docs/Assessment_Pattern_V1-00.md](docs/Assessment_Pattern_V1-00.md) — assessments:
  notebook-first content, engagement-data cell, copy-out checklist
- [docs/Study_Pattern_V1-00.md](docs/Study_Pattern_V1-00.md) — studies: verbatim
  anchor + overlay, reproduction gate, published-PDF exception

Everything else:

- [docs/Mercury_Notebook_Pattern_V1-00.md](docs/Mercury_Notebook_Pattern_V1-00.md) —
  the shared mechanics (reactivity, gate, staging, appendix, packaging)
- [docs/Notebook_Review_Prompt_V1-00.md](docs/Notebook_Review_Prompt_V1-00.md) — the
  LLM review; run as `/review-notebook <path>`
- [docs/brand.md](docs/brand.md) — NTT DATA palette for stages and exports
- [docs/Athena_TEI.md](docs/Athena_TEI.md) + `docs/Athena API.yaml` — the core/Athena
  layer
- [README.md](README.md) — taxonomy, quick start, engagement workflow