palladium/docs/Assessment_Pattern_V1-00.md

# Assessment Pattern v1.0.0

How Palladium ships **Assessments**: reusable workshop instruments (diagnostics,
discovery workshops, maturity assessments) built as Mercury notebook deliverables.
An Assessment is a **master** — client-clean, undated, maintained in this repo —
that becomes a client deliverable only as an **engagement copy** made outside the
repo. Reference implementation:
[`assessments/CX_Discovery_Workshop/`](../assessments/CX_Discovery_Workshop/).

## 🐾 Red Panda Approval™

This pattern follows Red Panda Approval standards (see `CLAUDE.md` for the rubric).

**Audience note:** written to be loaded whole by an LLM agent building or modifying
an assessment. This document holds what assessments ADD to the
[Mercury Notebook Deliverable Pattern](Mercury_Notebook_Pattern_V1-00.md) — the
mechanics every master obeys (reactivity contract, gate, stage/backstage, packaging,
appendix) live there and are **not restated here**. `CLAUDE.md` is the always-on
contract and takes precedence where documents disagree.

---

## What an Assessment is

| | Study | **Assessment** |
|---|---|---|
| Reproduces | a dated publication / engagement | — (it IS the instrument) |
| Naming | `YYYYMM_…` (dated) | `Instrument_Name` (undated, living) |
| "Numbers" | dollars (NPV/ROI/payback) | qualitative state (status, scores, coverage) |
| Client data | overlay on a published anchor | the `engagement-data` cell, filled per copy |
| Lifecycle | frozen once delivered | master evolves; copies freeze |

The **three-layer contract** (from `CLAUDE.md`), which assessments realize most
completely:

1. **Mercury** — the polished client-facing stage a workshop runs on.
2. **Jupyter cells** — the consultant's surface: workshop **content** and **client
   data** are edited in tagged notebook cells, never in `.py`.
3. **Python modules** — the per-assessment engine: schema + session logic, reusable,
   typed (`mypy --strict`), and pinned by tests.

## Master → engagement copy lifecycle

The master in `assessments/` MUST stay client-clean: placeholder engagement data,
generic content, nothing a client said. Running one for a client means **copying it
out** (checklist below); the copy acquires client data, becomes confidential, and
never merges back. Content improvements discovered on an engagement are hand-carried
back to the master as clean edits.

---

## Anatomy

```
assessments/<Instrument_Name>/
├── notebooks/<instrument>.ipynb   # THE deliverable — content + data + presentation
├── <instrumentlib>/               # the engine — CODE ONLY, no content
│   ├── session.py                 #   schema, vocabulary, session logic, export payload
│   └── staging.py                 #   stage/backstage (+ backstage_md) — self-contained copy
├── tests/                         # content/engagement pins (read from the notebook) + engine pins
├── scripts/export_report.py       # execute once → exports/*.html + LLM-ready *.md
├── docs/                          # source material (.md/.png only — see .gitignore)
├── exports/                       # generated — never committed
├── config.toml                    # Mercury app shell (brand theme, welcome)
└── pyproject.toml                 # whole toolchain as core deps (Mercury Pattern §7)
```

Masters are **runtime-self-contained**: the engine package (including its own
`staging.py` copy) lives in the master, so an engagement copy runs standalone with no
dependency back on Palladium. Repo-level machinery (the structural test suite, the
review prompt) is *maintenance* tooling — an engagement copy never needs it.

---

## The notebook-first content model

Everything a consultant edits lives in **tagged cells** of the deliverable notebook.
The tag taxonomy (enforced by `tests/test_notebooks.py` from the repo root):

| Tag | Count | What it holds | Position rule |
|---|---|---|---|
| `topic-bank` | exactly 1 | the instrument's content: topics, prompts, scoring bands… | above the widgets |
| `engagement-data` | exactly 1 | client facts for THIS session (placeholders in the master) | above the widgets |
| `presentation` | any | setup, widget construction, stage rendering | — |
| `gate` | exactly 1 | the verification gate (Mercury Pattern §4) | after the analytics |
| `data-appendix` | exactly 1 | the machine-readable session record | **last code cell** |

Rules for content cells (`topic-bank`, `engagement-data`):

- **Self-contained** — no magics, no shell escapes, imports only from the assessment's
  own engine. *Why:* tests and scripts exec the cell by its tag without a kernel; a
  `%magic` breaks that contract.
- **Stable keys** — slugs (`key="channels"`) are identities that widgets, captured
  notes, and the export JSON key off. NEVER renumber or rename casually; a title edit
  renames its sidebar widget label, which resets that widget's state mid-session
  (Mercury's widget cache is label-keyed).
- **Verbatim anchor discipline** — when the content structures a source document
  (a survey, a question bank), wording tracks the source; the cell's own comment block
  carries the editing rules so they travel with the content.
- **Above the widgets** — both content cells sit above the widget cell, so Mercury
  never re-runs them on a sidebar change (it re-runs only cells BELOW the changed
  widget).

## The engagement-data cell contract

```python
# ── Engagement data — EDIT PER ENGAGEMENT (tagged: engagement-data) ──
ENGAGEMENT: dict[str, object] = {
    "client": "",           # e.g. "Acme Corp" — shown on the stage board header
    "workshop_date": "",    # ISO date, e.g. "2026-08-12"
    "facilitator": "",
    "attendees": (),        # tuple of "Name — role" strings
}
```

- The **master ships placeholders** (empty strings / empty tuple); the engagement copy
  ships real values. Everything that checks this cell — the gate, `pytest`, the repo
  suite — pins **shape only** (key set, types), NEVER emptiness or a value, so master
  and filled copy both stay green.
- Where the values flow: the stage board header (rendered only when `client` is
  non-empty, so the master's stage is unchanged), the export preamble, and the
  appendix JSON `meta`.
- Extend the dict per instrument (e.g. `current_spend`, `agent_count` for a sizing
  assessment) — client data belongs HERE, in the cell, not in the engine and not
  hardcoded in presentation cells.

## Engine parameterization

The engine package holds schema + logic only, and every session-level function takes
the content as its **first argument** — the engine never imports content:

```python
SESSION = build_session(TOPICS, STATUS_BY_TOPIC, NOTES_BY_TOPIC, DONE_SUBTOPICS)
```

*Why:* content stays in the notebook (editable), logic stays testable — tests feed the
real bank from the tagged cell, or synthetic banks for edge cases. Widgets are built by
a **runtime loop over the content** (with per-widget distinct labels — Mercury's cache
is label-keyed), so a content edit propagates to sidebar, board, script, gate, and
export with zero further wiring.

---

## Testing an Assessment

All layers of the Mercury Pattern's testing section apply; assessments add the
**exec-by-tag** recipe for content:

```python
# tests/conftest.py — the content fixtures read the NOTEBOOK, no kernel
def tagged_cell_ns(tag: str) -> dict[str, Any]:
    nb = nbformat.read(NOTEBOOK, as_version=4)
    cells = [c for c in nb.cells if tag in c.metadata.get("tags", [])]
    assert len(cells) == 1
    ns: dict[str, Any] = {}
    exec(compile(cells[0].source, f"{NOTEBOOK.name} [{tag}]", "exec"), ns)
    return ns
```

- **Content pins** — counts, key order, uniqueness, well-formedness of the bank
  (`tests/test_topics.py` in the reference). Re-pin deliberately on content changes.
- **Engagement pins** — shape of `ENGAGEMENT`, its flow into the export meta, and the
  cell's above-the-widgets position (`tests/test_engagement.py`).
- **Engine pins** — session logic against hand-checked values (`tests/test_session.py`).
- **Staging** — `backstage`/`backstage_md` render only off stage.
- **Gate + repo suite** — the in-notebook gate re-counts the bank on every execution;
  `make check-notebooks` (repo root) enforces the tag taxonomy from outside.

## Export handoff (the LLM-input artifact)

`scripts/export_report.py` — execute ONCE, convert twice, post-process:

1. `nbconvert --execute` to a temp copy (never combine `--execute` with tag-stripping
   in one call — a cell could be removed before it runs).
2. **HTML** from the executed copy — full presentation, human review.
3. **Markdown** from the executed copy with
   `TagRemovePreprocessor.remove_cell_tags={"presentation"}` — the LLM artifact keeps
   content, script, gate, and appendix; drops setup/widget/board source and widget-repr
   noise.
4. Prepend a generated **preamble**: what the document is, the engagement line (read
   from the `engagement-data` cell via nbformat), how to read it, and the instruction
   that the final fenced JSON block is the source of truth.

The appendix cell emits its table + JSON through **`backstage_md()`** (one
`text/markdown` display), so the `.md` export carries a clean fenced ```json block as
the last thing in the file — not an indented text blob.

## Stage polish

`config.toml` carries the brand ([`docs/brand.md`](brand.md)) — the reference's file
is the copy-me. Beyond the palette: name the deliverable notebook in
`[welcome] message`, keep the footer attributed, set the brand state colors
(`success_color`, `warning_color`, `danger_color`), and keep **web-safe font stacks**
(no `font_url` — a client screen must not depend on a network font fetch). Engine
palettes may deviate deliberately (the reference's on-board complete-green `#00a34c`
is darker than brand Success `#00cb5d` for contrast on white cards) — note such
choices where they live.

---

## Copy-out checklist (canonical)

Running an assessment for a client — the master never touches client data:

1. **Copy** the master directory out of Palladium to your engagement location
   (it is self-contained — no repo machinery comes along or is needed).
2. **Rename** it `YYYYMM_Client_Instrument` (e.g. `202608_Acme_CX_Discovery`).
3. **Provision**: `python -m venv .venv && .venv/bin/pip install -e ".[dev]"`
   (editable installs pin absolute paths — a copied venv is broken; always recreate).
4. **Fill the `engagement-data` cell** — and only that cell — with the client facts.
5. **Verify**: `pytest` and the headless gate
   (`jupyter nbconvert --to notebook --execute --inplace notebooks/*.ipynb`) — both
   stay green by design (shape-only pins).
6. The copy is now **confidential**: it never merges back, never returns to this
   repo. Improvements you discover on the engagement are hand-carried to the master
   as clean, client-free edits.

## Assessment anti-patterns

- ❌ **Facilitator prose in markdown cells** — markdown ALWAYS renders on the
  Mercury stage; run-books, "backstage" explanations, and section headings for
  hidden sections all leak to the client. The stage gets a client-neutral title
  and the board; facilitator orientation goes through `backstage_md()` in a
  `presentation`-tagged code cell (JupyterLab and the HTML export show it; the
  stage and the LLM `.md` export don't).
- ❌ **Client data in a master** — a client's name in `assessments/` means the
  copy-out step was skipped; scrub and move it out.
- ❌ **Content in `.py`** — topic banks, prompts, survey text are content; content
  lives in tagged cells. (This includes notebook *generators* that hold content as
  Python strings — the same mistake one indirection deeper.)
- ❌ **Untagged content cells** — the repo suite, the tests, and the export pipeline
  all find content by tag; an untagged content cell is invisible to all three.
- ❌ **Emptiness pins** — asserting `ENGAGEMENT["client"] == ""` breaks every filled
  engagement copy; pin shape, never values.
- ❌ **Widget labels derived from unstable text** — labels come from content titles;
  editing a title mid-session resets that widget. Edit titles between sessions.