feat: add master notebook library scaffolding and review tooling
Add CLAUDE.md defining the Palladium master notebook conventions and Red Panda Approval criteria, plus a review-notebook slash command for LLM-driven notebook review. Expand .gitignore to block client/engagement documents and generated exports, keeping masters client-clean while allowing text/image sources. Normalize slider widget numeric values from floats to integers in notebook JSON.
This commit is contained in:
236
docs/Assessment_Pattern_V1-00.md
Normal file
236
docs/Assessment_Pattern_V1-00.md
Normal file
@@ -0,0 +1,236 @@
|
||||
# Assessment Pattern v1.0.0
|
||||
|
||||
How Palladium ships **Assessments**: reusable workshop instruments (diagnostics,
|
||||
discovery workshops, maturity assessments) built as Mercury notebook deliverables.
|
||||
An Assessment is a **master** — client-clean, undated, maintained in this repo —
|
||||
that becomes a client deliverable only as an **engagement copy** made outside the
|
||||
repo. Reference implementation:
|
||||
[`assessments/CX_Discovery_Workshop/`](../assessments/CX_Discovery_Workshop/).
|
||||
|
||||
## 🐾 Red Panda Approval™
|
||||
|
||||
This pattern follows Red Panda Approval standards (see `CLAUDE.md` for the rubric).
|
||||
|
||||
**Audience note:** written to be loaded whole by an LLM agent building or modifying
|
||||
an assessment. This document holds what assessments ADD to the
|
||||
[Mercury Notebook Deliverable Pattern](Mercury_Notebook_Pattern_V1-00.md) — the
|
||||
mechanics every master obeys (reactivity contract, gate, stage/backstage, packaging,
|
||||
appendix) live there and are **not restated here**. `CLAUDE.md` is the always-on
|
||||
contract and takes precedence where documents disagree.
|
||||
|
||||
---
|
||||
|
||||
## What an Assessment is
|
||||
|
||||
| | Study | **Assessment** |
|
||||
|---|---|---|
|
||||
| Reproduces | a dated publication / engagement | — (it IS the instrument) |
|
||||
| Naming | `YYYYMM_…` (dated) | `Instrument_Name` (undated, living) |
|
||||
| "Numbers" | dollars (NPV/ROI/payback) | qualitative state (status, scores, coverage) |
|
||||
| Client data | overlay on a published anchor | the `engagement-data` cell, filled per copy |
|
||||
| Lifecycle | frozen once delivered | master evolves; copies freeze |
|
||||
|
||||
The **three-layer contract** (from `CLAUDE.md`), which assessments realize most
|
||||
completely:
|
||||
|
||||
1. **Mercury** — the polished client-facing stage a workshop runs on.
|
||||
2. **Jupyter cells** — the consultant's surface: workshop **content** and **client
|
||||
data** are edited in tagged notebook cells, never in `.py`.
|
||||
3. **Python modules** — the per-assessment engine: schema + session logic, reusable,
|
||||
typed (`mypy --strict`), and pinned by tests.
|
||||
|
||||
## Master → engagement copy lifecycle
|
||||
|
||||
The master in `assessments/` MUST stay client-clean: placeholder engagement data,
|
||||
generic content, nothing a client said. Running one for a client means **copying it
|
||||
out** (checklist below); the copy acquires client data, becomes confidential, and
|
||||
never merges back. Content improvements discovered on an engagement are hand-carried
|
||||
back to the master as clean edits.
|
||||
|
||||
---
|
||||
|
||||
## Anatomy
|
||||
|
||||
```
|
||||
assessments/<Instrument_Name>/
|
||||
├── notebooks/<instrument>.ipynb # THE deliverable — content + data + presentation
|
||||
├── <instrumentlib>/ # the engine — CODE ONLY, no content
|
||||
│ ├── session.py # schema, vocabulary, session logic, export payload
|
||||
│ └── staging.py # stage/backstage (+ backstage_md) — self-contained copy
|
||||
├── tests/ # content/engagement pins (read from the notebook) + engine pins
|
||||
├── scripts/export_report.py # execute once → exports/*.html + LLM-ready *.md
|
||||
├── docs/ # source material (.md/.png only — see .gitignore)
|
||||
├── exports/ # generated — never committed
|
||||
├── config.toml # Mercury app shell (brand theme, welcome)
|
||||
└── pyproject.toml # whole toolchain as core deps (Mercury Pattern §7)
|
||||
```
|
||||
|
||||
Masters are **runtime-self-contained**: the engine package (including its own
|
||||
`staging.py` copy) lives in the master, so an engagement copy runs standalone with no
|
||||
dependency back on Palladium. Repo-level machinery (the structural test suite, the
|
||||
review prompt) is *maintenance* tooling — an engagement copy never needs it.
|
||||
|
||||
---
|
||||
|
||||
## The notebook-first content model
|
||||
|
||||
Everything a consultant edits lives in **tagged cells** of the deliverable notebook.
|
||||
The tag taxonomy (enforced by `tests/test_notebooks.py` from the repo root):
|
||||
|
||||
| Tag | Count | What it holds | Position rule |
|
||||
|---|---|---|---|
|
||||
| `topic-bank` | exactly 1 | the instrument's content: topics, prompts, scoring bands… | above the widgets |
|
||||
| `engagement-data` | exactly 1 | client facts for THIS session (placeholders in the master) | above the widgets |
|
||||
| `presentation` | any | setup, widget construction, stage rendering | — |
|
||||
| `gate` | exactly 1 | the verification gate (Mercury Pattern §4) | after the analytics |
|
||||
| `data-appendix` | exactly 1 | the machine-readable session record | **last code cell** |
|
||||
|
||||
Rules for content cells (`topic-bank`, `engagement-data`):
|
||||
|
||||
- **Self-contained** — no magics, no shell escapes, imports only from the assessment's
|
||||
own engine. *Why:* tests and scripts exec the cell by its tag without a kernel; a
|
||||
`%magic` breaks that contract.
|
||||
- **Stable keys** — slugs (`key="channels"`) are identities that widgets, captured
|
||||
notes, and the export JSON key off. NEVER renumber or rename casually; a title edit
|
||||
renames its sidebar widget label, which resets that widget's state mid-session
|
||||
(Mercury's widget cache is label-keyed).
|
||||
- **Verbatim anchor discipline** — when the content structures a source document
|
||||
(a survey, a question bank), wording tracks the source; the cell's own comment block
|
||||
carries the editing rules so they travel with the content.
|
||||
- **Above the widgets** — both content cells sit above the widget cell, so Mercury
|
||||
never re-runs them on a sidebar change (it re-runs only cells BELOW the changed
|
||||
widget).
|
||||
|
||||
## The engagement-data cell contract
|
||||
|
||||
```python
|
||||
# ── Engagement data — EDIT PER ENGAGEMENT (tagged: engagement-data) ──
|
||||
ENGAGEMENT: dict[str, object] = {
|
||||
"client": "", # e.g. "Acme Corp" — shown on the stage board header
|
||||
"workshop_date": "", # ISO date, e.g. "2026-08-12"
|
||||
"facilitator": "",
|
||||
"attendees": (), # tuple of "Name — role" strings
|
||||
}
|
||||
```
|
||||
|
||||
- The **master ships placeholders** (empty strings / empty tuple); the engagement copy
|
||||
ships real values. Everything that checks this cell — the gate, `pytest`, the repo
|
||||
suite — pins **shape only** (key set, types), NEVER emptiness or a value, so master
|
||||
and filled copy both stay green.
|
||||
- Where the values flow: the stage board header (rendered only when `client` is
|
||||
non-empty, so the master's stage is unchanged), the export preamble, and the
|
||||
appendix JSON `meta`.
|
||||
- Extend the dict per instrument (e.g. `current_spend`, `agent_count` for a sizing
|
||||
assessment) — client data belongs HERE, in the cell, not in the engine and not
|
||||
hardcoded in presentation cells.
|
||||
|
||||
## Engine parameterization
|
||||
|
||||
The engine package holds schema + logic only, and every session-level function takes
|
||||
the content as its **first argument** — the engine never imports content:
|
||||
|
||||
```python
|
||||
SESSION = build_session(TOPICS, STATUS_BY_TOPIC, NOTES_BY_TOPIC, DONE_SUBTOPICS)
|
||||
```
|
||||
|
||||
*Why:* content stays in the notebook (editable), logic stays testable — tests feed the
|
||||
real bank from the tagged cell, or synthetic banks for edge cases. Widgets are built by
|
||||
a **runtime loop over the content** (with per-widget distinct labels — Mercury's cache
|
||||
is label-keyed), so a content edit propagates to sidebar, board, script, gate, and
|
||||
export with zero further wiring.
|
||||
|
||||
---
|
||||
|
||||
## Testing an Assessment
|
||||
|
||||
All layers of the Mercury Pattern's testing section apply; assessments add the
|
||||
**exec-by-tag** recipe for content:
|
||||
|
||||
```python
|
||||
# tests/conftest.py — the content fixtures read the NOTEBOOK, no kernel
|
||||
def tagged_cell_ns(tag: str) -> dict[str, Any]:
|
||||
nb = nbformat.read(NOTEBOOK, as_version=4)
|
||||
cells = [c for c in nb.cells if tag in c.metadata.get("tags", [])]
|
||||
assert len(cells) == 1
|
||||
ns: dict[str, Any] = {}
|
||||
exec(compile(cells[0].source, f"{NOTEBOOK.name} [{tag}]", "exec"), ns)
|
||||
return ns
|
||||
```
|
||||
|
||||
- **Content pins** — counts, key order, uniqueness, well-formedness of the bank
|
||||
(`tests/test_topics.py` in the reference). Re-pin deliberately on content changes.
|
||||
- **Engagement pins** — shape of `ENGAGEMENT`, its flow into the export meta, and the
|
||||
cell's above-the-widgets position (`tests/test_engagement.py`).
|
||||
- **Engine pins** — session logic against hand-checked values (`tests/test_session.py`).
|
||||
- **Staging** — `backstage`/`backstage_md` render only off stage.
|
||||
- **Gate + repo suite** — the in-notebook gate re-counts the bank on every execution;
|
||||
`make check-notebooks` (repo root) enforces the tag taxonomy from outside.
|
||||
|
||||
## Export handoff (the LLM-input artifact)
|
||||
|
||||
`scripts/export_report.py` — execute ONCE, convert twice, post-process:
|
||||
|
||||
1. `nbconvert --execute` to a temp copy (never combine `--execute` with tag-stripping
|
||||
in one call — a cell could be removed before it runs).
|
||||
2. **HTML** from the executed copy — full presentation, human review.
|
||||
3. **Markdown** from the executed copy with
|
||||
`TagRemovePreprocessor.remove_cell_tags={"presentation"}` — the LLM artifact keeps
|
||||
content, script, gate, and appendix; drops setup/widget/board source and widget-repr
|
||||
noise.
|
||||
4. Prepend a generated **preamble**: what the document is, the engagement line (read
|
||||
from the `engagement-data` cell via nbformat), how to read it, and the instruction
|
||||
that the final fenced JSON block is the source of truth.
|
||||
|
||||
The appendix cell emits its table + JSON through **`backstage_md()`** (one
|
||||
`text/markdown` display), so the `.md` export carries a clean fenced ```json block as
|
||||
the last thing in the file — not an indented text blob.
|
||||
|
||||
## Stage polish
|
||||
|
||||
`config.toml` carries the brand ([`docs/brand.md`](brand.md)) — the reference's file
|
||||
is the copy-me. Beyond the palette: name the deliverable notebook in
|
||||
`[welcome] message`, keep the footer attributed, set the brand state colors
|
||||
(`success_color`, `warning_color`, `danger_color`), and keep **web-safe font stacks**
|
||||
(no `font_url` — a client screen must not depend on a network font fetch). Engine
|
||||
palettes may deviate deliberately (the reference's on-board complete-green `#00a34c`
|
||||
is darker than brand Success `#00cb5d` for contrast on white cards) — note such
|
||||
choices where they live.
|
||||
|
||||
---
|
||||
|
||||
## Copy-out checklist (canonical)
|
||||
|
||||
Running an assessment for a client — the master never touches client data:
|
||||
|
||||
1. **Copy** the master directory out of Palladium to your engagement location
|
||||
(it is self-contained — no repo machinery comes along or is needed).
|
||||
2. **Rename** it `YYYYMM_Client_Instrument` (e.g. `202608_Acme_CX_Discovery`).
|
||||
3. **Provision**: `python -m venv .venv && .venv/bin/pip install -e ".[dev]"`
|
||||
(editable installs pin absolute paths — a copied venv is broken; always recreate).
|
||||
4. **Fill the `engagement-data` cell** — and only that cell — with the client facts.
|
||||
5. **Verify**: `pytest` and the headless gate
|
||||
(`jupyter nbconvert --to notebook --execute --inplace notebooks/*.ipynb`) — both
|
||||
stay green by design (shape-only pins).
|
||||
6. The copy is now **confidential**: it never merges back, never returns to this
|
||||
repo. Improvements you discover on the engagement are hand-carried to the master
|
||||
as clean, client-free edits.
|
||||
|
||||
## Assessment anti-patterns
|
||||
|
||||
- ❌ **Facilitator prose in markdown cells** — markdown ALWAYS renders on the
|
||||
Mercury stage; run-books, "backstage" explanations, and section headings for
|
||||
hidden sections all leak to the client. The stage gets a client-neutral title
|
||||
and the board; facilitator orientation goes through `backstage_md()` in a
|
||||
`presentation`-tagged code cell (JupyterLab and the HTML export show it; the
|
||||
stage and the LLM `.md` export don't).
|
||||
- ❌ **Client data in a master** — a client's name in `assessments/` means the
|
||||
copy-out step was skipped; scrub and move it out.
|
||||
- ❌ **Content in `.py`** — topic banks, prompts, survey text are content; content
|
||||
lives in tagged cells. (This includes notebook *generators* that hold content as
|
||||
Python strings — the same mistake one indirection deeper.)
|
||||
- ❌ **Untagged content cells** — the repo suite, the tests, and the export pipeline
|
||||
all find content by tag; an untagged content cell is invisible to all three.
|
||||
- ❌ **Emptiness pins** — asserting `ENGAGEMENT["client"] == ""` breaks every filled
|
||||
engagement copy; pin shape, never values.
|
||||
- ❌ **Widget labels derived from unstable text** — labels come from content titles;
|
||||
editing a title mid-session resets that widget. Edit titles between sessions.
|
||||
@@ -690,3 +690,27 @@ All arithmetic uses Python `Decimal` to avoid floating-point drift. Values are s
|
||||
---
|
||||
|
||||
*TEI Tool — Athena*
|
||||
|
||||
---
|
||||
|
||||
## Palladium conventions (client-side)
|
||||
|
||||
Relocated from the root README (2026-07); these are Palladium's bridges where the
|
||||
Forrester methodology and the Athena TEI API differ.
|
||||
|
||||
### Methodology bridges (Palladium ↔ Athena)
|
||||
|
||||
| Topic | Athena behaviour | Palladium convention |
|
||||
|---|---|---|
|
||||
| **Cost risk adjustment** | Costs are never risk-adjusted server-side | Cost values are pushed pre-multiplied by `(1 + risk_adj)`; field-level adjustment stays 0 |
|
||||
| **Year-0 "Initial" costs** | No year-0 concept; non-annual values are folded into Year 1 | Each cost gets a companion non-annual `<key>_initial` field. `TEIClient` folds them back into an `initial` key on read. Athena discounts these as Year 1 (Forrester doesn't discount Year 0) — expect ≈0.15% drift on cost PV |
|
||||
|
||||
### Scenario analysis (`core.calculations.SCENARIOS`)
|
||||
|
||||
Three scenarios model uncertainty in adoption and realization:
|
||||
|
||||
| Scenario | Adoption | Risk delta | Effect |
|
||||
|----------|----------|------------|--------|
|
||||
| Conservative | 80% | +10pp on benefits | Lower benefits, higher modelled cost |
|
||||
| Moderate | 100% | 0 | Base case (= published study) |
|
||||
| Aggressive | 115% | –5pp on benefits | Higher benefits, lower padding on cost |
|
||||
|
||||
@@ -1,21 +1,32 @@
|
||||
# Mercury Notebook Deliverable Pattern v1.0.0
|
||||
# Mercury Notebook Deliverable Pattern v1.1.0
|
||||
|
||||
Standardizes how Palladium studies ship business-case deliverables: a Mercury-served
|
||||
Jupyter notebook **is** the artifact — math in a self-contained study package,
|
||||
Standardizes how Palladium masters — **Studies and Assessments** — ship notebook
|
||||
deliverables: a Mercury-served Jupyter notebook **is** the artifact — logic in a
|
||||
self-contained study package, content and client data in tagged notebook cells,
|
||||
presentation in reactive notebooks, verification gates pinning every number, and
|
||||
LLM-readable exports. All new studies follow this pattern; the Streamlit app path is
|
||||
LLM-readable exports. All masters follow this pattern; the Streamlit app path is
|
||||
retired by it.
|
||||
|
||||
## 🐾 Red Panda Approval™
|
||||
|
||||
This pattern follows Red Panda Approval standards.
|
||||
This pattern follows Red Panda Approval standards (see `CLAUDE.md` for the rubric).
|
||||
|
||||
**Audience note:** this document is written to be loaded whole as context by an LLM
|
||||
agent building or modifying a study. Rules are imperative (MUST/SHOULD/NEVER), each
|
||||
agent building or modifying a master. Rules are imperative (MUST/SHOULD/NEVER), each
|
||||
with a one-line *why*. Long code lives in the runnable template
|
||||
[`template/MercuryNotebook/`](../template/MercuryNotebook/) — copy it to start a study;
|
||||
snippets here are excerpts from it. The full-scale reference implementation is the CTM
|
||||
Genesys study, [`studies/202607_CTM_GenesysCX/`](../studies/202607_CTM_GenesysCX/).
|
||||
snippets here are excerpts from it (note: the template still encodes the py-engine
|
||||
model only). The reference implementation of the **notebook-first content model** is
|
||||
[`assessments/CX_Discovery_Workshop/`](../assessments/CX_Discovery_Workshop/); the
|
||||
largest worked multi-notebook example remains the CTM Genesys study,
|
||||
[`studies/202607_CTM_GenesysCX/`](../studies/202607_CTM_GenesysCX/).
|
||||
|
||||
**Docs map:** this file holds the **shared mechanics** every master obeys.
|
||||
[`Assessment_Pattern_V1-00.md`](Assessment_Pattern_V1-00.md) and
|
||||
[`Study_Pattern_V1-00.md`](Study_Pattern_V1-00.md) specialize it per master type —
|
||||
read the one for your master type first. `CLAUDE.md` at the repo root is the
|
||||
always-on contract (rubric, three-layer contract, taxonomy, risk tiers,
|
||||
confidentiality) and takes precedence where documents disagree.
|
||||
|
||||
---
|
||||
|
||||
@@ -53,9 +64,11 @@ Instead, this pattern defines:
|
||||
|
||||
```
|
||||
palladium/
|
||||
├── docs/ # repo-wide docs (this pattern, brand.md)
|
||||
├── docs/ # repo-wide docs (the patterns, brand.md)
|
||||
├── template/
|
||||
│ └── MercuryNotebook/ # copy-me starting point (runnable)
|
||||
├── assessments/
|
||||
│ └── Instrument_Name/ # reusable workshop master (UNDATED), e.g. CX_Discovery_Workshop
|
||||
└── studies/
|
||||
├── YYYYMM_TEI_Vendor_Product/ # vendor TEI study, e.g. 202602_TEI_Amazon_Connect
|
||||
└── YYYYMM_Client_EngagementName/ # client study, e.g. 202607_CTM_GenesysCX
|
||||
@@ -63,6 +76,10 @@ palladium/
|
||||
|
||||
- Names MUST use **underscores, never dashes** — dashed directories can't be Python
|
||||
packages, and everything in a study is importable code.
|
||||
- **Studies are dated** (`YYYYMM_` — they reproduce a dated publication or engagement);
|
||||
**Assessments are undated** (living instruments). An **engagement copy** of a master
|
||||
is stamped `YYYYMM_Client_Instrument` at copy-time and lives OUTSIDE this repo — see
|
||||
the copy-out checklist in the Assessment Pattern and `CLAUDE.md` § Confidentiality.
|
||||
- Every study is self-contained with this layout (from the template):
|
||||
|
||||
```
|
||||
@@ -93,11 +110,11 @@ JupyterLab by analysts, executed headless by nbconvert for exports. NEVER build
|
||||
parallel UI (Streamlit, Dash, a second "app" rendering the same model) — two surfaces
|
||||
over one model always diverge, and the notebook already *is* the interactive surface.
|
||||
|
||||
### 2 · Engine/presentation split
|
||||
### 2 · Engine/presentation split — and where content lives
|
||||
|
||||
Notebooks hold **no math**. Every number a stakeholder sees is computed in the study
|
||||
package and imported. *Why:* the package is testable and diffable; notebook cells are
|
||||
neither. The test suite pins the engine; the notebook only arranges its outputs.
|
||||
package and imported. *Why:* the package is testable and diffable for *logic*; the
|
||||
test suite pins the engine; the notebook only arranges its outputs.
|
||||
|
||||
```python
|
||||
# notebook cell — arrange and render, never compute
|
||||
@@ -106,6 +123,15 @@ inc, net_by = case_flows(total_by_year, ben_by_year)
|
||||
kpi = case_kpis(inc, net_by, DISCOUNT_RATE)
|
||||
```
|
||||
|
||||
The split cuts the other way for **content and client data**: notebooks hold no math,
|
||||
but they DO hold the human-authored content (topic banks, survey text, facilitation
|
||||
prompts) and the client facts (spend, headcount, engagement identity) — in **tagged
|
||||
cells** (`topic-bank`, `engagement-data`), edited in Jupyter. Content NEVER lives in a
|
||||
`.py` file. *Why:* the notebook is the document the consultant reads and edits;
|
||||
content hidden in importable modules defeats the point of a notebook. Tests pin
|
||||
content by reading the tagged cells with nbformat and exec'ing them — no kernel
|
||||
needed. Full contract: the [Assessment Pattern](Assessment_Pattern_V1-00.md).
|
||||
|
||||
### 3 · The Mercury reactivity contract
|
||||
|
||||
Mercury re-executes only the cells **below** a changed widget's cell — never the
|
||||
@@ -416,8 +442,15 @@ Each of these cost a debugging session or a client-facing embarrassment. Don't.
|
||||
|
||||
## Testing
|
||||
|
||||
Layers, from inner to outer — every study ships all four:
|
||||
Layers, from inner to outer — every master ships layers 1–4; layer 0 covers the whole
|
||||
library from the repo root:
|
||||
|
||||
0. **Repo-level structural suite** (`tests/test_notebooks.py`, run from the ROOT venv:
|
||||
`make check-notebooks`) — kernel-free nbformat checks on every committed notebook:
|
||||
parses, python3 kernel, no error outputs, cleanly executed top-to-bottom, and (for
|
||||
notebook-first masters) the tagged-cell taxonomy. Every notebook must be classified
|
||||
in `tests/nbcheck.py`; grandfathered ones skip the notebook-first tier with a
|
||||
recorded reason.
|
||||
1. **Engine pins** (`tests/`) — hand-checked acceptance numbers for every model
|
||||
function; the contracted overlay AND the verbatim record both pinned, so neither can
|
||||
drift. Compute by hand first, then pin (see
|
||||
|
||||
94
docs/Notebook_Review_Prompt_V1-00.md
Normal file
94
docs/Notebook_Review_Prompt_V1-00.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Notebook Review Prompt v1.0.0
|
||||
|
||||
The canonical LLM review prompt for a Palladium master notebook. Model-agnostic:
|
||||
paste everything below the line into any capable LLM together with the notebook's
|
||||
raw `.ipynb` JSON, or run it in-repo with `/review-notebook <path>`. It hunts the
|
||||
class of problems deterministic checks can't: content oddities, cross-cell rot,
|
||||
stale numbers, and stage leakage.
|
||||
|
||||
---
|
||||
|
||||
## Role
|
||||
|
||||
You are reviewing one **Palladium master notebook** — a Mercury-served Jupyter
|
||||
deliverable a client will see on a live call. Your input is the raw `.ipynb` JSON
|
||||
(cell sources, tags, outputs, in document order). Before judging anything, read the
|
||||
contracts it was built against:
|
||||
|
||||
- `docs/Mercury_Notebook_Pattern_V1-00.md` — reactivity, gate, stage/backstage,
|
||||
appendix
|
||||
- `docs/Assessment_Pattern_V1-00.md` or `docs/Study_Pattern_V1-00.md` — whichever
|
||||
matches the master's type (assessments/… vs studies/…)
|
||||
- The master's own `README.md` and, for cross-checks, its engine package
|
||||
(`discoverylib/`, `teicalc/`, …) — you MAY open engine `.py` files to verify a
|
||||
claim, and should when a number or name is in doubt.
|
||||
|
||||
## What NOT to re-check
|
||||
|
||||
`tests/test_notebooks.py` already enforces these deterministically — do not spend
|
||||
review effort or findings on them: the file parses as nbformat 4; kernel is python3;
|
||||
no error outputs; execution counts are monotonic top-to-bottom; the unique tags
|
||||
(`topic-bank`, `engagement-data`, `gate`, `data-appendix`) appear at most once;
|
||||
notebook-first masters have all four, the appendix last, engagement-data above the
|
||||
widgets; presentation cells emit no stream output; content cells use no magics.
|
||||
|
||||
## Check catalogue
|
||||
|
||||
Work cell by cell in document order, then once more end-to-end per theme:
|
||||
|
||||
1. **Content oddities** — typos and grammar in client-visible text; duplicated or
|
||||
near-duplicate prompts/items; terminology drifting from the source document the
|
||||
content anchors (the README names it); `key` slugs that don't match their titles;
|
||||
counts/minutes in comments or prose that don't match the actual content; tone that
|
||||
doesn't belong in front of a client.
|
||||
*Spot it by:* reading the content cells as an editor, then recounting anything a
|
||||
comment or heading claims.
|
||||
2. **Broken cross-cell references** — names used before their defining cell in
|
||||
document order; a widget's `.value` read in the cell that defines it (frozen at
|
||||
first render); variables shadowed or redefined with a different meaning; cells
|
||||
that only work because of stale kernel state (would fail on a fresh top-to-bottom
|
||||
run); imports used but not imported in any earlier cell.
|
||||
*Spot it by:* tracing each name in a cell back to its defining cell index.
|
||||
3. **Stale numbers** — figures hardcoded in markdown or annotations that the engine
|
||||
or gate could contradict; outputs inconsistent with the current source (a changed
|
||||
cell whose committed output still shows the old result); gate pins that disagree
|
||||
with the test-suite pins; totals in prose that don't equal the content.
|
||||
*Spot it by:* comparing markdown claims ↔ committed outputs ↔ gate asserts ↔
|
||||
engine constants.
|
||||
4. **Tag/metadata issues** — content sitting in an untagged cell (invisible to
|
||||
tests and the export pipeline); a cell whose comment says one thing and whose tag
|
||||
says another; presentation logic inside a content cell or vice versa.
|
||||
*Spot it by:* asking, for each cell, "who consumes this — and would they find it?"
|
||||
5. **Stage leakage** — anything client-inappropriate reachable on the Mercury stage:
|
||||
markdown cells ALWAYS render on stage, so internal notes/instructions in markdown
|
||||
are leaks; diagnostics via bare `print` instead of `backstage()`; placeholder or
|
||||
internal wording in the board/stage HTML; a filled engagement value where the
|
||||
master should have a placeholder.
|
||||
*Spot it by:* simulating the stage — markdown cells + non-backstage outputs of
|
||||
code cells are what the client sees.
|
||||
6. **Export quality** — appendix JSON missing state a figure or table shows;
|
||||
values that won't JSON-serialize; the `.md` handoff missing something an LLM
|
||||
would need to draft the write-up; preamble claims that don't match the document.
|
||||
*Spot it by:* reading the appendix cell's payload against everything rendered
|
||||
above it.
|
||||
|
||||
## Output format
|
||||
|
||||
Report findings **ordered by severity**, one line each, exactly:
|
||||
|
||||
```
|
||||
[BLOCKER] cell <index> (<tag or first source line>): <finding> — <evidence> — <suggested fix>
|
||||
[SHOULD-FIX] cell <index> (<tag or first source line>): <finding> — <evidence> — <suggested fix>
|
||||
[NIT] cell <index> (<tag or first source line>): <finding> — <evidence> — <suggested fix>
|
||||
```
|
||||
|
||||
- **BLOCKER** — a client would see something wrong, or the notebook lies (stale
|
||||
number, stage leak, broken reference).
|
||||
- **SHOULD-FIX** — correctness/maintainability debt that won't embarrass anyone
|
||||
today.
|
||||
- **NIT** — polish.
|
||||
|
||||
End with one verdict line: `VERDICT: <n> blocker(s), <n> should-fix, <n> nit(s) —
|
||||
<one-sentence overall judgement>`. If a theme produced no findings, do not pad —
|
||||
finding nothing is a valid result; say `No findings.` above the verdict if the whole
|
||||
review is clean. Output nothing outside this format.
|
||||
107
docs/Study_Pattern_V1-00.md
Normal file
107
docs/Study_Pattern_V1-00.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# Study Pattern v1.0.0
|
||||
|
||||
How Palladium ships **Studies**: dated reproductions of a published base document —
|
||||
a Forrester TEI study or similar — personalized to a client as an overlay. A Study is
|
||||
a **master**: the anchor is public/published data, the master stays client-clean, and
|
||||
client personalization happens in an engagement copy outside this repo. References:
|
||||
[`studies/202512_TEI_Genesys_CX_Cloud/`](../studies/202512_TEI_Genesys_CX_Cloud/) and
|
||||
[`studies/202602_TEI_Amazon_Connect/`](../studies/202602_TEI_Amazon_Connect/)
|
||||
(both package `teicalc`).
|
||||
|
||||
## 🐾 Red Panda Approval™
|
||||
|
||||
This pattern follows Red Panda Approval standards (see `CLAUDE.md` for the rubric).
|
||||
|
||||
**Audience note:** written to be loaded whole by an LLM agent building or modifying a
|
||||
study. This document holds what studies ADD to the
|
||||
[Mercury Notebook Deliverable Pattern](Mercury_Notebook_Pattern_V1-00.md) — shared
|
||||
mechanics live there and are not restated. `CLAUDE.md` takes precedence where
|
||||
documents disagree.
|
||||
|
||||
**Status note — read this before "fixing" a study:** this pattern is grounded in the
|
||||
TEI twins **as they are today**: math and anchors in the `.py` engine, hand-authored
|
||||
notebooks, no tagged-cell taxonomy. The notebook-first evolution — client data in an
|
||||
`engagement-data` cell, tagged content cells, the Assessment Pattern's cell taxonomy —
|
||||
**lands with the first study redesign** (a recorded identified opportunity in
|
||||
`CLAUDE.md`), not by piecemeal edits. Until then the repo-level structural suite
|
||||
grandfathers the study notebooks by name, and their current shape is the correct
|
||||
shape. `studies/202607_CTM_GenesysCX/` is a grandfathered real-client engagement
|
||||
study — frozen; see `CLAUDE.md` § Known liabilities.
|
||||
|
||||
---
|
||||
|
||||
## What a Study is
|
||||
|
||||
| | **Study** | Assessment |
|
||||
|---|---|---|
|
||||
| Reproduces | a **dated publication** (TEI) or signed engagement | — |
|
||||
| Naming | `YYYYMM_TEI_Vendor_Product` / `YYYYMM_Client_Engagement` | undated |
|
||||
| Anchor | the publication's composite, **verbatim** | the instrument's content |
|
||||
| "Numbers" | dollars — NPV / ROI / payback | qualitative state |
|
||||
| Client data | overlay rescale on the anchor | `engagement-data` cell |
|
||||
|
||||
The `YYYYMM_` prefix is the **publication/engagement date** (e.g. `202512_` = the
|
||||
December 2025 Forrester study), not the date you worked on it — a Study is a snapshot
|
||||
of a dated document and stays dated.
|
||||
|
||||
## Verbatim anchor + contracted overlay
|
||||
|
||||
The heart of a Study (mechanics in the Mercury Pattern, *Recommended Practices*):
|
||||
|
||||
- `anchor.py` holds the publication's tables as `*_VERBATIM` constants — **NEVER
|
||||
edited**. The reproduction is only worth anything if the anchor is the exact
|
||||
published record.
|
||||
- `overlay.py` layers client personalization over the anchor — in the current twins, a
|
||||
🟡 first-order linear rescale by client drivers (agents / contacts / growth):
|
||||
"the composite at your size", never presented as "your TEI".
|
||||
- Notebooks read through the `anchor()` helper so the walk *published → personalized*
|
||||
stays explicit and auditable on stage.
|
||||
|
||||
## The reproduction gate
|
||||
|
||||
A Study's gate (Mercury Pattern §4) MUST pin the **published headline numbers** —
|
||||
the reproduction is the smoke test:
|
||||
|
||||
- Amazon Connect twin: NPV **$78,713,715**, ROI **342%**, payback **<6 months**.
|
||||
- Genesys CX Cloud twin: NPV **$10,783,468**, ROI **266%**.
|
||||
|
||||
Plus the **composite-scale identity**: the overlay at the composite's own drivers must
|
||||
reproduce the anchor exactly (rescale(1.0) == anchor) — the proof the personalization
|
||||
layer adds nothing at scale 1.
|
||||
|
||||
## Confidentiality posture
|
||||
|
||||
- The **published source PDF** (the Forrester study) is public and MAY live in the
|
||||
study's `docs/` — it is the one exception to the binary-documents block in
|
||||
`.gitignore`: add it with `git add -f` and name the source in the commit message.
|
||||
- Everything else follows `CLAUDE.md` § Confidentiality: masters client-clean, client
|
||||
personalization only in an engagement copy outside the repo, and NEVER commit client
|
||||
documents (SOWs, quotes, NDA'd vendor decks). The CTM study predates this rule and
|
||||
is grandfathered — do not use it as precedent.
|
||||
|
||||
## Layout & testing
|
||||
|
||||
Layout and packaging are the Mercury Pattern's (self-contained `teicalc`-style engine,
|
||||
per-study venv, whole toolchain as core deps). Testing is the standard four layers
|
||||
plus repo-level layer 0; a Study's engine pins carry BOTH the verbatim record and the
|
||||
overlay, so neither can drift:
|
||||
|
||||
```
|
||||
tests/test_anchor.py # the published tables, pinned line by line
|
||||
tests/test_overlay.py # rescale identity at composite scale + spot rescales
|
||||
tests/test_scenarios.py # scenario framing over the overlay
|
||||
tests/test_model.py # finance primitives (NPV, payback) hand-checked
|
||||
tests/test_staging.py # stage/backstage
|
||||
```
|
||||
|
||||
## Variants
|
||||
|
||||
The Mercury Pattern's variants map onto Studies as:
|
||||
|
||||
- **Variant 4 — TEI composite reproduction** — the canonical Study (both twins).
|
||||
- **Variant 1 — corrected/pressure-tested business case** — engagement-study form
|
||||
(CTM's `ctm_business_case_corrected`): vendor claims verbatim, omitted costs added.
|
||||
- **Variant 2 — scenario notebook on a thin module** — a second question over the same
|
||||
engine (CTM's `migration_wfm`).
|
||||
- **Variant 3 — exploratory calculator** — pre-anchor what-if surface that graduates
|
||||
into Variant 1 as facts arrive.
|
||||
Reference in New Issue
Block a user