kernbench2/.claude/commands/paper.md

---
description: Generate the 1H HW-SW codesign technical paper (PDF) — propose/confirm a TOC, then synthesize ADRs + live KernBench benchmark results into a sectioned LaTeX paper compiled with Tectonic.
---

# `/paper` — HW-SW Codesign Technical Report Generator

Produces a **technical-paper-style PDF** at
`docs/report/1H-codesign-paper/build/main.pdf`, synthesizing the
ADR corpus, SPEC.md, and **live KernBench benchmark results** into a
publication-shaped report on the three 1H optimizations:

1. **GEMM Acceleration** using the composite command
2. **All-Reduce Acceleration** using PE_IPCQ
3. **Fused Grouped Query Attention** using both composite command and PE_IPCQ

plus an explanation of the simulation platform itself (why KernBench
exists, its execution model, and how it computes latency / how accurate
that is).

Audience is the **internal team** (engineers + architects), but the
report reads like a real technical paper: **do not cite `ADR-NNNN` or
`SPEC §X.Y` anywhere** — not in prose, not in a reference list. ADRs and
SPEC are *grounding sources only* (use their content; never name them).
The **References** section, if used, lists **external literature only**
(e.g. FlashAttention, Megatron-LM, GPT-3, Llama 3) — actual papers, not
internal docs.

---

## Operating principles (read first)

- **Confirm the TOC before writing.** The first thing `/paper` does is
  present the outline (saved `toc.md` if present, else the default
  below) and **stop for the user's confirmation / adjustments**. Do not
  generate sections until the user approves the TOC.
- **Ground every number.** Every quantitative result in prose, tables,
  or captions MUST trace to (a) a committed bench artifact
  (`*.csv` / `*.json` / `*.png` under `1H_milestone_output/` or
  `docs/diagrams/`), (b) a freshly-run bench output, or (c) a specific
  ADR/SPEC quotation. **Never invent latencies, speedups, or
  utilizations.** If a needed number does not exist, generate it via a
  bench run (below) or state the gap — do not guess.
- **Reuse existing results by default.** The three milestone benches
  already have committed outputs. Do **not** re-run a bench unless the
  user asks to regenerate, or the artifact is missing/stale.
- **Design / proposal status honesty.** Some designs are `Proposed`,
  not yet implemented (e.g. ADR-0061, ADR-0065). Present these as
  proposed and clearly separate "modeled & measured" from
  "designed, not yet measured." Do not report measurements for
  unimplemented paths.
- **English prose.** Architectural terms (composite command, PE_IPCQ,
  GQA, TCM, SIP, CUBE, PE) used as-is.

---

## Invocation

- `/paper` — **default**. Resolve the period, present the TOC
  (`toc.md` if it exists, else the default TOC), and **stop** for
  confirmation. Once the user approves (possibly after editing the TOC
  in conversation), persist `toc.md` and proceed to full build.
- `/paper toc` — only (re)negotiate and persist the TOC. No sections,
  no build.
- `/paper build` — assume the saved `toc.md` is approved; run the full
  pipeline (sections → figures → compile PDF) without re-confirming.
- `/paper section <id>` — regenerate a single section (`<id>` = a
  filename stem under `sections/`, e.g. `04-allreduce`) and recompile.
- `/paper figures` — (re)generate only the benchmark figures/data
  (runs/refreshes benches), no prose changes, then recompile.
- `/paper pdf` — only recompile the existing `.tex` to PDF (Tectonic).

If `toc.md` does not yet exist, any mode except `/paper`/`/paper toc`
first falls back to presenting the default TOC for confirmation.

---

## Output layout

```
docs/report/1H-codesign-paper/
  toc.md                      # the agreed outline (persisted; human-editable)
  main.tex                    # preamble + \input of each section
  refs.bib                    # ADR/SPEC bibliography (optional; or inline refs)
  sections/
    00-exec-summary.tex
    01-introduction.tex
    02-platform.tex           # includes 3.4 Modeled Hardware Configuration
    03-gemm.tex
    04-allreduce.tex
    05-gqa.tex
    06-discussion.tex
    07-conclusion.tex
    08-future-work.tex
  figures/                    # PNGs copied in from bench outputs (portable)
  build/                      # Tectonic output; main.pdf lives here
```

Report-specific benchmark/plot harnesses (when a figure must be
generated, see *Generating missing results*) live **isolated** under:

```
scripts/paper/                # report-only harnesses — NOT registered benches
```

This directory is intentionally separate from `src/kernbench/benches/`
so the report's harnesses never collide with other people's benches
(the bench registry only audits `src/kernbench/benches/`). Treat
`scripts/paper/*` and everything under `docs/report/1H-codesign-paper/`
as **derived artifacts** (like `docs/diagrams/`): creating/updating them
does not require Phase 2 approval, but they MUST stay consistent with
SPEC.md and ADRs.

---

## Default TOC

If `toc.md` is absent, propose this and ask the user to confirm or edit:

1. **Executive Summary** — the attention-optimization goal, the two
   enabling optimizations (GEMM composite command + PE_IPCQ communication),
   the fused GQA capstone, the headline results, and the bottom-line
   recommendation on which HW changes are worth keeping. May run a
   paragraph or two longer than a terse abstract; front-loads conclusions.
2. **Introduction** — the 1H focus is **attention-kernel optimization**:
   FlashAttention-style tiling and Grouped Query Attention (GQA), building
   on prior work on Multi-Head Attention (MHA, studied earlier — reference
   it as the established baseline, not re-derived here). Optimizing a fused
   attention kernel requires two enabling optimizations, each studied in
   its own right and then **combined inside the fused kernel**:
   (a) **GEMM optimization** (the composite command) and
   (b) **communication optimization** (PE_IPCQ). Frame the three result
   sections as *two enablers → the fused GQA capstone that uses both*.
   Motivate why HW-SW codesign (not software alone) is required to realize
   the gains.
3. **The KernBench Platform**
   1. *Why KernBench* — source-level kernel execution with **no compiler /
      software-stack dependency**; isolate algorithm-level optimization from
      other layers so a kernel can fully exploit the hardware without E2E
      cross-layer entanglement.
   2. *Execution model* — discrete-event graph; `runtime API → sim_engine →
      components`; PE pipeline + composite commands; 2-pass data/timing model.
   3. *Latency model & accuracy* — traversal-based golden invariants
      (every hop pays > 0, deterministic, explicit connectivity); the
      structural CPU-dispatch cost model; per-edge BW occupancy, HBM
      pseudo-channel parallelism, flit streaming; known simplifications &
      what calibration anchors the constants.
   4. *Modeled Hardware Configuration* — the shared, platform-wide
      quantitative config used across **all** experiments, read from
      `topology.yaml`: SIP/CUBE/PE hierarchy (counts & nesting), PE clock &
      compute throughput, TCM/SRAM/HBM capacities & bandwidths, NOC /
      inter-CUBE / inter-SIP link bandwidths & latencies, and the cost-model
      constants. Per-experiment workload parameters (GEMM shapes, All-Reduce
      topologies, GQA seq/head/user counts) stay in each evaluation section,
      not here.
4. **GEMM Acceleration via Composite Command** — necessity · design · results · analysis.
5. **All-Reduce Acceleration via PE_IPCQ** — necessity · design · results · analysis.
6. **Fused Grouped Query Attention** (composite + PE_IPCQ) — necessity · design · results · analysis.
7. **Discussion** — which HW changes are meaningful, and why (cross-cutting).
8. **Conclusion**.
9. **Future Work — 2H** — add the FFN/MoE layer toward full LLM decoding;
   analyze how compute & data should be distributed for agentic / MoE
   workloads.
10. **References** *(optional)* — **external literature only** (e.g.
    FlashAttention, Megatron-LM, GPT-3, Llama 3). **No ADR/SPEC entries.**

Each of §4/§5/§6 follows the same four-beat structure:
**(a) why the optimization is needed**, **(b) the design (how it works)**,
**(c) experimental results** (run/plot/measure), **(d) analysis & meaning**.

---

## Source map (ADR / SPEC → section)

Use these as **grounding sources only** — read them for content, but
**never name an ADR or SPEC in the paper**. Verify each quantitative
claim against the actual file before asserting it (CLAUDE.md: no
asserting architectural numbers from memory).

- **Modeled Hardware Configuration (§3.4)**: `topology.yaml`
  (authoritative for counts, clocks, BW, link params, cost-model
  constants) is the **primary** source — read it directly and build the
  HW table from it. Cross-check against ADR-0003 (hierarchy/scope),
  ADR-0004 (memory BW guarantee), ADR-0033 (latency model), ADR-0064
  (cost-model defaults). If `topology.yaml` and an ADR disagree, the
  loaded `topology.yaml` is what the sim actually runs — report that
  value and flag the mismatch.

- **Platform — why/execution/latency**: `SPEC §0`, `SPEC §0.1`
  (golden invariants), `SPEC §1 R1/R2/R3`, `SPEC §2.1` (graph execution);
  ADR-0003 (hierarchy/scope), ADR-0007 (runtime↔sim_engine boundary),
  ADR-0014 (PE pipeline / composite), ADR-0020 (2-pass data/timing),
  ADR-0033 (latency model assumptions & simplifications),
  ADR-0064 (structural CPU-dispatch cost model: `FIXED + logical_bytes×R`).
- **GEMM / composite**: ADR-0014 (composite command D3.2/D3.3),
  ADR-0042 (tile-plan generators), ADR-0044 (GEMM eval harness),
  ADR-0064 (cost model — composite's per-command BW reduction),
  ADR-0065 (flat-ops `CompositeCmd`, *Proposed*).
- **All-Reduce / PE_IPCQ**: ADR-0023 (PE_IPCQ — control plane, ring
  buffers, VC split), ADR-0025 (direction addressing), ADR-0032
  (intercube all-reduce), ADR-0043 (allreduce eval harness),
  ADR-0047 (AHBM CCL backend), ADR-0050 (CCL algorithm contract).
- **GQA / fused attention**: ADR-0060 (fused GQA kernel),
  ADR-0061 (`tl.broadcast`/`tl.repeat`, *Proposed*),
  ADR-0062 (lazy `tl.load`), ADR-0063 (`tl.scratch_scope`),
  ADR-0064 (cost model), ADR-0065 (flat-ops composite + `softmax_merge`,
  *Proposed*). KO mirrors in `docs/adr-ko/`; DDD-0060/0065 design docs if present.
- **Eval/milestone pattern**: ADR-0054 (self-contained milestone benches);
  ADR-0045 (bench contract).

---

## Result map (bench → artifact → section) + run commands

All runs use the repo's `topology.yaml` unless a section needs a
specific topology. Reuse committed artifacts unless told to regenerate.

### §4 GEMM — `milestone-1h-gemm`
- Artifacts (committed): `src/kernbench/benches/1H_milestone_output/gemm/`
  → `gemm_sweep.json`, `gemm_stage_breakdown.png`,
  `gemm_mac_utilization_measured.png`,
  `gemm_mac_utilization_theoretical_vs_measured.png`.
- Regenerate: `kernbench run --topology topology.yaml --bench milestone-1h-gemm`
  (full 24-shape sweep ≈ minutes; `MILESTONE_FAST=1` reuses committed JSON).

### §5 All-Reduce — `milestone-1h-ccl`
- Artifacts (committed): `src/kernbench/benches/1H_milestone_output/ccl/`
  → `summary.csv`, `topology.png`, per-topology
  `AllReduce_LRAB_*` PNGs, buffer-kind sweep
  `AllReduce_LRAB_2Dtorus_..._with_TCM_SRAM_HBM.{png,csv}`,
  `comparison_mesh_vs_ring_vs_2DTorus_vs_theoretical_vs_fsim.png`.
- Regenerate: `kernbench run --topology topology.yaml --bench milestone-1h-ccl`.

### §6 GQA — `milestone-gqa-headline`
- Artifact (committed): `src/kernbench/benches/1H_milestone_output/gqa_headline/sweep.json`.
  **Caveat:** this currently holds only `op_log_summary` **op-counts**
  (`gemm_count`, `ipcq_copy_count`, `dma_read_count`, `dma_write_count`)
  for 4 panels (single/multi-user × prefill/decode) — **no latency,
  no figures**.
- Regenerate counts:
  `GQA_HEADLINE_RUN=1 kernbench run --topology topology.yaml --bench milestone-gqa-headline`.
- For a real GQA **results** section, generate latency + figures via an
  isolated report harness — see next.

---

## Generating missing results (isolation rule)

When a section needs an artifact that does not yet exist (today: GQA
latency and any GQA figure), create a **report-dedicated harness** under
`scripts/paper/` — never under `src/kernbench/benches/`. Rules:

- One file per concern, prefixed `paper_`, e.g.
  `scripts/paper/paper_gqa_latency.py` (drives the engine, emits
  latency + a baseline-vs-optimized comparison),
  `scripts/paper/paper_plot_gqa.py` (renders figures from that data).
- Follow the **ADR-0054 self-contained pattern**: the harness builds its
  own `GraphEngine`(s), imports the existing GQA kernel modules
  (`_gqa_attention_{prefill,decode}_{short,long}.py`,
  `milestone_gqa_headline.py`), runs them, and reads latency from the
  engine's completion timestamps / op log. It does **not** register a
  bench and does **not** import or mutate other people's benches.
- Output figures land in `docs/report/1H-codesign-paper/figures/` and,
  if useful as a shared diagram, mirror into `docs/diagrams/gqa_plots/`.
- **Only measure what the current code actually runs.** ADR-0065's GQA
  opt2 (`softmax_merge`, two-composite decode) is *Proposed*; if it is
  not wired in production, present it as designed-not-measured and base
  the measured results on the implemented path. Verify implementation
  status by reading the code before claiming a measurement.
- These harnesses are derived-artifact tooling (no Phase 2 needed). **But**
  if the report ever requires changing a *registered production bench*
  under `src/kernbench/benches/` or any production module, that is a
  non-trivial production change — STOP and follow CLAUDE.md Phase 1 →
  approval → Phase 2.

---

## Procedure

### Step 0 — Toolchain (Tectonic)

Ensure a working Tectonic before any compile step. In order:

1. If `tectonic` is on PATH (`tectonic --version` succeeds), use it.
2. Else try, in order, whichever succeeds (capture output, don't hang):
   - `winget install --id TectonicProject.Tectonic --silent --accept-package-agreements --accept-source-agreements`
   - `scoop install tectonic`
   - Direct download (no admin): fetch the latest Windows `x86_64-pc-windows-msvc`
     release zip from `https://github.com/tectonic-typesetting/tectonic/releases`
     via PowerShell `Invoke-WebRequest`, expand it, and place `tectonic.exe`
     under `docs/report/1H-codesign-paper/build/.tools/`. Use that absolute
     path for compilation.
3. Re-verify with `tectonic --version`. If all methods fail, write the
   `.tex` files anyway, **report the failure**, and tell the user the
   exact command to compile manually (or to use Overleaf). Do not block
   the rest of the pipeline.

Tectonic fetches LaTeX packages on first compile (needs network once).

### Step 1 — Period + TOC

- Period from system date: month 1–6 → `{YYYY}-1H`, else `{YYYY}-2H`.
  (Report dir is fixed at `1H-codesign-paper` for this half; adjust the
  title page text with the period.)
- If `toc.md` exists, present it; else present the **Default TOC**.
- **Stop and ask the user to confirm or edit.** Incorporate edits.
  Persist the agreed outline to `toc.md`. (In `/paper build` mode, skip
  the stop and trust the saved `toc.md`.)

### Step 2 — Platform section (§3, incl. §3.4 HW config)

Write `sections/02-platform.tex` from the platform source map, covering
all four sub-parts:

- §3.1–3.3 (why / execution model / latency model + accuracy). Include
  the cost-model formula `dispatch_cycles = FIXED + logical_bytes×R` with
  the committed defaults (verify against ADR-0064 before quoting), and
  explicitly address **how accurate** the latency is: what is modeled
  precisely (per-edge BW occupancy, HBM pseudo-channels, flit streaming,
  per-component overhead) and the known simplifications.
- §3.4 *Modeled Hardware Configuration*: read `topology.yaml` directly
  and build a compact, table-driven summary — SIP/CUBE/PE hierarchy
  (counts & nesting), PE clock & compute throughput, TCM/SRAM/HBM
  capacity & bandwidth, NOC / inter-CUBE / inter-SIP link bandwidth &
  latency, and the cost-model constants (`FIXED_PER_CMD`, `R`, composite
  cap). Every number from `topology.yaml`. This is the shared config;
  later sections say "the modeled hardware" and only add their own
  workload params.

Do not name the source ADRs/SPEC in the prose.

### Step 3 — The three result sections (§4 GEMM, §5 All-Reduce, §6 GQA)

For each, in order, produce the four beats:

- **(a) Necessity** — from the ADR Context: the bottleneck the
  optimization targets (e.g. per-command dispatch overhead for GEMM;
  collective comm cost / HoL blocking for All-Reduce; KV-load-bound
  decode + softmax merge for GQA).
- **(b) Design** — from the ADR Decisions: composite command tile
  pipeline (ADR-0014); PE_IPCQ control/data split + ring buffers + VC
  (ADR-0023); fused attention with online-softmax, lazy load, scratch
  recycling (ADR-0060/0062/0063). Note Proposed-vs-Accepted status.
- **(c) Results** — reuse committed artifacts (GEMM, All-Reduce). For
  GQA, generate latency + figures via the isolated `scripts/paper/`
  harness (above). Copy chosen PNGs into `figures/`, `\includegraphics`
  them, and build result tables from the CSV/JSON — quoting actual
  numbers, with a one-line note of the run command and source artifact.
- **(d) Analysis** — interpret: what the curve/table shows, why the HW
  feature produces it, where it saturates or fails to help, and the
  cost (area/complexity) implied. Tie back to the cost model.

### Step 4 — Discussion, Conclusion, Future Work (§6/§7/§8)

- **Discussion**: synthesize across the three — which HW changes
  (composite command, PE_IPCQ, the cost-model-exposed BW reduction)
  carry their weight, and under what regimes.
- **Conclusion**: the codesign thesis, supported by the measured
  results; state plainly which HW changes are meaningful.
- **Future Work (2H)**: FFN / MoE layer toward full LLM decoding; how
  compute & data should be distributed for agentic / MoE workloads
  (expert routing, token dispersion, all-to-all vs all-reduce, capacity).
  Frame as the next codesign questions, not claims.

### Step 5 — Assemble & compile

- `main.tex`: article/IEEEtran-style preamble, title with the period,
  author, executive-summary input, then `\input{sections/...}` in TOC
  order, then references. Keep the preamble minimal (graphicx, booktabs,
  hyperref, amsmath) so Tectonic compiles cleanly.
- Compile: `tectonic main.tex --outdir build` (or the resolved Tectonic
  path) from `docs/report/1H-codesign-paper/`. Re-run if references need
  a second pass. Confirm `build/main.pdf` exists and report its size.

### Step 6 — Chat report

Emit to chat (not to any file):

```
## /paper — Build Summary
**PDF:** docs/report/1H-codesign-paper/build/main.pdf  (<N> pages, <size>)
**Toolchain:** tectonic <version | how installed>
**Sections written:** <list>
**Benches: reused vs regenerated**
  - GEMM: <reused committed | regenerated via milestone-1h-gemm>
  - All-Reduce: <reused | regenerated>
  - GQA: <generated via scripts/paper/paper_gqa_latency.py | counts-only>
**Figures included:** <list of figures/*.png>
**Grounding check:** every result number traced to <artifact list>
**Proposed-not-measured items flagged:** <e.g. ADR-0065 opt2, ADR-0061>
**Open gaps / suggested next runs:** <…>
```

---

## Constraints (do not violate)

1. **TOC first.** Never write sections before the TOC is confirmed
   (default mode). `toc.md` is the contract.
2. **No fabricated numbers.** Every quantitative claim traces to a
   committed artifact, a fresh bench run, or an ADR/SPEC quote. Verify
   architectural constants against the source file, not memory.
3. **Reuse by default.** Do not re-run a bench unless asked or the
   artifact is missing/stale.
4. **Isolation.** Report harnesses live only in `scripts/paper/`; never
   register a bench, never touch other people's benches.
5. **Production-change gate.** Any change to a registered bench or
   production module triggers CLAUDE.md Phase 1 → approval → Phase 2.
   Report tooling and `docs/report/...` are derived artifacts (exempt).
6. **Status honesty.** Mark Proposed designs as proposed; never present
   an unimplemented path as measured.
7. **No internal references.** Never name `ADR-NNNN` or `SPEC §X.Y` in
   prose, captions, or the reference list — they are grounding sources
   only. A References section, if present, holds **external literature
   only**. Hardware numbers come from `topology.yaml`.
8. **Determinism.** Sections in TOC order; same inputs → same paper.
9. **English prose**, internal-audience tone.
```