--- description: Generate the 1H HW-SW codesign technical paper (PDF) — propose/confirm a TOC, then synthesize ADRs + live KernBench benchmark results into a sectioned LaTeX paper compiled with Tectonic. --- # `/paper` — HW-SW Codesign Technical Report Generator Produces a **technical-paper-style PDF** at `docs/report/1H-codesign-paper/build/main.pdf`, synthesizing the ADR corpus, SPEC.md, and **live KernBench benchmark results** into a publication-shaped report on the three 1H optimizations: 1. **GEMM Acceleration** using the composite command 2. **All-Reduce Acceleration** using PE_IPCQ 3. **Fused Grouped Query Attention** using both composite command and PE_IPCQ plus an explanation of the simulation platform itself (why KernBench exists, its execution model, and how it computes latency / how accurate that is). Audience is the **internal team** (engineers + architects), but the report reads like a real technical paper: **do not cite `ADR-NNNN` or `SPEC §X.Y` anywhere** — not in prose, not in a reference list. ADRs and SPEC are *grounding sources only* (use their content; never name them). The **References** section, if used, lists **external literature only** (e.g. FlashAttention, Megatron-LM, GPT-3, Llama 3) — actual papers, not internal docs. --- ## Operating principles (read first) - **Confirm the TOC before writing.** The first thing `/paper` does is present the outline (saved `toc.md` if present, else the default below) and **stop for the user's confirmation / adjustments**. Do not generate sections until the user approves the TOC. - **Ground every number.** Every quantitative result in prose, tables, or captions MUST trace to (a) a committed bench artifact (`*.csv` / `*.json` / `*.png` under `1H_milestone_output/` or `docs/diagrams/`), (b) a freshly-run bench output, or (c) a specific ADR/SPEC quotation. **Never invent latencies, speedups, or utilizations.** If a needed number does not exist, generate it via a bench run (below) or state the gap — do not guess. - **Reuse existing results by default.** The three milestone benches already have committed outputs. Do **not** re-run a bench unless the user asks to regenerate, or the artifact is missing/stale. - **Design / proposal status honesty.** Some designs are `Proposed`, not yet implemented (e.g. ADR-0061, ADR-0065). Present these as proposed and clearly separate "modeled & measured" from "designed, not yet measured." Do not report measurements for unimplemented paths. - **English prose.** Architectural terms (composite command, PE_IPCQ, GQA, TCM, SIP, CUBE, PE) used as-is. --- ## Invocation - `/paper` — **default**. Resolve the period, present the TOC (`toc.md` if it exists, else the default TOC), and **stop** for confirmation. Once the user approves (possibly after editing the TOC in conversation), persist `toc.md` and proceed to full build. - `/paper toc` — only (re)negotiate and persist the TOC. No sections, no build. - `/paper build` — assume the saved `toc.md` is approved; run the full pipeline (sections → figures → compile PDF) without re-confirming. - `/paper section ` — regenerate a single section (`` = a filename stem under `sections/`, e.g. `04-allreduce`) and recompile. - `/paper figures` — (re)generate only the benchmark figures/data (runs/refreshes benches), no prose changes, then recompile. - `/paper pdf` — only recompile the existing `.tex` to PDF (Tectonic). If `toc.md` does not yet exist, any mode except `/paper`/`/paper toc` first falls back to presenting the default TOC for confirmation. --- ## Output layout ``` docs/report/1H-codesign-paper/ toc.md # the agreed outline (persisted; human-editable) main.tex # preamble + \input of each section refs.bib # ADR/SPEC bibliography (optional; or inline refs) sections/ 00-exec-summary.tex 01-introduction.tex 02-platform.tex # includes 3.4 Modeled Hardware Configuration 03-gemm.tex 04-allreduce.tex 05-gqa.tex 06-discussion.tex 07-conclusion.tex 08-future-work.tex figures/ # PNGs copied in from bench outputs (portable) build/ # Tectonic output; main.pdf lives here ``` Report-specific benchmark/plot harnesses (when a figure must be generated, see *Generating missing results*) live **isolated** under: ``` scripts/paper/ # report-only harnesses — NOT registered benches ``` This directory is intentionally separate from `src/kernbench/benches/` so the report's harnesses never collide with other people's benches (the bench registry only audits `src/kernbench/benches/`). Treat `scripts/paper/*` and everything under `docs/report/1H-codesign-paper/` as **derived artifacts** (like `docs/diagrams/`): creating/updating them does not require Phase 2 approval, but they MUST stay consistent with SPEC.md and ADRs. --- ## Default TOC If `toc.md` is absent, propose this and ask the user to confirm or edit: 1. **Executive Summary** — the attention-optimization goal, the two enabling optimizations (GEMM composite command + PE_IPCQ communication), the fused GQA capstone, the headline results, and the bottom-line recommendation on which HW changes are worth keeping. May run a paragraph or two longer than a terse abstract; front-loads conclusions. 2. **Introduction** — the 1H focus is **attention-kernel optimization**: FlashAttention-style tiling and Grouped Query Attention (GQA), building on prior work on Multi-Head Attention (MHA, studied earlier — reference it as the established baseline, not re-derived here). Optimizing a fused attention kernel requires two enabling optimizations, each studied in its own right and then **combined inside the fused kernel**: (a) **GEMM optimization** (the composite command) and (b) **communication optimization** (PE_IPCQ). Frame the three result sections as *two enablers → the fused GQA capstone that uses both*. Motivate why HW-SW codesign (not software alone) is required to realize the gains. 3. **The KernBench Platform** 1. *Why KernBench* — source-level kernel execution with **no compiler / software-stack dependency**; isolate algorithm-level optimization from other layers so a kernel can fully exploit the hardware without E2E cross-layer entanglement. 2. *Execution model* — discrete-event graph; `runtime API → sim_engine → components`; PE pipeline + composite commands; 2-pass data/timing model. 3. *Latency model & accuracy* — traversal-based golden invariants (every hop pays > 0, deterministic, explicit connectivity); the structural CPU-dispatch cost model; per-edge BW occupancy, HBM pseudo-channel parallelism, flit streaming; known simplifications & what calibration anchors the constants. 4. *Modeled Hardware Configuration* — the shared, platform-wide quantitative config used across **all** experiments, read from `topology.yaml`: SIP/CUBE/PE hierarchy (counts & nesting), PE clock & compute throughput, TCM/SRAM/HBM capacities & bandwidths, NOC / inter-CUBE / inter-SIP link bandwidths & latencies, and the cost-model constants. Per-experiment workload parameters (GEMM shapes, All-Reduce topologies, GQA seq/head/user counts) stay in each evaluation section, not here. 4. **GEMM Acceleration via Composite Command** — necessity · design · results · analysis. 5. **All-Reduce Acceleration via PE_IPCQ** — necessity · design · results · analysis. 6. **Fused Grouped Query Attention** (composite + PE_IPCQ) — necessity · design · results · analysis. 7. **Discussion** — which HW changes are meaningful, and why (cross-cutting). 8. **Conclusion**. 9. **Future Work — 2H** — add the FFN/MoE layer toward full LLM decoding; analyze how compute & data should be distributed for agentic / MoE workloads. 10. **References** *(optional)* — **external literature only** (e.g. FlashAttention, Megatron-LM, GPT-3, Llama 3). **No ADR/SPEC entries.** Each of §4/§5/§6 follows the same four-beat structure: **(a) why the optimization is needed**, **(b) the design (how it works)**, **(c) experimental results** (run/plot/measure), **(d) analysis & meaning**. --- ## Source map (ADR / SPEC → section) Use these as **grounding sources only** — read them for content, but **never name an ADR or SPEC in the paper**. Verify each quantitative claim against the actual file before asserting it (CLAUDE.md: no asserting architectural numbers from memory). - **Modeled Hardware Configuration (§3.4)**: `topology.yaml` (authoritative for counts, clocks, BW, link params, cost-model constants) is the **primary** source — read it directly and build the HW table from it. Cross-check against ADR-0003 (hierarchy/scope), ADR-0004 (memory BW guarantee), ADR-0033 (latency model), ADR-0064 (cost-model defaults). If `topology.yaml` and an ADR disagree, the loaded `topology.yaml` is what the sim actually runs — report that value and flag the mismatch. - **Platform — why/execution/latency**: `SPEC §0`, `SPEC §0.1` (golden invariants), `SPEC §1 R1/R2/R3`, `SPEC §2.1` (graph execution); ADR-0003 (hierarchy/scope), ADR-0007 (runtime↔sim_engine boundary), ADR-0014 (PE pipeline / composite), ADR-0020 (2-pass data/timing), ADR-0033 (latency model assumptions & simplifications), ADR-0064 (structural CPU-dispatch cost model: `FIXED + logical_bytes×R`). - **GEMM / composite**: ADR-0014 (composite command D3.2/D3.3), ADR-0042 (tile-plan generators), ADR-0044 (GEMM eval harness), ADR-0064 (cost model — composite's per-command BW reduction), ADR-0065 (flat-ops `CompositeCmd`, *Proposed*). - **All-Reduce / PE_IPCQ**: ADR-0023 (PE_IPCQ — control plane, ring buffers, VC split), ADR-0025 (direction addressing), ADR-0032 (intercube all-reduce), ADR-0043 (allreduce eval harness), ADR-0047 (AHBM CCL backend), ADR-0050 (CCL algorithm contract). - **GQA / fused attention**: ADR-0060 (fused GQA kernel), ADR-0061 (`tl.broadcast`/`tl.repeat`, *Proposed*), ADR-0062 (lazy `tl.load`), ADR-0063 (`tl.scratch_scope`), ADR-0064 (cost model), ADR-0065 (flat-ops composite + `softmax_merge`, *Proposed*). KO mirrors in `docs/adr-ko/`; DDD-0060/0065 design docs if present. - **Eval/milestone pattern**: ADR-0054 (self-contained milestone benches); ADR-0045 (bench contract). --- ## Result map (bench → artifact → section) + run commands All runs use the repo's `topology.yaml` unless a section needs a specific topology. Reuse committed artifacts unless told to regenerate. ### §4 GEMM — `milestone-1h-gemm` - Artifacts (committed): `src/kernbench/benches/1H_milestone_output/gemm/` → `gemm_sweep.json`, `gemm_stage_breakdown.png`, `gemm_mac_utilization_measured.png`, `gemm_mac_utilization_theoretical_vs_measured.png`. - Regenerate: `kernbench run --topology topology.yaml --bench milestone-1h-gemm` (full 24-shape sweep ≈ minutes; `MILESTONE_FAST=1` reuses committed JSON). ### §5 All-Reduce — `milestone-1h-ccl` - Artifacts (committed): `src/kernbench/benches/1H_milestone_output/ccl/` → `summary.csv`, `topology.png`, per-topology `AllReduce_LRAB_*` PNGs, buffer-kind sweep `AllReduce_LRAB_2Dtorus_..._with_TCM_SRAM_HBM.{png,csv}`, `comparison_mesh_vs_ring_vs_2DTorus_vs_theoretical_vs_fsim.png`. - Regenerate: `kernbench run --topology topology.yaml --bench milestone-1h-ccl`. ### §6 GQA — `milestone-gqa-headline` - Artifact (committed): `src/kernbench/benches/1H_milestone_output/gqa_headline/sweep.json`. **Caveat:** this currently holds only `op_log_summary` **op-counts** (`gemm_count`, `ipcq_copy_count`, `dma_read_count`, `dma_write_count`) for 4 panels (single/multi-user × prefill/decode) — **no latency, no figures**. - Regenerate counts: `GQA_HEADLINE_RUN=1 kernbench run --topology topology.yaml --bench milestone-gqa-headline`. - For a real GQA **results** section, generate latency + figures via an isolated report harness — see next. --- ## Generating missing results (isolation rule) When a section needs an artifact that does not yet exist (today: GQA latency and any GQA figure), create a **report-dedicated harness** under `scripts/paper/` — never under `src/kernbench/benches/`. Rules: - One file per concern, prefixed `paper_`, e.g. `scripts/paper/paper_gqa_latency.py` (drives the engine, emits latency + a baseline-vs-optimized comparison), `scripts/paper/paper_plot_gqa.py` (renders figures from that data). - Follow the **ADR-0054 self-contained pattern**: the harness builds its own `GraphEngine`(s), imports the existing GQA kernel modules (`_gqa_attention_{prefill,decode}_{short,long}.py`, `milestone_gqa_headline.py`), runs them, and reads latency from the engine's completion timestamps / op log. It does **not** register a bench and does **not** import or mutate other people's benches. - Output figures land in `docs/report/1H-codesign-paper/figures/` and, if useful as a shared diagram, mirror into `docs/diagrams/gqa_plots/`. - **Only measure what the current code actually runs.** ADR-0065's GQA opt2 (`softmax_merge`, two-composite decode) is *Proposed*; if it is not wired in production, present it as designed-not-measured and base the measured results on the implemented path. Verify implementation status by reading the code before claiming a measurement. - These harnesses are derived-artifact tooling (no Phase 2 needed). **But** if the report ever requires changing a *registered production bench* under `src/kernbench/benches/` or any production module, that is a non-trivial production change — STOP and follow CLAUDE.md Phase 1 → approval → Phase 2. --- ## Procedure ### Step 0 — Toolchain (Tectonic) Ensure a working Tectonic before any compile step. In order: 1. If `tectonic` is on PATH (`tectonic --version` succeeds), use it. 2. Else try, in order, whichever succeeds (capture output, don't hang): - `winget install --id TectonicProject.Tectonic --silent --accept-package-agreements --accept-source-agreements` - `scoop install tectonic` - Direct download (no admin): fetch the latest Windows `x86_64-pc-windows-msvc` release zip from `https://github.com/tectonic-typesetting/tectonic/releases` via PowerShell `Invoke-WebRequest`, expand it, and place `tectonic.exe` under `docs/report/1H-codesign-paper/build/.tools/`. Use that absolute path for compilation. 3. Re-verify with `tectonic --version`. If all methods fail, write the `.tex` files anyway, **report the failure**, and tell the user the exact command to compile manually (or to use Overleaf). Do not block the rest of the pipeline. Tectonic fetches LaTeX packages on first compile (needs network once). ### Step 1 — Period + TOC - Period from system date: month 1–6 → `{YYYY}-1H`, else `{YYYY}-2H`. (Report dir is fixed at `1H-codesign-paper` for this half; adjust the title page text with the period.) - If `toc.md` exists, present it; else present the **Default TOC**. - **Stop and ask the user to confirm or edit.** Incorporate edits. Persist the agreed outline to `toc.md`. (In `/paper build` mode, skip the stop and trust the saved `toc.md`.) ### Step 2 — Platform section (§3, incl. §3.4 HW config) Write `sections/02-platform.tex` from the platform source map, covering all four sub-parts: - §3.1–3.3 (why / execution model / latency model + accuracy). Include the cost-model formula `dispatch_cycles = FIXED + logical_bytes×R` with the committed defaults (verify against ADR-0064 before quoting), and explicitly address **how accurate** the latency is: what is modeled precisely (per-edge BW occupancy, HBM pseudo-channels, flit streaming, per-component overhead) and the known simplifications. - §3.4 *Modeled Hardware Configuration*: read `topology.yaml` directly and build a compact, table-driven summary — SIP/CUBE/PE hierarchy (counts & nesting), PE clock & compute throughput, TCM/SRAM/HBM capacity & bandwidth, NOC / inter-CUBE / inter-SIP link bandwidth & latency, and the cost-model constants (`FIXED_PER_CMD`, `R`, composite cap). Every number from `topology.yaml`. This is the shared config; later sections say "the modeled hardware" and only add their own workload params. Do not name the source ADRs/SPEC in the prose. ### Step 3 — The three result sections (§4 GEMM, §5 All-Reduce, §6 GQA) For each, in order, produce the four beats: - **(a) Necessity** — from the ADR Context: the bottleneck the optimization targets (e.g. per-command dispatch overhead for GEMM; collective comm cost / HoL blocking for All-Reduce; KV-load-bound decode + softmax merge for GQA). - **(b) Design** — from the ADR Decisions: composite command tile pipeline (ADR-0014); PE_IPCQ control/data split + ring buffers + VC (ADR-0023); fused attention with online-softmax, lazy load, scratch recycling (ADR-0060/0062/0063). Note Proposed-vs-Accepted status. - **(c) Results** — reuse committed artifacts (GEMM, All-Reduce). For GQA, generate latency + figures via the isolated `scripts/paper/` harness (above). Copy chosen PNGs into `figures/`, `\includegraphics` them, and build result tables from the CSV/JSON — quoting actual numbers, with a one-line note of the run command and source artifact. - **(d) Analysis** — interpret: what the curve/table shows, why the HW feature produces it, where it saturates or fails to help, and the cost (area/complexity) implied. Tie back to the cost model. ### Step 4 — Discussion, Conclusion, Future Work (§6/§7/§8) - **Discussion**: synthesize across the three — which HW changes (composite command, PE_IPCQ, the cost-model-exposed BW reduction) carry their weight, and under what regimes. - **Conclusion**: the codesign thesis, supported by the measured results; state plainly which HW changes are meaningful. - **Future Work (2H)**: FFN / MoE layer toward full LLM decoding; how compute & data should be distributed for agentic / MoE workloads (expert routing, token dispersion, all-to-all vs all-reduce, capacity). Frame as the next codesign questions, not claims. ### Step 5 — Assemble & compile - `main.tex`: article/IEEEtran-style preamble, title with the period, author, executive-summary input, then `\input{sections/...}` in TOC order, then references. Keep the preamble minimal (graphicx, booktabs, hyperref, amsmath) so Tectonic compiles cleanly. - Compile: `tectonic main.tex --outdir build` (or the resolved Tectonic path) from `docs/report/1H-codesign-paper/`. Re-run if references need a second pass. Confirm `build/main.pdf` exists and report its size. ### Step 6 — Chat report Emit to chat (not to any file): ``` ## /paper — Build Summary **PDF:** docs/report/1H-codesign-paper/build/main.pdf ( pages, ) **Toolchain:** tectonic **Sections written:** **Benches: reused vs regenerated** - GEMM: - All-Reduce: - GQA: **Figures included:** **Grounding check:** every result number traced to **Proposed-not-measured items flagged:** **Open gaps / suggested next runs:** <…> ``` --- ## Constraints (do not violate) 1. **TOC first.** Never write sections before the TOC is confirmed (default mode). `toc.md` is the contract. 2. **No fabricated numbers.** Every quantitative claim traces to a committed artifact, a fresh bench run, or an ADR/SPEC quote. Verify architectural constants against the source file, not memory. 3. **Reuse by default.** Do not re-run a bench unless asked or the artifact is missing/stale. 4. **Isolation.** Report harnesses live only in `scripts/paper/`; never register a bench, never touch other people's benches. 5. **Production-change gate.** Any change to a registered bench or production module triggers CLAUDE.md Phase 1 → approval → Phase 2. Report tooling and `docs/report/...` are derived artifacts (exempt). 6. **Status honesty.** Mark Proposed designs as proposed; never present an unimplemented path as measured. 7. **No internal references.** Never name `ADR-NNNN` or `SPEC §X.Y` in prose, captions, or the reference list — they are grounding sources only. A References section, if present, holds **external literature only**. Hardware numbers come from `topology.yaml`. 8. **Determinism.** Sections in TOC order; same inputs → same paper. 9. **English prose**, internal-audience tone. ```