paper: add /paper skill + 1H HW-SW codesign report (GEMM, All-Reduce, fused GQA)
New `/paper` slash-command skill that synthesizes ADR/SPEC content and live KernBench benchmark results into a sectioned LaTeX technical paper compiled to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds every number in committed artifacts or fresh bench runs, and keeps report-only benches isolated. This commit also includes the first generated report: - docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures, toc.md contract, and the built 8-page main.pdf. Covers the platform (source-level kernels, latency model + accuracy, HW config from topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and fused GQA combining both, plus discussion/conclusion/2H future work. - scripts/paper/ — isolated report harnesses (not registered benches): paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine occupancy (the milestone only emitted op-counts); paper_plot_gqa.py renders the GQA figures. GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this config, and the proposed two-composite softmax_merge decode is marked designed-not-measured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,73 @@
|
||||
# KernBench — 2026 1H HW-SW Codesign Report — Table of Contents
|
||||
|
||||
> Agreed outline (the `/paper` contract). `/paper build` follows this.
|
||||
> Period: 2026 1H. Language: English. References: external literature only
|
||||
> (no ADR/SPEC named anywhere in the paper). PDF via Tectonic.
|
||||
|
||||
## Section order → files
|
||||
|
||||
1. **Executive Summary** — `sections/00-exec-summary.tex`
|
||||
The attention-optimization goal, the two enabling optimizations (GEMM
|
||||
composite command + PE_IPCQ communication), the fused GQA capstone, the
|
||||
headline results, and the bottom-line recommendation on which HW changes
|
||||
are worth keeping. Front-loads conclusions; may run slightly longer than
|
||||
a terse abstract.
|
||||
|
||||
2. **Introduction** — `sections/01-introduction.tex`
|
||||
1H focus is **attention-kernel optimization**: FlashAttention-style
|
||||
tiling and Grouped Query Attention (GQA), building on prior work on
|
||||
Multi-Head Attention (MHA, studied earlier — referenced as the
|
||||
established baseline, not re-derived). Optimizing a fused attention
|
||||
kernel requires two enabling optimizations, each studied in its own
|
||||
right and then **combined inside the fused kernel**:
|
||||
(a) **GEMM optimization** (the composite command), and
|
||||
(b) **communication optimization** (PE_IPCQ).
|
||||
Narrative arc: *two enablers → the fused GQA capstone that uses both.*
|
||||
Motivate why HW-SW codesign (not software alone) is required.
|
||||
|
||||
3. **The KernBench Platform** — `sections/02-platform.tex`
|
||||
- 3.1 *Why KernBench* — source-level kernel execution, no compiler /
|
||||
SW-stack dependency; isolate algorithm-level optimization.
|
||||
- 3.2 *Execution model* — discrete-event graph; runtime API → sim_engine
|
||||
→ components; PE pipeline + composite commands; 2-pass data/timing.
|
||||
- 3.3 *Latency model & accuracy* — traversal-based golden invariants;
|
||||
structural CPU-dispatch cost model; known simplifications + calibration.
|
||||
- 3.4 *Modeled Hardware Configuration* — shared platform-wide config from
|
||||
`topology.yaml` (hierarchy counts, clocks, capacities/BW, link params,
|
||||
cost-model constants). Per-experiment workload params live in §4–§6.
|
||||
|
||||
4. **GEMM Acceleration via Composite Command** — `sections/03-gemm.tex`
|
||||
necessity · design · results · analysis. (+ GEMM workload params)
|
||||
|
||||
5. **All-Reduce Acceleration via PE_IPCQ** — `sections/04-allreduce.tex`
|
||||
necessity · design · results · analysis. (+ All-Reduce topologies/sizes)
|
||||
|
||||
6. **Fused Grouped Query Attention** (composite + PE_IPCQ) — `sections/05-gqa.tex`
|
||||
necessity · design · results · analysis. (+ GQA seq/head/user configs)
|
||||
|
||||
7. **Discussion** — `sections/06-discussion.tex`
|
||||
Which HW changes are meaningful, and under what regimes (cross-cutting).
|
||||
|
||||
8. **Conclusion** — `sections/07-conclusion.tex`
|
||||
The codesign thesis, stated plainly, supported by the measured results.
|
||||
|
||||
9. **Future Work — 2H** — `sections/08-future-work.tex`
|
||||
Add the FFN/MoE layer toward full LLM decoding; how compute & data should
|
||||
be distributed for agentic / MoE workloads.
|
||||
|
||||
10. **References** *(optional)* — external literature only (FlashAttention,
|
||||
Megatron-LM, GPT-3, Llama 3). No ADR/SPEC entries.
|
||||
|
||||
## Per-section structure (§4/§5/§6)
|
||||
|
||||
Each of the three optimization sections follows the same four beats:
|
||||
**(a) necessity → (b) design → (c) experimental results → (d) analysis & meaning.**
|
||||
|
||||
## Build notes
|
||||
|
||||
- GEMM (§4) and All-Reduce (§5): reuse committed figures/CSVs under
|
||||
`src/kernbench/benches/1H_milestone_output/{gemm,ccl}/`.
|
||||
- GQA (§6): committed `sweep.json` holds op-counts only — latency + figures
|
||||
are generated by an isolated harness under `scripts/paper/` at build time.
|
||||
Measure only the implemented path; mark proposed designs (e.g. the
|
||||
flat-ops `softmax_merge` decode opt2) as designed-not-measured.
|
||||
Reference in New Issue
Block a user