# KernBench — 2026 1H HW-SW Codesign Report — Table of Contents

> Agreed outline (the `/paper` contract). `/paper build` follows this.
> Period: 2026 1H. Language: English. References: external literature only
> (no ADR/SPEC named anywhere in the paper). PDF via Tectonic.

## Section order → files

1. **Executive Summary** — `sections/00-exec-summary.tex`
   The attention-optimization goal, the two enabling optimizations (GEMM
   composite command + PE_IPCQ communication), the fused GQA capstone, the
   headline results, and the bottom-line recommendation on which HW changes
   are worth keeping. Front-loads conclusions; may run slightly longer than
   a terse abstract.

2. **Introduction** — `sections/01-introduction.tex`
   1H focus is **attention-kernel optimization**: FlashAttention-style
   tiling and Grouped Query Attention (GQA), building on prior work on
   Multi-Head Attention (MHA, studied earlier — referenced as the
   established baseline, not re-derived). Optimizing a fused attention
   kernel requires two enabling optimizations, each studied in its own
   right and then **combined inside the fused kernel**:
   (a) **GEMM optimization** (the composite command), and
   (b) **communication optimization** (PE_IPCQ).
   Narrative arc: *two enablers → the fused GQA capstone that uses both.*
   Motivate why HW-SW codesign (not software alone) is required.

3. **The KernBench Platform** — `sections/02-platform.tex`
   - 3.1 *Why KernBench* — source-level kernel execution, no compiler /
     SW-stack dependency; isolate algorithm-level optimization.
   - 3.2 *Execution model* — discrete-event graph; runtime API → sim_engine
     → components; PE pipeline + composite commands; 2-pass data/timing.
   - 3.3 *Latency model & accuracy* — traversal-based golden invariants;
     structural CPU-dispatch cost model; known simplifications + calibration.
   - 3.4 *Modeled Hardware Configuration* — shared platform-wide config from
     `topology.yaml` (hierarchy counts, clocks, capacities/BW, link params,
     cost-model constants). Per-experiment workload params live in §4–§6.

4. **GEMM Acceleration via Composite Command** — `sections/03-gemm.tex`
   necessity · design · results · analysis. (+ GEMM workload params)

5. **All-Reduce Acceleration via PE_IPCQ** — `sections/04-allreduce.tex`
   necessity · design · results · analysis. (+ All-Reduce topologies/sizes)

6. **Fused Grouped Query Attention** (composite + PE_IPCQ) — `sections/05-gqa.tex`
   necessity · design · results · analysis. (+ GQA seq/head/user configs)

7. **Supporting Agentic Workloads** — `sections/06-agentic.tex`
   How the §6 GQA design extends to agentic fan-out/fan-in: shared-prefix KV
   reuse, batched replicated-Q attention, schema/pointer-based fan-in.
   Implementation split across three levels (agentic framework / runtime /
   kernel) with each component's responsibilities. Design, not yet measured.

8. **Hardware Performance-Spec Search for GQA** — `sections/07-hw-spec-search.tex`
   *Work in progress.* Joint sweep of GEMM TFLOPS, MATH-engine ALU count,
   CUBE↔CUBE (die-to-die) BW, SIP↔SIP (card-to-card) BW to find the balanced
   spec for GQA. Intent + method only; data/figures/conclusions deferred.

9. **Discussion** — `sections/08-discussion.tex`
   Which HW changes are meaningful, and under what regimes (cross-cutting).

10. **Conclusion** — `sections/09-conclusion.tex`
    The codesign thesis, stated plainly, supported by the measured results.

11. **Future Work — 2H** — `sections/10-future-work.tex`
    Add the FFN/MoE layer toward full LLM decoding; how compute & data should
    be distributed for agentic / MoE workloads.

12. **References** *(optional)* — external literature only (FlashAttention,
    Megatron-LM, GPT-3, Llama 3). No ADR/SPEC entries.

## Per-section structure (§4/§5/§6)

Each of the three optimization sections follows the same four beats:
**(a) necessity → (b) design → (c) experimental results → (d) analysis & meaning.**

## Build notes

- GEMM (§4) and All-Reduce (§5): reuse committed figures/CSVs under
  `src/kernbench/benches/1H_milestone_output/{gemm,ccl}/`.
- GQA (§6): committed `sweep.json` holds op-counts only — latency + figures
  are generated by an isolated harness under `scripts/paper/` at build time.
  Measure only the implemented path; mark proposed designs (e.g. the
  flat-ops `softmax_merge` decode opt2) as designed-not-measured.