paper: add /paper skill + 1H HW-SW codesign report (GEMM, All-Reduce, fused GQA)

New `/paper` slash-command skill that synthesizes ADR/SPEC content and live
KernBench benchmark results into a sectioned LaTeX technical paper compiled
to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds
every number in committed artifacts or fresh bench runs, and keeps
report-only benches isolated.

This commit also includes the first generated report:
- docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures,
  toc.md contract, and the built 8-page main.pdf. Covers the platform
  (source-level kernels, latency model + accuracy, HW config from
  topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and
  fused GQA combining both, plus discussion/conclusion/2H future work.
- scripts/paper/ — isolated report harnesses (not registered benches):
  paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine
  occupancy (the milestone only emitted op-counts); paper_plot_gqa.py
  renders the GQA figures.

GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are
generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this
config, and the proposed two-composite softmax_merge decode is marked
designed-not-measured.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 22:15:14 -07:00
parent 4a55ae5c0b
commit dd525bfcb7
25 changed files with 1561 additions and 0 deletions
@@ -0,0 +1,39 @@
\section*{Executive Summary}
\addcontentsline{toc}{section}{Executive Summary}
This report studies how to make attention-centric LLM kernels fast through
hardware--software codesign, using KernBench, a discrete-event simulator
that runs kernels at the source level---without a compiler or other
software-stack dependency---so that algorithm-level optimizations and the
hardware features that support them can be isolated and measured.
The 1H work centers on attention. Building on prior multi-head-attention
work, we target FlashAttention-style Grouped-Query Attention (GQA) and find
that a fast fused attention kernel rests on two enabling optimizations,
each studied in its own right and then combined inside the fused kernel:
a \textbf{composite command} that issues a tiled GEMM as a single,
self-routing pipeline, and \textbf{PE\_IPCQ}, a per-PE on-device collective
engine for all-reduce.
The measured results form a single, consistent story. For GEMM, the
composite command lets compute-rich shapes reach
\textasciitilde\SI{78}{\percent} of peak MAC efficiency, with the simulator's
measured efficiency tracking the analytic prediction within
\SIrange{10}{20}{\percent}---both a performance result and a validation of
the latency model. For all-reduce, PE\_IPCQ delivers on-device collectives
whose latency tracks the interconnect's physical limits; a 2D-torus
inter-device fabric is \SI{20}{}--\SI{25}{\percent} faster than a mesh or
ring at scale, and staging the collective in on-PE TCM rather than HBM or
SRAM saves a further \SI{14}{}--\SI{38}{\percent}. For fused GQA, the two
enablers combine: because the composite command keeps the MAC array nearly
idle, the kernel is exposed as overwhelmingly data-movement bound---the DMA
engine outweighs the compute engines by two to three orders of magnitude,
and on-device IPCQ reduction traffic scales with users and devices.
\textbf{Bottom line.} For an attention-dominated decoder, the hardware
changes worth keeping are the ones that move and reduce data faster: the
single-command self-routing GEMM pipeline (which makes the MAC array
usable and exposes the real bottleneck), the on-device PE\_IPCQ collective
with its compute/communication virtual-channel split, fast on-PE staging
memory, and wrap-around (torus) inter-device links. Additional raw MAC
throughput is not the limiting factor for this workload.