paper: add /paper skill + 1H HW-SW codesign report (GEMM, All-Reduce, fused GQA)
New `/paper` slash-command skill that synthesizes ADR/SPEC content and live KernBench benchmark results into a sectioned LaTeX technical paper compiled to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds every number in committed artifacts or fresh bench runs, and keeps report-only benches isolated. This commit also includes the first generated report: - docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures, toc.md contract, and the built 8-page main.pdf. Covers the platform (source-level kernels, latency model + accuracy, HW config from topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and fused GQA combining both, plus discussion/conclusion/2H future work. - scripts/paper/ — isolated report harnesses (not registered benches): paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine occupancy (the milestone only emitted op-counts); paper_plot_gqa.py renders the GQA figures. GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this config, and the proposed two-composite softmax_merge decode is marked designed-not-measured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,39 @@
|
||||
\section*{Executive Summary}
|
||||
\addcontentsline{toc}{section}{Executive Summary}
|
||||
|
||||
This report studies how to make attention-centric LLM kernels fast through
|
||||
hardware--software codesign, using KernBench, a discrete-event simulator
|
||||
that runs kernels at the source level---without a compiler or other
|
||||
software-stack dependency---so that algorithm-level optimizations and the
|
||||
hardware features that support them can be isolated and measured.
|
||||
|
||||
The 1H work centers on attention. Building on prior multi-head-attention
|
||||
work, we target FlashAttention-style Grouped-Query Attention (GQA) and find
|
||||
that a fast fused attention kernel rests on two enabling optimizations,
|
||||
each studied in its own right and then combined inside the fused kernel:
|
||||
a \textbf{composite command} that issues a tiled GEMM as a single,
|
||||
self-routing pipeline, and \textbf{PE\_IPCQ}, a per-PE on-device collective
|
||||
engine for all-reduce.
|
||||
|
||||
The measured results form a single, consistent story. For GEMM, the
|
||||
composite command lets compute-rich shapes reach
|
||||
\textasciitilde\SI{78}{\percent} of peak MAC efficiency, with the simulator's
|
||||
measured efficiency tracking the analytic prediction within
|
||||
\SIrange{10}{20}{\percent}---both a performance result and a validation of
|
||||
the latency model. For all-reduce, PE\_IPCQ delivers on-device collectives
|
||||
whose latency tracks the interconnect's physical limits; a 2D-torus
|
||||
inter-device fabric is \SI{20}{}--\SI{25}{\percent} faster than a mesh or
|
||||
ring at scale, and staging the collective in on-PE TCM rather than HBM or
|
||||
SRAM saves a further \SI{14}{}--\SI{38}{\percent}. For fused GQA, the two
|
||||
enablers combine: because the composite command keeps the MAC array nearly
|
||||
idle, the kernel is exposed as overwhelmingly data-movement bound---the DMA
|
||||
engine outweighs the compute engines by two to three orders of magnitude,
|
||||
and on-device IPCQ reduction traffic scales with users and devices.
|
||||
|
||||
\textbf{Bottom line.} For an attention-dominated decoder, the hardware
|
||||
changes worth keeping are the ones that move and reduce data faster: the
|
||||
single-command self-routing GEMM pipeline (which makes the MAC array
|
||||
usable and exposes the real bottleneck), the on-device PE\_IPCQ collective
|
||||
with its compute/communication virtual-channel split, fast on-PE staging
|
||||
memory, and wrap-around (torus) inter-device links. Additional raw MAC
|
||||
throughput is not the limiting factor for this workload.
|
||||
Reference in New Issue
Block a user