ywkang
dd525bfcb7
New `/paper` slash-command skill that synthesizes ADR/SPEC content and live KernBench benchmark results into a sectioned LaTeX technical paper compiled to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds every number in committed artifacts or fresh bench runs, and keeps report-only benches isolated. This commit also includes the first generated report: - docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures, toc.md contract, and the built 8-page main.pdf. Covers the platform (source-level kernels, latency model + accuracy, HW config from topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and fused GQA combining both, plus discussion/conclusion/2H future work. - scripts/paper/ — isolated report harnesses (not registered benches): paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine occupancy (the milestone only emitted op-counts); paper_plot_gqa.py renders the GQA figures. GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this config, and the proposed two-composite softmax_merge decode is marked designed-not-measured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
28 lines
1.6 KiB
TeX
28 lines
1.6 KiB
TeX
\section{Conclusion}
|
|
\label{sec:conclusion}
|
|
|
|
This 1H work set out to make attention-centric LLM kernels fast through
|
|
hardware--software codesign, and to do so on a platform that isolates
|
|
algorithm-level behavior from the rest of the software stack. The result is
|
|
a coherent picture rather than three separate optimizations. A composite
|
|
command that issues a tiled GEMM as one self-routing pipeline makes the MAC
|
|
array usable---reaching \textasciitilde\SI{78}{\percent} of peak on
|
|
compute-rich shapes with measured efficiency tracking theory---and, just as
|
|
importantly, makes compute cheap enough that the real bottleneck becomes
|
|
visible. A per-PE on-device collective engine, PE\_IPCQ, turns all-reduce
|
|
into a primitive whose latency follows the interconnect's physical limits,
|
|
with topology and staging-memory choices each worth tens of percent. Fused
|
|
Grouped-Query Attention then combines the two and shows the payoff and the
|
|
lesson at once: the kernel is data-movement bound, so the optimizations
|
|
that move and reduce data---not those that add arithmetic---are what
|
|
determine its speed.
|
|
|
|
The practical conclusion for the hardware roadmap is therefore specific.
|
|
The changes worth keeping are the single-command self-routing GEMM
|
|
pipeline, the on-device PE\_IPCQ collective with its compute/communication
|
|
virtual-channel split, fast on-PE staging memory, and wrap-around
|
|
inter-device links. Additional MAC throughput is not, for this workload, a
|
|
meaningful investment. KernBench made these conclusions measurable by
|
|
holding everything except the algorithm and the hardware fixed; the next
|
|
half extends the same method beyond attention to the rest of the decoder.
|