paper: add /paper skill + 1H HW-SW codesign report (GEMM, All-Reduce, fused GQA)
New `/paper` slash-command skill that synthesizes ADR/SPEC content and live KernBench benchmark results into a sectioned LaTeX technical paper compiled to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds every number in committed artifacts or fresh bench runs, and keeps report-only benches isolated. This commit also includes the first generated report: - docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures, toc.md contract, and the built 8-page main.pdf. Covers the platform (source-level kernels, latency model + accuracy, HW config from topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and fused GQA combining both, plus discussion/conclusion/2H future work. - scripts/paper/ — isolated report harnesses (not registered benches): paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine occupancy (the milestone only emitted op-counts); paper_plot_gqa.py renders the GQA figures. GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this config, and the proposed two-composite softmax_merge decode is marked designed-not-measured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,113 @@
|
||||
\section{All-Reduce Acceleration via PE\_IPCQ}
|
||||
\label{sec:allreduce}
|
||||
|
||||
\subsection{Why it is needed}
|
||||
|
||||
Distributing a transformer across devices turns every tensor-parallel
|
||||
layer into a collective: partial results computed on different PEs, CUBEs,
|
||||
and SIPs must be summed and redistributed with an all-reduce. If that
|
||||
collective is handled by the host or by a generic DMA path, three problems
|
||||
appear. The reduction traffic competes with the kernel's own compute DMA
|
||||
on the same links, causing head-of-line blocking; there is no efficient
|
||||
peer-to-peer ring primitive, so data takes extra hops; and the ordering is
|
||||
hard to make deterministic. The hardware question is how to perform
|
||||
collectives \emph{on the device}, overlapped with compute and reproducible
|
||||
run-to-run.
|
||||
|
||||
\subsection{Design}
|
||||
|
||||
KernBench models a dedicated per-PE collective engine, \textbf{PE\_IPCQ}
|
||||
(inter-PE communication queue). It is a control-plane block: it owns the
|
||||
ring-buffer address arithmetic, head/tail pointers, peer-pointer caches,
|
||||
backpressure, and the four-direction (N/S/E/W) neighbor map, with eight
|
||||
ring buffers per PE (four directions $\times$ \{tx, rx\}). Crucially,
|
||||
PE\_IPCQ does \emph{not} move data itself---it delegates the actual
|
||||
transfer to PE\_DMA, keeping a clean control/data split. To stop
|
||||
collective traffic from blocking compute, PE\_DMA is extended into a
|
||||
two-channel virtual-channel model: \texttt{vc\_compute} carries tile
|
||||
load/store for GEMM and vector math, \texttt{vc\_comm} carries IPCQ sends,
|
||||
each with an independent state machine. The same physical link is shared
|
||||
but progresses in chunks (\SI{256}{\byte}), so a large GEMM DMA does not
|
||||
lock the link end-to-end against a pending reduction. On top of this
|
||||
substrate the collective runs a hierarchical local-reduce / global
|
||||
all-reduce-broadcast schedule across whatever inter-device topology the
|
||||
configuration specifies.
|
||||
|
||||
\subsection{Results}
|
||||
|
||||
Following the milestone-evaluation convention, the collective sweep builds
|
||||
its own six-device (six-SIP, $2\times3$) configurations---distinct from
|
||||
the two-SIP default of Table~\ref{tab:hw}---and measures all-reduce
|
||||
latency as a function of payload size for three inter-device topologies:
|
||||
a 1D ring, a 2D mesh (no wrap), and a 2D torus. Table~\ref{tab:allreduce}
|
||||
and Figure~\ref{fig:allreduce-cmp} report the result.
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
\caption{All-reduce latency (ns) across six devices, by topology and
|
||||
per-PE payload. Lower is better; the torus wins at every size.}
|
||||
\label{tab:allreduce}
|
||||
\small
|
||||
\begin{tabular}{@{}rrrr@{}}
|
||||
\toprule
|
||||
\textbf{Bytes/PE} & \textbf{2D mesh} & \textbf{Ring 1D} & \textbf{2D torus} \\
|
||||
\midrule
|
||||
256 & 2667 & 2365 & 1701 \\
|
||||
4{,}096 & 4450 & 4082 & 3038 \\
|
||||
16{,}384 & 8900 & 8217 & 6403 \\
|
||||
65{,}536 & 26705 & 24766 & 19865 \\
|
||||
98{,}304 & 38574 & 35798 & 28840 \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\end{table}
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=\linewidth]{allreduce_comparison.png}
|
||||
\caption{All-reduce latency vs.\ per-PE payload for the three topologies,
|
||||
against the analytic torus model and an external full-system simulator
|
||||
(FSIM) reference. The measured torus tracks the analytic curve within a
|
||||
small constant factor; the FSIM single-device point
|
||||
(\SI{366}{\micro\second}) sits an order of magnitude above the
|
||||
KernBench algorithmic latency, illustrating the difference between an
|
||||
achievable-kernel number and a full end-to-end-stack number.}
|
||||
\label{fig:allreduce-cmp}
|
||||
\end{figure}
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=\linewidth]{allreduce_buffer_kind.png}
|
||||
\caption{Effect of IPCQ staging-buffer placement (2D torus). At
|
||||
\SI{64}{\kibi\byte}/PE, TCM staging (\SI{19865}{\nano\second}) beats HBM
|
||||
(\SI{23081}{\nano\second}) by \textasciitilde\SI{14}{\percent} and SRAM
|
||||
(\SI{32201}{\nano\second}) by \textasciitilde\SI{38}{\percent}; at small
|
||||
payloads the three are indistinguishable.}
|
||||
\label{fig:allreduce-buf}
|
||||
\end{figure}
|
||||
|
||||
Three findings follow. First, topology matters and the effect grows with
|
||||
size: at \SI{96}{\kilo\byte}/PE the 2D torus is \SI{25}{\percent} faster
|
||||
than the mesh and \SI{19}{\percent} faster than the ring, because its
|
||||
wrap-around links shorten the worst-case reduction path. Second, the
|
||||
measured curves grow smoothly with payload and stay within a small
|
||||
constant factor of the analytic torus model, with the gap reflecting real
|
||||
link serialization that the analytic model idealizes away. Third, where
|
||||
the IPCQ staging buffer lives is a first-order knob: keeping it in on-PE
|
||||
TCM is materially faster than HBM or shared SRAM at large payloads
|
||||
(Figure~\ref{fig:allreduce-buf}).
|
||||
|
||||
\subsection{Analysis and meaning}
|
||||
|
||||
PE\_IPCQ turns the collective from an off-device, contention-prone
|
||||
operation into an on-device primitive whose latency tracks the
|
||||
interconnect's physical limits. The control/data split keeps the engine
|
||||
small while reusing PE\_DMA for movement, and the virtual-channel split is
|
||||
what lets a reduction proceed without stalling the compute DMA that feeds
|
||||
the very GEMMs producing the partials---the same property the fused
|
||||
attention kernel relies on when it interleaves KV reduction with score
|
||||
computation (\S\ref{sec:gqa}). For the hardware roadmap the results argue
|
||||
two things: provisioning wrap-around (torus) inter-device links is worth
|
||||
roughly a \SI{20}{}--\SI{25}{\percent} collective-latency reduction at
|
||||
scale, and giving the IPCQ a fast on-PE staging buffer (TCM) rather than
|
||||
forcing it through HBM or SRAM is worth another \SI{14}{}--%
|
||||
\SI{38}{\percent} at large payloads.
|
||||
Reference in New Issue
Block a user