paper: §2 polish + §5 fresh all-reduce data + kernbench probe table

§2 platform:
- Add a "memory-centric AHBM" sentence to the device-and-execution
  intro: each PE is paired with dedicated HBM bandwidth and on-PE
  TCM, so the performance question is about feeding compute from
  locally-attached memory + moving the unavoidable inter-PE traffic
  efficiently. Makes the AHBM character of the platform visible
  before §2.3 starts unpacking the latency model.
- Promote "Congestion and contention modeling" from \paragraph to
  \subsubsection: this is the platform's core differentiator over a
  peak-BW roofline, so it deserves its own heading.
- Rename "Control-plane (issue) cost model" to "Command dispatch
  overhead model" -- describes what it actually charges (the PE_CPU
  paying a fixed + per-byte cost to push a command to one of the
  accelerator engines) without the more abstract "control-plane"
  framing.
- Add a third Accuracy cross-check from kernbench probe: a
  PE→HBM DMA-distance sweep that confirms monotonic hop progression,
  bandwidth saturation matching the per-edge model, and the built-in
  invariants (D2H >= H2D, cross-CUBE best < worst). New
  Table~\ref{tab:probe-pe-dma} reports per-traversal latency and
  utilisation at 32KiB / 1MiB across five hop classes.
- Captured probe output as figures/probe_pe_dma_summary.txt for
  reproducibility.

§5 PE_IPCQ / all-reduce:
- Re-ran milestone-1h-ccl on current sim_engine (post-ADR-0064 Rev2
  and the IPCQ slot-wrap Phase-2 race fix). Updated the topology-
  comparison table and the buffer-kind caption to the fresh raw
  latencies. Headline ratios are preserved:
    torus vs mesh saving at 96 KB/PE: 24.8% (was 24.8%) -> ~25%
    torus vs ring saving at 96 KB/PE: 19.3% (was 19.3%) -> ~19%
    TCM vs HBM saving at 64 KB/PE: 13.4% (was 13.9%) -> ~13%
    TCM vs SRAM saving at 64 KB/PE: 37.2% (was 38.3%) -> ~37%
  The Executive Summary's "up to ~14%" / "up to ~38%" framing stays
  consistent with these post values.

Artifacts refreshed in src/kernbench/benches/1H_milestone_output/:
- ccl/summary.csv + per-topology PNGs + buffer-kind CSV/PNG
- ccl/comparison_mesh_vs_ring_vs_2DTorus_vs_theoretical_vs_fsim.png
- gemm/* (re-run yields identical pe_window structure; PNGs
  refreshed)

Paper figures synced to the fresh artifacts:
- figures/allreduce_comparison.png
- figures/allreduce_buffer_kind.png

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-16 10:38:47 -07:00
parent 7c346dec1b
commit e3f08972da
18 changed files with 241 additions and 93 deletions
@@ -8,7 +8,7 @@ execution model it presents to a kernel writer, how its latency model
turns that execution into a number, and the concrete hardware
configuration used for every experiment that follows.
\subsection{Why KernBench: source-level kernels without a software stack}
\subsection{Why KernBench}
\label{sec:why}
In a production end-to-end (E2E) stack, kernel performance is entangled
@@ -55,7 +55,15 @@ physical distance and bandwidth.}
Each CUBE contains eight PEs, shared SRAM, HBM controllers, an
\textsf{M\_CPU} control processor, and an intra-CUBE router mesh
(Fig.~\ref{fig:cube-arch}). Together these components form the
execution substrate for all kernels evaluated in this report.
execution substrate for all kernels evaluated in this report. This
organization reflects the memory-centric nature of AHBM: each PE is
paired with a dedicated slice of HBM bandwidth and local on-PE
storage (TCM), so compute lives next to the data it consumes rather
than fetching it through a far-away memory controller. The platform's
performance question is therefore not ``how many FLOPs can the chip
do'' but ``how well can a kernel keep each compute engine fed from
its locally-attached memory while moving the unavoidable traffic
between PEs efficiently.''
\begin{figure}[t]
\centering
@@ -195,7 +203,10 @@ collective engines hold the request for their service time before
releasing it downstream. Each of these is attached to a specific node
or edge in the graph; together they make up the entire latency budget.
\paragraph{Congestion: where bottlenecks emerge.} The simulator's
\subsubsection{Congestion and contention modeling}
\label{sec:congestion}
The simulator's
sharpness comes from how it models contention for those nodes and
edges. \emph{Every directed edge has a FIFO}: an arriving flit takes
its bandwidth-limited transfer time on top of whatever earlier flits
@@ -215,9 +226,11 @@ it reveals where the real bottlenecks form and which hardware levers
actually relieve them---which is exactly the question the codesign
work in this report turns on.
\paragraph{Control-plane (issue) cost model.} The cost of \emph{issuing}
a command is modelled structurally rather than with a per-operation
calibration table. The PE control processor charges, per command,
\paragraph{Command dispatch overhead model.} The cost incurred by
the \textsf{PE\_CPU} when it dispatches a command to one of the
accelerator engines is modelled structurally rather than with a
per-operation calibration table. The \textsf{PE\_CPU} charges, per
command,
\[
d_{\text{cmd}} = \textsf{FIXED} + b_{\text{logical}} \cdot R,
\]
@@ -267,14 +280,92 @@ hardware--software design trade-offs (tiling A vs.\ B, topology X
vs.\ Y, with vs.\ without composite command, mesh vs.\ torus) that
are the primary objective of this work.
A third source of confidence comes from directly probing the
simulator's per-traversal behaviour. Running \texttt{kernbench probe}
on the modelled topology issues a sequence of PE-to-HBM DMA reads at
progressively greater hop distances and reports the per-component
overhead, per-edge serialization, and per-PC drain that the model
charges (Table~\ref{tab:probe-pe-dma}). Three properties stand out.
First, the reported latency increases \emph{monotonically} with hop
count---from \SI{141}{\nano\second} at the local HBM slice to
\SI{677}{\nano\second} at the worst-case remote-CUBE slice---with the
increment per added hop matching the per-router overhead and the
UCIe-link cost in Table~\ref{tab:hw}. Second, the bandwidth-saturation
curves track the wire's serialization model: at \SI{1}{\mebi\byte}
transfers the local PE DMA reaches \SI{100}{\percent} of the per-edge
bandwidth limit, while the longest cross-CUBE path saturates at
\SI{97.8}{\percent}---exactly the gap that the per-edge propagation
and per-router overhead model would predict. Third, the simulator
passes the internal-consistency invariants the probe builds in
(monotonic hop progression; D2H reads at least as long as the
equivalent H2D writes because of the reverse-path acknowledgement;
cross-CUBE best less than cross-CUBE worst). These are properties
that hold \emph{only} when the underlying graph traversal, FIFO
contention, and propagation models are internally
self-consistent---a model error in any of them would surface as a
non-monotonic or under-utilising curve here long before it polluted a
kernel-level measurement.
\begin{table}[t]
\centering
\caption{PE\,$\to$\,HBM DMA latency probe at varying hop distances
(\SI{32}{\kibi\byte} transfer; output captured from \texttt{kernbench
probe}). \emph{Util\%} is the achieved bandwidth as a fraction of the
path's bottleneck-edge bandwidth.}
\label{tab:probe-pe-dma}
\small
\begin{tabular}{@{}lrrr@{}}
\toprule
Case & Latency~(\si{\nano\second}) & Util\% (\,32\,KiB) & Util\% (\,1\,MiB) \\
\midrule
PE\,$\to$\,local HBM & 141.0 & 90.8 & 100.0 \\
PE\,$\to$\,same-half HBM & 147.9 & 86.6 & ~99.9 \\
PE\,$\to$\,cross-half HBM & 161.2 & 79.4 & ~99.8 \\
PE\,$\to$\,cross-CUBE (best) & 330.5 & 77.5 & ~99.6 \\
PE\,$\to$\,cross-CUBE (worst) & 677.1 & 37.8 & ~97.8 \\
\bottomrule
\end{tabular}
\end{table}
\subsection{Modeled hardware configuration}
\label{sec:hw}
Table~\ref{tab:hw} summarizes the hardware configuration used for
every experiment in this report. It is read directly from the
simulator's topology description; per-experiment workload parameters
(matrix shapes, collective sizes, sequence lengths) are stated in
their respective sections rather than here.
Table~\ref{tab:hw} summarizes the hardware configuration
used throughout this report. The intent of this
configuration is not to model a specific product, but to
represent a realistic memory-centric accelerator and to
provide a consistent baseline for evaluating hardware--
software co-design mechanisms.
The compute capability within each CUBE is provisioned
such that inference workloads can effectively saturate
the available HBM bandwidth. The aggregate compute
throughput is therefore balanced against memory-system
bandwidth rather than being intentionally over- or
under-provisioned. HBM bandwidth is distributed evenly
across the PEs within a CUBE, with each PE responsible
for servicing approximately one-eighth of the CUBE's
memory bandwidth through its dedicated HBM channels.
The on-chip interconnect is configured using bandwidth
and latency parameters representative of commercially
available mesh-network IPs. The goal is not to study a
particular NoC implementation, but rather to evaluate
kernel behavior under a realistic baseline communication
fabric.
For die-to-die communication, UCIe-A bandwidth
characteristics are used as the reference point for
inter-CUBE links. Inter-SIP communication is modeled
using PCIe-class links. Together, these assumptions
provide a representative communication hierarchy for
evaluating collective communication and distributed
kernel execution.
Unless otherwise stated, all experiments use this
configuration. Workload-specific parameters such as
matrix dimensions, sequence lengths, and collective
payload sizes are introduced in their respective sections.
\begin{table}[t]
\centering
@@ -52,11 +52,11 @@ per-PE payload. Lower is better; the torus wins at every size.}
\toprule
\textbf{Bytes/PE} & \textbf{2D mesh} & \textbf{Ring 1D} & \textbf{2D torus} \\
\midrule
256 & 2667 & 2365 & 1701 \\
4{,}096 & 4450 & 4082 & 3038 \\
16{,}384 & 8900 & 8217 & 6403 \\
65{,}536 & 26705 & 24766 & 19865 \\
98{,}304 & 38574 & 35798 & 28840 \\
256 & 4189 & 3883 & 2957 \\
4{,}096 & 5566 & 5240 & 4031 \\
16{,}384 & 10016 & 9376 & 7396 \\
65{,}536 & 27821 & 25925 & 20858 \\
98{,}304 & 39690 & 36957 & 29833 \\
\bottomrule
\end{tabular}
\end{table}
@@ -78,9 +78,9 @@ achievable-kernel number and a full end-to-end-stack number.}
\centering
\includegraphics[width=\linewidth]{allreduce_buffer_kind.png}
\caption{Effect of IPCQ staging-buffer placement (2D torus). At
\SI{64}{\kibi\byte}/PE, TCM staging (\SI{19865}{\nano\second}) beats HBM
(\SI{23081}{\nano\second}) by \textasciitilde\SI{14}{\percent} and SRAM
(\SI{32201}{\nano\second}) by \textasciitilde\SI{38}{\percent}; at small
\SI{64}{\kibi\byte}/PE, TCM staging (\SI{20858}{\nano\second}) beats HBM
(\SI{24074}{\nano\second}) by \textasciitilde\SI{13}{\percent} and SRAM
(\SI{33194}{\nano\second}) by \textasciitilde\SI{37}{\percent}; at small
payloads the three are indistinguishable.}
\label{fig:allreduce-buf}
\end{figure}