paper(1H): 6-case GQA long-ctx + IPCQ design alternatives + GEMM terminology

- §6 GQA: rewrite long-context decode from 4-case to 6-case. Data
  Placement Policy now presents the six placement options and their
  intrinsic per-PE memory / per-token comm costs (no winner predicted);
  the long-context subsection selects the best placement for that regime
  (Case 6 ★, both-axes S_kv shard) from the measured 6-case sweep. Add
  KV-sharding diagram + analytical budget/summary figures.
- Regenerate the 6-row decode sweep (milestone-1h-gqa) and the
  sweep-dependent decode panels (latency/traffic/parallelism/memory) so
  figures, prose, and sweep_decode.json are mutually consistent.
- §5 All-Reduce: add IPCQ design alternatives (architecture + decision
  matrix) as design-rationale schematics (illustrative step-counts, not
  measured) and the bench-generated topology diagram.
- §4 GEMM: rename "user-orchestrated/user-level" -> "kernel-orchestrated/
  kernel-level" (orchestration runs in the kernel program vs the
  scheduler-orchestrated composite); minor accuracy fixes (7.18 TFLOP/s
  ~10% below peak; ~781 ns DMA).
- Recompile build/main.pdf.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-22 00:55:54 -07:00
parent bb668a3ac3
commit e9a5c438e3
20 changed files with 351 additions and 154 deletions
@@ -127,6 +127,63 @@ across whatever inter-device topology the configuration specifies,
with the IPCQ ring buffer placed in on-PE TCM, PE-local HBM, or
cube-shared SRAM---the third knob the results section sweeps.
\subsection{Design alternatives}
\label{sec:ipcq-alternatives}
PE\_IPCQ is one point in a small space of hardware mechanisms for
moving a short message from one PE to a neighbor and signalling its
arrival. Three established alternatives anchor the space, each the
HW realization of a familiar host-networking idea: a \emph{doorbell +
polling} scheme (the classic MMIO doorbell---write the payload by DMA,
write a doorbell, let the peer poll or take an interrupt); a
\emph{hardware message queue} (HMQ, the NVLink-style descriptor engine
that pushes a queue entry to the peer, with large payloads still
riding a second DMA); and a \emph{completion-queue} design (RDMA-CQ,
the InfiniBand/RoCE pattern where a DMA write auto-posts a completion
entry the peer's CQ polls). PE\_IPCQ is the fourth: a hardware ring
with credit return, splitting the control plane into PE\_IPCQ and the
data plane into PE\_DMA, with head updates riding the payload and tail
updates riding a 16\,B side-channel credit (\S\ref{sec:allreduce}).
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{ipcq_alternatives_architecture_stacked.png}
\caption{Per-send data and control flow for the four PE-to-PE
signalling mechanisms (sender\,$\rightarrow$\,NoC\,$\rightarrow$\,receiver).
Doorbell and RDMA-CQ each issue two fabric transactions (payload then
doorbell / completion) and leave the peer polling or taking an
interrupt; HMQ adds a dedicated descriptor engine but still moves large
payloads on a second DMA; PE\_IPCQ folds head-pointer signalling into
the payload flit train and returns the tail credit on a side channel,
so a send is one MMIO write and a receive is a flip-flop read. This is
a \emph{design schematic}, not a measured comparison.}
\label{fig:ipcq-arch}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{ipcq_alternatives_decision_matrix.png}
\caption{Why the ring+credit design was chosen, across five criteria:
single-send latency, whether the host CPU sits on the critical path,
whether the receiver must poll or take a wake-up interrupt, whether the
control and data datapaths are duplicated, and whether the mechanism is
right-sized for single-owner PE-to-PE traffic (rather than a
multi-tenant fabric). PE\_IPCQ is the only design that clears every
criterion. The accompanying per-send step-count tally
($\sim$28 control events for IPCQ versus $\sim$38 for HMQ, $\sim$53 for
RDMA-CQ, and $\sim$56 for doorbell+polling) is an \emph{illustrative}
order-of-magnitude comparator over hand-counted pipeline steps---not a
simulator measurement. The measured, simulator-grounded results follow
in the next subsection.}
\label{fig:ipcq-decision}
\end{figure}
The qualitative comparison motivates the design but is not a
quantitative claim: the cycle-step tallies above are hand-counted
control events, deliberately separated from the measured latencies that
follow. Everything in the results subsection runs on the PE\_IPCQ
substrate and is simulator-grounded.
\subsection{Results}
All measurements in this section run on the PE\_IPCQ substrate
@@ -138,8 +195,21 @@ the collective sweep builds its own six-device (six-SIP, $2\times3$)
configurations---distinct from the two-SIP default of
Table~\ref{tab:hw}---and measures all-reduce latency as a function of
payload size for three inter-device topologies: a 1D ring, a 2D mesh
(no wrap), and a 2D torus. Table~\ref{tab:allreduce} and
Figure~\ref{fig:allreduce-cmp} report the result.
(no wrap), and a 2D torus (Figure~\ref{fig:allreduce-topo}).
Table~\ref{tab:allreduce} and Figure~\ref{fig:allreduce-cmp} report the
result.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{allreduce_topology.png}
\caption{The three six-device ($2\times3$) inter-device topologies the
collective sweep runs over, and the hierarchical local-reduce /
global all-reduce-broadcast schedule mapped onto each: a 1D ring, a 2D
mesh (no wrap-around), and a 2D torus (wrap-around links on both axes).
The torus's wrap links shorten the worst-case reduction path, which is
what the latency sweep below rewards.}
\label{fig:allreduce-topo}
\end{figure}
\begin{table}[t]
\centering