paper(1H): 6-case GQA long-ctx + IPCQ design alternatives + GEMM terminology
- §6 GQA: rewrite long-context decode from 4-case to 6-case. Data Placement Policy now presents the six placement options and their intrinsic per-PE memory / per-token comm costs (no winner predicted); the long-context subsection selects the best placement for that regime (Case 6 ★, both-axes S_kv shard) from the measured 6-case sweep. Add KV-sharding diagram + analytical budget/summary figures. - Regenerate the 6-row decode sweep (milestone-1h-gqa) and the sweep-dependent decode panels (latency/traffic/parallelism/memory) so figures, prose, and sweep_decode.json are mutually consistent. - §5 All-Reduce: add IPCQ design alternatives (architecture + decision matrix) as design-rationale schematics (illustrative step-counts, not measured) and the bench-generated topology diagram. - §4 GEMM: rename "user-orchestrated/user-level" -> "kernel-orchestrated/ kernel-level" (orchestration runs in the kernel program vs the scheduler-orchestrated composite); minor accuracy fixes (7.18 TFLOP/s ~10% below peak; ~781 ns DMA). - Recompile build/main.pdf. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -127,6 +127,63 @@ across whatever inter-device topology the configuration specifies,
|
||||
with the IPCQ ring buffer placed in on-PE TCM, PE-local HBM, or
|
||||
cube-shared SRAM---the third knob the results section sweeps.
|
||||
|
||||
\subsection{Design alternatives}
|
||||
\label{sec:ipcq-alternatives}
|
||||
|
||||
PE\_IPCQ is one point in a small space of hardware mechanisms for
|
||||
moving a short message from one PE to a neighbor and signalling its
|
||||
arrival. Three established alternatives anchor the space, each the
|
||||
HW realization of a familiar host-networking idea: a \emph{doorbell +
|
||||
polling} scheme (the classic MMIO doorbell---write the payload by DMA,
|
||||
write a doorbell, let the peer poll or take an interrupt); a
|
||||
\emph{hardware message queue} (HMQ, the NVLink-style descriptor engine
|
||||
that pushes a queue entry to the peer, with large payloads still
|
||||
riding a second DMA); and a \emph{completion-queue} design (RDMA-CQ,
|
||||
the InfiniBand/RoCE pattern where a DMA write auto-posts a completion
|
||||
entry the peer's CQ polls). PE\_IPCQ is the fourth: a hardware ring
|
||||
with credit return, splitting the control plane into PE\_IPCQ and the
|
||||
data plane into PE\_DMA, with head updates riding the payload and tail
|
||||
updates riding a 16\,B side-channel credit (\S\ref{sec:allreduce}).
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=\linewidth]{ipcq_alternatives_architecture_stacked.png}
|
||||
\caption{Per-send data and control flow for the four PE-to-PE
|
||||
signalling mechanisms (sender\,$\rightarrow$\,NoC\,$\rightarrow$\,receiver).
|
||||
Doorbell and RDMA-CQ each issue two fabric transactions (payload then
|
||||
doorbell / completion) and leave the peer polling or taking an
|
||||
interrupt; HMQ adds a dedicated descriptor engine but still moves large
|
||||
payloads on a second DMA; PE\_IPCQ folds head-pointer signalling into
|
||||
the payload flit train and returns the tail credit on a side channel,
|
||||
so a send is one MMIO write and a receive is a flip-flop read. This is
|
||||
a \emph{design schematic}, not a measured comparison.}
|
||||
\label{fig:ipcq-arch}
|
||||
\end{figure}
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=\linewidth]{ipcq_alternatives_decision_matrix.png}
|
||||
\caption{Why the ring+credit design was chosen, across five criteria:
|
||||
single-send latency, whether the host CPU sits on the critical path,
|
||||
whether the receiver must poll or take a wake-up interrupt, whether the
|
||||
control and data datapaths are duplicated, and whether the mechanism is
|
||||
right-sized for single-owner PE-to-PE traffic (rather than a
|
||||
multi-tenant fabric). PE\_IPCQ is the only design that clears every
|
||||
criterion. The accompanying per-send step-count tally
|
||||
($\sim$28 control events for IPCQ versus $\sim$38 for HMQ, $\sim$53 for
|
||||
RDMA-CQ, and $\sim$56 for doorbell+polling) is an \emph{illustrative}
|
||||
order-of-magnitude comparator over hand-counted pipeline steps---not a
|
||||
simulator measurement. The measured, simulator-grounded results follow
|
||||
in the next subsection.}
|
||||
\label{fig:ipcq-decision}
|
||||
\end{figure}
|
||||
|
||||
The qualitative comparison motivates the design but is not a
|
||||
quantitative claim: the cycle-step tallies above are hand-counted
|
||||
control events, deliberately separated from the measured latencies that
|
||||
follow. Everything in the results subsection runs on the PE\_IPCQ
|
||||
substrate and is simulator-grounded.
|
||||
|
||||
\subsection{Results}
|
||||
|
||||
All measurements in this section run on the PE\_IPCQ substrate
|
||||
@@ -138,8 +195,21 @@ the collective sweep builds its own six-device (six-SIP, $2\times3$)
|
||||
configurations---distinct from the two-SIP default of
|
||||
Table~\ref{tab:hw}---and measures all-reduce latency as a function of
|
||||
payload size for three inter-device topologies: a 1D ring, a 2D mesh
|
||||
(no wrap), and a 2D torus. Table~\ref{tab:allreduce} and
|
||||
Figure~\ref{fig:allreduce-cmp} report the result.
|
||||
(no wrap), and a 2D torus (Figure~\ref{fig:allreduce-topo}).
|
||||
Table~\ref{tab:allreduce} and Figure~\ref{fig:allreduce-cmp} report the
|
||||
result.
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=\linewidth]{allreduce_topology.png}
|
||||
\caption{The three six-device ($2\times3$) inter-device topologies the
|
||||
collective sweep runs over, and the hierarchical local-reduce /
|
||||
global all-reduce-broadcast schedule mapped onto each: a 1D ring, a 2D
|
||||
mesh (no wrap-around), and a 2D torus (wrap-around links on both axes).
|
||||
The torus's wrap links shorten the worst-case reduction path, which is
|
||||
what the latency sweep below rewards.}
|
||||
\label{fig:allreduce-topo}
|
||||
\end{figure}
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
|
||||
Reference in New Issue
Block a user