paper(platform): Table 1 -> table*, shrink Fig 1 row 1, move Fig 1/2 source up
- Probe table (Table 1, tab:probe-pe-dma) promoted to table* so it
spans both columns and no longer gets clipped at the column edge.
- Fig 1 (hw-arch) row 1 (SIP, CUBE subfigures) trimmed: widths
0.495 -> 0.42 textwidth and a height=0.6\linewidth cap added so
the figure occupies less vertical space and floats can land
earlier in the document.
- Fig 1 and Fig 2 source blocks relocated to the top of §2 so they
enter LaTeX's float queue before any §2 body text. As a result
Fig 1 now lands on page 3 and Fig 2 on page 4 (was 4 and 5).
- Table 2 (tab:hw, modeled hardware config) reverted to plain
\begin{table}; the previous \begin{table*} reshape was not
requested.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Binary file not shown.
@@ -1,3 +1,62 @@
|
|||||||
|
\begin{figure*}[t]
|
||||||
|
\centering
|
||||||
|
\begin{subfigure}[b]{0.42\textwidth}
|
||||||
|
\centering
|
||||||
|
\includegraphics[width=\linewidth,height=0.6\linewidth,keepaspectratio]{sip_architecture.pdf}
|
||||||
|
\caption{SIP level: $4{\times}4$ CUBE mesh + IO chiplet.}
|
||||||
|
\label{fig:sip-arch}
|
||||||
|
\end{subfigure}\hfill
|
||||||
|
\begin{subfigure}[b]{0.42\textwidth}
|
||||||
|
\centering
|
||||||
|
\includegraphics[width=\linewidth,height=0.6\linewidth,keepaspectratio]{cube_architecture.pdf}
|
||||||
|
\caption{CUBE level (zoom-in of one CUBE in (a)): 8 PEs, HBM
|
||||||
|
channels, M\_CPU, SRAM, NoC router mesh, UCIe links.}
|
||||||
|
\label{fig:cube-arch}
|
||||||
|
\end{subfigure}
|
||||||
|
|
||||||
|
\vspace{0.6em}
|
||||||
|
|
||||||
|
\begin{subfigure}[t]{\textwidth}
|
||||||
|
\centering
|
||||||
|
\includegraphics[width=\linewidth]{pe_architecture.png}
|
||||||
|
\caption{PE level (zoom-in of one PE in (b)): \textsf{PE\_CPU}
|
||||||
|
dispatches commands to \textsf{PE\_SCHED}, which routes tile-token
|
||||||
|
streams through \textsf{PE\_DMA}, \textsf{PE\_FETCH\_STORE}, and
|
||||||
|
\textsf{GEMM}/\textsf{MATH} engines; \textsf{PE\_IPCQ} is the
|
||||||
|
on-device collective control plane.}
|
||||||
|
\label{fig:pe-arch}
|
||||||
|
\end{subfigure}
|
||||||
|
\caption{Modeled hardware graph at the SIP, CUBE, and PE levels
|
||||||
|
(one example configuration; specific parameters in \S\ref{sec:hw} /
|
||||||
|
Table~\ref{tab:hw}). KernBench is not tied to this particular
|
||||||
|
arrangement: each box is a modeled component node, each line a
|
||||||
|
directed link with bandwidth and propagation attributes, and any
|
||||||
|
topology that respects those attributes is supported.}
|
||||||
|
\label{fig:hw-arch}
|
||||||
|
\end{figure*}
|
||||||
|
|
||||||
|
\begin{figure*}[t]
|
||||||
|
\centering
|
||||||
|
\includegraphics[width=\textwidth]{latency_model.png}
|
||||||
|
\caption{Conceptual schematic of the latency model. Two source nodes
|
||||||
|
(Requester A/B) inject flits through a chain of routers into a
|
||||||
|
destination node; on the shared edge between routers, flits from the
|
||||||
|
two transactions are interleaved flit-by-flit by the wire's FIFO
|
||||||
|
arrival order. End-to-end latency is the sum of four contributions:
|
||||||
|
\textbf{per-node overhead} (the switch's fixed processing cost,
|
||||||
|
shown in light yellow---the same colour as the destination's
|
||||||
|
processing-logic block), \textbf{per-edge transmission}
|
||||||
|
($\textit{flit\_size}/\textit{BW}$ on each wire),
|
||||||
|
\textbf{drain} (per-flit service occupancy at the destination's
|
||||||
|
channel), and \textbf{queuing delay} (waiting in a FIFO when a
|
||||||
|
shared resource is busy). The places where queuing actually
|
||||||
|
accumulates are highlighted in green: the router output queue and
|
||||||
|
the destination's input queue. This is the model, not a
|
||||||
|
measurement; specific bandwidths and overheads are listed in
|
||||||
|
\S\ref{sec:hw} (Table~\ref{tab:hw}).}
|
||||||
|
\label{fig:latency-model}
|
||||||
|
\end{figure*}
|
||||||
|
|
||||||
\section{The KernBench Platform}
|
\section{The KernBench Platform}
|
||||||
\label{sec:platform}
|
\label{sec:platform}
|
||||||
|
|
||||||
@@ -71,43 +130,6 @@ composite form is the substrate for the GEMM optimization
|
|||||||
(\S\ref{sec:gemm}) and, combined with on-PE collectives, for fused
|
(\S\ref{sec:gemm}) and, combined with on-PE collectives, for fused
|
||||||
attention (\S\ref{sec:gqa}).
|
attention (\S\ref{sec:gqa}).
|
||||||
|
|
||||||
\begin{figure*}[t]
|
|
||||||
\centering
|
|
||||||
\begin{subfigure}[b]{0.495\textwidth}
|
|
||||||
\centering
|
|
||||||
\includegraphics[width=\linewidth]{sip_architecture.pdf}
|
|
||||||
\caption{SIP level: $4{\times}4$ CUBE mesh + IO chiplet.}
|
|
||||||
\label{fig:sip-arch}
|
|
||||||
\end{subfigure}\hfill
|
|
||||||
\begin{subfigure}[b]{0.495\textwidth}
|
|
||||||
\centering
|
|
||||||
\includegraphics[width=\linewidth]{cube_architecture.pdf}
|
|
||||||
\caption{CUBE level (zoom-in of one CUBE in (a)): 8 PEs, HBM
|
|
||||||
channels, M\_CPU, SRAM, NoC router mesh, UCIe links.}
|
|
||||||
\label{fig:cube-arch}
|
|
||||||
\end{subfigure}
|
|
||||||
|
|
||||||
\vspace{0.6em}
|
|
||||||
|
|
||||||
\begin{subfigure}[t]{\textwidth}
|
|
||||||
\centering
|
|
||||||
\includegraphics[width=\linewidth]{pe_architecture.png}
|
|
||||||
\caption{PE level (zoom-in of one PE in (b)): \textsf{PE\_CPU}
|
|
||||||
dispatches commands to \textsf{PE\_SCHED}, which routes tile-token
|
|
||||||
streams through \textsf{PE\_DMA}, \textsf{PE\_FETCH\_STORE}, and
|
|
||||||
\textsf{GEMM}/\textsf{MATH} engines; \textsf{PE\_IPCQ} is the
|
|
||||||
on-device collective control plane.}
|
|
||||||
\label{fig:pe-arch}
|
|
||||||
\end{subfigure}
|
|
||||||
\caption{Modeled hardware graph at the SIP, CUBE, and PE levels
|
|
||||||
(one example configuration; specific parameters in \S\ref{sec:hw} /
|
|
||||||
Table~\ref{tab:hw}). KernBench is not tied to this particular
|
|
||||||
arrangement: each box is a modeled component node, each line a
|
|
||||||
directed link with bandwidth and propagation attributes, and any
|
|
||||||
topology that respects those attributes is supported.}
|
|
||||||
\label{fig:hw-arch}
|
|
||||||
\end{figure*}
|
|
||||||
|
|
||||||
KernBench is layered along the flow of a request:
|
KernBench is layered along the flow of a request:
|
||||||
|
|
||||||
\begin{itemize}
|
\begin{itemize}
|
||||||
@@ -143,28 +165,6 @@ wire, \textbf{drain} (per-flit service occupancy) at the destination,
|
|||||||
and \textbf{queuing delay} at the shared FIFOs that the wire and the
|
and \textbf{queuing delay} at the shared FIFOs that the wire and the
|
||||||
destination share between concurrent transactions.
|
destination share between concurrent transactions.
|
||||||
|
|
||||||
\begin{figure*}[t]
|
|
||||||
\centering
|
|
||||||
\includegraphics[width=\textwidth]{latency_model.png}
|
|
||||||
\caption{Conceptual schematic of the latency model. Two source nodes
|
|
||||||
(Requester A/B) inject flits through a chain of routers into a
|
|
||||||
destination node; on the shared edge between routers, flits from the
|
|
||||||
two transactions are interleaved flit-by-flit by the wire's FIFO
|
|
||||||
arrival order. End-to-end latency is the sum of four contributions:
|
|
||||||
\textbf{per-node overhead} (the switch's fixed processing cost,
|
|
||||||
shown in light yellow---the same colour as the destination's
|
|
||||||
processing-logic block), \textbf{per-edge transmission}
|
|
||||||
($\textit{flit\_size}/\textit{BW}$ on each wire),
|
|
||||||
\textbf{drain} (per-flit service occupancy at the destination's
|
|
||||||
channel), and \textbf{queuing delay} (waiting in a FIFO when a
|
|
||||||
shared resource is busy). The places where queuing actually
|
|
||||||
accumulates are highlighted in green: the router output queue and
|
|
||||||
the destination's input queue. This is the model, not a
|
|
||||||
measurement; specific bandwidths and overheads are listed in
|
|
||||||
\S\ref{sec:hw} (Table~\ref{tab:hw}).}
|
|
||||||
\label{fig:latency-model}
|
|
||||||
\end{figure*}
|
|
||||||
|
|
||||||
\paragraph{The hardware as a graph.} The topology is compiled once at
|
\paragraph{The hardware as a graph.} The topology is compiled once at
|
||||||
configuration time into this graph and is never mutated during a run.
|
configuration time into this graph and is never mutated during a run.
|
||||||
There are no hidden shortcuts, implicit bypasses, or magic paths: if a
|
There are no hidden shortcuts, implicit bypasses, or magic paths: if a
|
||||||
@@ -308,7 +308,7 @@ self-consistent---a model error in any of them would surface as a
|
|||||||
non-monotonic or under-utilising curve here long before it polluted a
|
non-monotonic or under-utilising curve here long before it polluted a
|
||||||
kernel-level measurement.
|
kernel-level measurement.
|
||||||
|
|
||||||
\begin{table}[t]
|
\begin{table*}[t]
|
||||||
\centering
|
\centering
|
||||||
\caption{PE\,$\to$\,HBM DMA latency probe at varying hop distances
|
\caption{PE\,$\to$\,HBM DMA latency probe at varying hop distances
|
||||||
(\SI{32}{\kibi\byte} transfer; output captured from \texttt{kernbench
|
(\SI{32}{\kibi\byte} transfer; output captured from \texttt{kernbench
|
||||||
@@ -327,7 +327,7 @@ PE\,$\to$\,cross-CUBE (best) & 330.5 & 77.5 & ~99.6 \\
|
|||||||
PE\,$\to$\,cross-CUBE (worst) & 677.1 & 37.8 & ~97.8 \\
|
PE\,$\to$\,cross-CUBE (worst) & 677.1 & 37.8 & ~97.8 \\
|
||||||
\bottomrule
|
\bottomrule
|
||||||
\end{tabular}
|
\end{tabular}
|
||||||
\end{table}
|
\end{table*}
|
||||||
|
|
||||||
\subsection{Modeled hardware configuration}
|
\subsection{Modeled hardware configuration}
|
||||||
\label{sec:hw}
|
\label{sec:hw}
|
||||||
@@ -369,33 +369,44 @@ configuration. Workload-specific parameters such as
|
|||||||
matrix dimensions, sequence lengths, and collective
|
matrix dimensions, sequence lengths, and collective
|
||||||
payload sizes are introduced in their respective sections.
|
payload sizes are introduced in their respective sections.
|
||||||
|
|
||||||
\begin{table*}[t]
|
\begin{table}[t]
|
||||||
\centering
|
\centering
|
||||||
\caption{Modeled hardware configuration (shared by all experiments).}
|
\caption{Modeled hardware configuration (shared by all experiments).}
|
||||||
\label{tab:hw}
|
\label{tab:hw}
|
||||||
\small
|
\small
|
||||||
\begin{tabular}{@{}ll@{\hspace{3em}}ll@{}}
|
\begin{tabular}{@{}ll@{}}
|
||||||
\toprule
|
\toprule
|
||||||
\textbf{Parameter} & \textbf{Value} & \textbf{Parameter} & \textbf{Value} \\
|
\textbf{Parameter} & \textbf{Value} \\
|
||||||
\midrule
|
\midrule
|
||||||
\multicolumn{2}{@{}l}{\emph{Hierarchy}} & \multicolumn{2}{@{}l}{\emph{Memory (per CUBE)}} \\
|
\multicolumn{2}{@{}l}{\emph{Hierarchy}} \\
|
||||||
SIPs & 2 (1D ring) & HBM capacity & \SI{48}{\giga\byte} (8 slices) \\
|
SIPs & 2 (1D ring) \\
|
||||||
CUBEs per SIP & 16 ($4\times4$ mesh) & HBM aggregate BW & \SI{1024}{\giga\byte\per\second} \\
|
CUBEs per SIP & 16 ($4\times4$ mesh) \\
|
||||||
PEs per CUBE & 8 (4 corners $\times$ 2) & HBM pseudo-channels & 64 (8 per PE), \SI{32}{\giga\byte\per\second} each \\
|
PEs per CUBE & 8 (4 corners $\times$ 2) \\
|
||||||
PEs total & 256 & SRAM (shared) & \SI{32}{\mega\byte}, \SI{128}{\giga\byte\per\second} link \\
|
PEs total & 256 \\
|
||||||
& & HBM burst & \SI{256}{\byte} \\
|
|
||||||
\midrule
|
\midrule
|
||||||
\multicolumn{2}{@{}l}{\emph{Processing element (PE)}} & \multicolumn{2}{@{}l}{\emph{Interconnect}} \\
|
\multicolumn{2}{@{}l}{\emph{Processing element (PE)}} \\
|
||||||
GEMM engine peak & \SI{8}{\tera\flop\per\second} (f16) & Intra-CUBE NoC link & \SI{256}{\giga\byte\per\second}, \SI{2}{\nano\second}/router \\
|
GEMM engine peak & \SI{8}{\tera\flop\per\second} (f16) \\
|
||||||
TCM (on-PE) & \SI{16}{\mega\byte}, \SI{512}{\giga\byte\per\second} R/W & Inter-CUBE (UCIe PHY) & \SI{512}{\giga\byte\per\second}, \SI{8}{\nano\second}, XY routing \\
|
TCM (on-PE) & \SI{16}{\mega\byte}, \SI{512}{\giga\byte\per\second} R/W \\
|
||||||
\quad kernel scratch & \SI{1}{\mega\byte} & Inter-SIP (PCIe) & \SI{768}{\giga\byte\per\second} per endpoint \\
|
\quad kernel scratch & \SI{1}{\mega\byte} \\
|
||||||
DMA engines & 1 read + 1 write & & \\
|
DMA engines & 1 read + 1 write \\
|
||||||
CPU / scheduler overhead & \SI{2}{\nano\second} / \SI{1}{\nano\second} & & \\
|
CPU / scheduler overhead & \SI{2}{\nano\second} / \SI{1}{\nano\second} \\
|
||||||
\midrule
|
\midrule
|
||||||
\multicolumn{2}{@{}l}{\emph{Command-issue cost model (defaults)}} & & \\
|
\multicolumn{2}{@{}l}{\emph{Memory (per CUBE)}} \\
|
||||||
FIXED per command & 40 cycles & & \\
|
HBM capacity & \SI{48}{\giga\byte} (8 slices) \\
|
||||||
per-byte rate $R$ & 0.0625 cycles/byte (\SI{16}{\byte\per\cycle}) & & \\
|
HBM aggregate BW & \SI{1024}{\giga\byte\per\second} \\
|
||||||
composite size cap & \SI{1024}{\byte} & & \\
|
HBM pseudo-channels & 64 (8 per PE), \SI{32}{\giga\byte\per\second} each \\
|
||||||
|
SRAM (shared) & \SI{32}{\mega\byte}, \SI{128}{\giga\byte\per\second} link \\
|
||||||
|
HBM burst & \SI{256}{\byte} \\
|
||||||
|
\midrule
|
||||||
|
\multicolumn{2}{@{}l}{\emph{Interconnect}} \\
|
||||||
|
Intra-CUBE NoC link & \SI{256}{\giga\byte\per\second}, \SI{2}{\nano\second}/router \\
|
||||||
|
Inter-CUBE (UCIe PHY) & \SI{512}{\giga\byte\per\second}, \SI{8}{\nano\second}, XY routing \\
|
||||||
|
Inter-SIP (PCIe) & \SI{768}{\giga\byte\per\second} per endpoint \\
|
||||||
|
\midrule
|
||||||
|
\multicolumn{2}{@{}l}{\emph{Command-issue cost model (defaults)}} \\
|
||||||
|
FIXED per command & 40 cycles \\
|
||||||
|
per-byte rate $R$ & 0.0625 cycles/byte (\SI{16}{\byte\per\cycle}) \\
|
||||||
|
composite size cap & \SI{1024}{\byte} \\
|
||||||
\bottomrule
|
\bottomrule
|
||||||
\end{tabular}
|
\end{tabular}
|
||||||
\end{table*}
|
\end{table}
|
||||||
|
|||||||
Reference in New Issue
Block a user