paper: §2 figure layout + Accuracy subsection + Table 1 → 2-col; §6 trim historical 4-panel content

§2 platform:
- Accuracy promoted from \paragraph to \subsection (sec:accuracy);
  it sits at the same heading level as Why / Device / Latency / HW
  config and so reads as a first-class component of the platform
  description rather than a tail-end footnote.
- Three architecture diagrams (SIP, CUBE, PE) collapsed into one
  figure* using subcaption: Fig.~\ref{fig:hw-arch}(a) SIP and (b)
  CUBE side-by-side on row 1, (c) PE wide on row 2, total height
  capped at ~40% of the page. Subfigure cross-references rewritten
  to Fig.~ref{fig:hw-arch}\subref{...} in the body text. main.tex
  now pulls in the subcaption package.
- Table 1 (modeled hardware configuration) promoted to table*
  (two-column / full page width) and the row layout rewritten as a
  4-column tabular so Hierarchy + PE + Command-issue sit on the
  left and Memory + Interconnect sit on the right. The previous
  single-column rendering was getting cut at the right edge of the
  printed column.

§6 GQA:
- Removed the historical four-panel headline table (tab:gqa), the
  two figures (fig:gqa-lat, fig:gqa-break), and the prose paragraph
  that cited their per-panel numbers. Reason: the underlying
  milestone-gqa-headline bench has been simplified in collaborator
  commit 65c365f ("drop misleading single_user_/multi_user_
  panels") and no longer reproduces that data, so the section was
  left referencing a dataset the current bench cannot regenerate.
- The previously-added 4-cases long-context decode comparison is
  now §6's only Results subsection, retitled to
  "Results: long-context decode and parallelism strategies".

The headline GQA result that survives into the paper is therefore
the parallelism trade-off study (Case 4 / Cube-SP × PE-SP at 34 us
buying an 8x KV-memory reduction over the latency leaders).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/report/1H-codesign-paper/main.tex

@@ -13,6 +13,7 @@
 \usepackage{hyperref}
 \hypersetup{colorlinks=true,linkcolor=blue!50!black,citecolor=blue!50!black,urlcolor=blue!50!black}
 \usepackage{caption}
+\usepackage{subcaption}
 \captionsetup{font=small,labelfont=bf}
 \usepackage{microtype}
 \usepackage{tikz}

docs/report/1H-codesign-paper/sections/02-platform.tex

@@ -39,73 +39,73 @@ KernBench models AHBM as a hierarchy of SIPs, CUBEs, and processing
 elements (PEs). At the system level, multiple SIPs are connected
 through inter-package links, while each SIP contains a collection of
 CUBEs joined by an on-package interconnect
-(Fig.~\ref{fig:sip-arch}).
+(Fig.~\ref{fig:hw-arch}\subref{fig:sip-arch}).
-
-\begin{figure}[t]
-\centering
-\includegraphics[width=\linewidth]{sip_architecture.pdf}
-\caption{Modeled hardware graph at the \emph{SIP} level (one
-example configuration; specific parameters in \S\ref{sec:hw} /
-Table~\ref{tab:hw}). The SIP holds a $4{\times}4$ mesh of CUBEs and
-an IO chiplet, with each line a directed link labelled by its
-physical distance and bandwidth.}
-\label{fig:sip-arch}
-\end{figure}
 
 Each CUBE contains eight PEs, shared SRAM, HBM controllers, an
 \textsf{M\_CPU} control processor, and an intra-CUBE router mesh
-(Fig.~\ref{fig:cube-arch}). Together these components form the
+(Fig.~\ref{fig:hw-arch}\subref{fig:cube-arch}). Together these
-execution substrate for all kernels evaluated in this report. This
+components form the execution substrate for all kernels evaluated in
-organization reflects the memory-centric nature of AHBM: each PE is
+this report. This organization reflects the memory-centric nature of
-paired with a dedicated slice of HBM bandwidth and local on-PE
+AHBM: each PE is paired with a dedicated slice of HBM bandwidth and
-storage (TCM), so compute lives next to the data it consumes rather
+local on-PE storage (TCM), so compute lives next to the data it
-than fetching it through a far-away memory controller. The platform's
+consumes rather than fetching it through a far-away memory
-performance question is therefore not ``how many FLOPs can the chip
+controller. The platform's performance question is therefore not
-do'' but ``how well can a kernel keep each compute engine fed from
+``how many FLOPs can the chip do'' but ``how well can a kernel keep
-its locally-attached memory while moving the unavoidable traffic
+each compute engine fed from its locally-attached memory while
-between PEs efficiently.''
+moving the unavoidable traffic between PEs efficiently.''
-
-\begin{figure}[t]
-\centering
-\includegraphics[width=\linewidth]{cube_architecture.pdf}
-\caption{Modeled hardware graph at the \emph{CUBE} level---a
-zoom-in on one CUBE node from Fig.~\ref{fig:sip-arch}. The CUBE
-holds 8 PEs (each with its own HBM channels), the \textsf{M\_CPU}
-control processor, a shared SRAM, an intra-CUBE NoC router mesh, and
-UCIe links to neighbouring CUBEs.}
-\label{fig:cube-arch}
-\end{figure}
 
 Within a PE, commands are dispatched by \textsf{PE\_CPU} to
 \textsf{PE\_SCHED}, which routes work to specialized execution
 engines---DMA, FETCH/STORE, GEMM, vector-math, and IPCQ
-(Fig.~\ref{fig:pe-arch}). Commands come in two flavours. \emph{Atomic}
+(Fig.~\ref{fig:hw-arch}\subref{fig:pe-arch}). Commands come in two
-commands target a single engine---a plain DMA read, a GEMM tile, a
+flavours. \emph{Atomic} commands target a single engine---a plain
-vector-math op, an IPCQ send/receive---and are the natural unit for
+DMA read, a GEMM tile, a vector-math op, an IPCQ send/receive---and
-short or special-purpose work; the \textsf{PE\_CPU} itself also runs
+are the natural unit for short or special-purpose work; the
-control-plane work directly. \emph{Composite} commands, by contrast,
+\textsf{PE\_CPU} itself also runs control-plane work directly.
-carry an ordered pipeline of operations across multiple engines
+\emph{Composite} commands, by contrast, carry an ordered pipeline of
-(\textsf{DMA\_READ} $\rightarrow$ \textsf{FETCH} $\rightarrow$
+operations across multiple engines (\textsf{DMA\_READ} $\rightarrow$
-\textsf{GEMM}/\textsf{MATH} $\rightarrow$ \textsf{STORE} $\rightarrow$
+\textsf{FETCH} $\rightarrow$ \textsf{GEMM}/\textsf{MATH} $\rightarrow$
-\textsf{DMA\_WRITE}) that the \textsf{PE\_SCHED} tiles and streams
+\textsf{STORE} $\rightarrow$ \textsf{DMA\_WRITE}) that the
-without per-tile redispatch. The composite form is the substrate for
+\textsf{PE\_SCHED} tiles and streams without per-tile redispatch. The
-the GEMM optimization (\S\ref{sec:gemm}) and, combined with on-PE
+composite form is the substrate for the GEMM optimization
-collectives, for fused attention (\S\ref{sec:gqa}).
+(\S\ref{sec:gemm}) and, combined with on-PE collectives, for fused
+attention (\S\ref{sec:gqa}).
 
 \begin{figure*}[t]
+\centering
+\begin{subfigure}[t]{0.46\textwidth}
+  \centering
+  \includegraphics[width=\linewidth,height=0.85\linewidth,keepaspectratio]{sip_architecture.pdf}
+  \caption{SIP level: $4{\times}4$ CUBE mesh + IO chiplet.}
+  \label{fig:sip-arch}
+\end{subfigure}\hfill
+\begin{subfigure}[t]{0.46\textwidth}
+  \centering
+  \includegraphics[width=\linewidth,height=0.85\linewidth,keepaspectratio]{cube_architecture.pdf}
+  \caption{CUBE level (zoom-in of one CUBE in (a)): 8 PEs, HBM
+  channels, M\_CPU, SRAM, NoC router mesh, UCIe links.}
+  \label{fig:cube-arch}
+\end{subfigure}
+
+\vspace{0.6em}
+
+\begin{subfigure}[t]{\textwidth}
   \centering
   \includegraphics[width=\linewidth]{pe_architecture.png}
-\caption{Modeled PE architecture used in this report---one example
+  \caption{PE level (zoom-in of one PE in (b)): \textsf{PE\_CPU}
-configuration whose specific parameters are listed in
+  dispatches commands to \textsf{PE\_SCHED}, which routes tile-token
-\S\ref{sec:hw} (Table~\ref{tab:hw}); KernBench is not tied to this
+  streams through \textsf{PE\_DMA}, \textsf{PE\_FETCH\_STORE}, and
-particular decomposition and supports arbitrary PE block layouts
+  \textsf{GEMM}/\textsf{MATH} engines; \textsf{PE\_IPCQ} is the
-provided each component is given a port and bandwidth model.
+  on-device collective control plane.}
-\textsf{PE\_CPU} dispatches commands to the \textsf{PE\_SCHED}, which
-routes tile-token streams through the \textsf{PE\_DMA},
-\textsf{PE\_FETCH\_STORE}, and \textsf{GEMM}/\textsf{MATH} engines
-along on-chip links; the \textsf{PE\_IPCQ} provides the control plane
-for on-device collective communication.}
   \label{fig:pe-arch}
+\end{subfigure}
+\caption{Modeled hardware graph at the SIP, CUBE, and PE levels
+(one example configuration; specific parameters in \S\ref{sec:hw} /
+Table~\ref{tab:hw}). KernBench is not tied to this particular
+arrangement: each box is a modeled component node, each line a
+directed link with bandwidth and propagation attributes, and any
+topology that respects those attributes is supported.}
+\label{fig:hw-arch}
 \end{figure*}
 
 KernBench is layered along the flow of a request:
@@ -171,9 +171,8 @@ There are no hidden shortcuts, implicit bypasses, or magic paths: if a
 request reaches its destination, the path it took is explicit in the
 graph, and the latency it incurred is the sum of the per-node and
 per-edge costs paid along that path. The same graph representation
-applies recursively at every hierarchy level---system, SIP
+applies recursively at every hierarchy level---system, SIP, CUBE, and
-(Fig.~\ref{fig:sip-arch}), CUBE (Fig.~\ref{fig:cube-arch}), and PE
+PE (Fig.~\ref{fig:hw-arch}).
-(Fig.~\ref{fig:pe-arch}).
 
 \paragraph{From graph to discrete-event simulation.} The graph is
 driven by a discrete-event engine. Two kinds of events advance
@@ -247,7 +246,10 @@ queues accept. In the configurations measured here, command issue is
 not the bottleneck---data movement is---so this term stays small
 relative to DMA and collective time.
 
-\paragraph{Accuracy.} The model is precise about the effects that
+\subsection{Accuracy}
+\label{sec:accuracy}
+
+The model is precise about the effects that
 dominate kernel latency on this class of hardware: per-edge bandwidth
 occupancy and flit-level serialization, HBM pseudo-channel parallelism,
 and per-component switching overhead. Two independent cross-checks
@@ -367,44 +369,33 @@ configuration. Workload-specific parameters such as
 matrix dimensions, sequence lengths, and collective
 payload sizes are introduced in their respective sections.
 
-\begin{table}[t]
+\begin{table*}[t]
 \centering
 \caption{Modeled hardware configuration (shared by all experiments).}
 \label{tab:hw}
 \small
-\begin{tabular}{@{}ll@{}}
+\begin{tabular}{@{}ll@{\hspace{3em}}ll@{}}
 \toprule
-\textbf{Parameter} & \textbf{Value} \\
+\textbf{Parameter} & \textbf{Value} & \textbf{Parameter} & \textbf{Value} \\
 \midrule
-\multicolumn{2}{@{}l}{\emph{Hierarchy}} \\
+\multicolumn{2}{@{}l}{\emph{Hierarchy}} & \multicolumn{2}{@{}l}{\emph{Memory (per CUBE)}} \\
-SIPs & 2 (1D ring) \\
+SIPs                       & 2 (1D ring) & HBM capacity            & \SI{48}{\giga\byte} (8 slices) \\
-CUBEs per SIP & 16 ($4\times4$ mesh) \\
+CUBEs per SIP              & 16 ($4\times4$ mesh) & HBM aggregate BW & \SI{1024}{\giga\byte\per\second} \\
-PEs per CUBE & 8 (4 corners $\times$ 2) \\
+PEs per CUBE               & 8 (4 corners $\times$ 2) & HBM pseudo-channels & 64 (8 per PE), \SI{32}{\giga\byte\per\second} each \\
-PEs total & 256 \\
+PEs total                  & 256 & SRAM (shared)             & \SI{32}{\mega\byte}, \SI{128}{\giga\byte\per\second} link \\
+                           &     & HBM burst                 & \SI{256}{\byte} \\
 \midrule
-\multicolumn{2}{@{}l}{\emph{Processing element (PE)}} \\
+\multicolumn{2}{@{}l}{\emph{Processing element (PE)}} & \multicolumn{2}{@{}l}{\emph{Interconnect}} \\
-GEMM engine peak & \SI{8}{\tera\flop\per\second} (f16) \\
+GEMM engine peak           & \SI{8}{\tera\flop\per\second} (f16) & Intra-CUBE NoC link & \SI{256}{\giga\byte\per\second}, \SI{2}{\nano\second}/router \\
-TCM (on-PE) & \SI{16}{\mega\byte}, \SI{512}{\giga\byte\per\second} R/W \\
+TCM (on-PE)                & \SI{16}{\mega\byte}, \SI{512}{\giga\byte\per\second} R/W & Inter-CUBE (UCIe PHY) & \SI{512}{\giga\byte\per\second}, \SI{8}{\nano\second}, XY routing \\
-\quad kernel scratch & \SI{1}{\mega\byte} \\
+\quad kernel scratch       & \SI{1}{\mega\byte} & Inter-SIP (PCIe) & \SI{768}{\giga\byte\per\second} per endpoint \\
-DMA engines & 1 read + 1 write \\
+DMA engines                & 1 read + 1 write & & \\
-CPU / scheduler overhead & \SI{2}{\nano\second} / \SI{1}{\nano\second} \\
+CPU / scheduler overhead   & \SI{2}{\nano\second} / \SI{1}{\nano\second} & & \\
 \midrule
-\multicolumn{2}{@{}l}{\emph{Memory (per CUBE)}} \\
+\multicolumn{2}{@{}l}{\emph{Command-issue cost model (defaults)}} & & \\
-HBM capacity & \SI{48}{\giga\byte} (8 slices) \\
+FIXED per command          & 40 cycles & & \\
-HBM aggregate BW & \SI{1024}{\giga\byte\per\second} \\
+per-byte rate $R$          & 0.0625 cycles/byte (\SI{16}{\byte\per\cycle}) & & \\
-HBM pseudo-channels & 64 (8 per PE), \SI{32}{\giga\byte\per\second} each \\
+composite size cap         & \SI{1024}{\byte} & & \\
-SRAM (shared) & \SI{32}{\mega\byte}, \SI{128}{\giga\byte\per\second} link \\
-HBM burst & \SI{256}{\byte} \\
-\midrule
-\multicolumn{2}{@{}l}{\emph{Interconnect}} \\
-Intra-CUBE NoC link & \SI{256}{\giga\byte\per\second}, \SI{2}{\nano\second}/router \\
-Inter-CUBE (UCIe PHY) & \SI{512}{\giga\byte\per\second}, \SI{8}{\nano\second}, XY routing \\
-Inter-SIP (PCIe) & \SI{768}{\giga\byte\per\second} per endpoint \\
-\midrule
-\multicolumn{2}{@{}l}{\emph{Command-issue cost model (defaults)}} \\
-FIXED per command & 40 cycles \\
-per-byte rate $R$ & 0.0625 cycles/byte (\SI{16}{\byte\per\cycle}) \\
-composite size cap & \SI{1024}{\byte} \\
 \bottomrule
 \end{tabular}
-\end{table}
+\end{table*}

docs/report/1H-codesign-paper/sections/05-gqa.tex

@@ -41,70 +41,7 @@ that restructures the decode step into two stateful composites (a named
 \textsf{softmax\_merge} recipe) is designed but not yet wired into the
 measured path; results below reflect the implemented kernel only.
 
-\subsection{Results}
+\subsection{Results: long-context decode and parallelism strategies}
-
-We measure four headline panels that vary the user count $C$ and the phase:
-single- and multi-user prefill ($T_q=4$, $S_{kv}=16$), and single- and
-multi-user decode ($P=8$ PEs, $S_{kv}=64$ and $128$), all at $d_{\text{head}}=64$
-and $G=8$. For each panel we harvest end-to-end latency
-(max event end minus min event start, the same window convention as the
-GEMM study) together with the per-engine busy time and the operation mix.
-Figure~\ref{fig:gqa-lat} and Figure~\ref{fig:gqa-break} report the result;
-the underlying numbers are in Table~\ref{tab:gqa}.
-
-\begin{table}[t]
-\centering
-\caption{Fused GQA per-panel latency and operation mix. Compute (GEMM,
-MATH) is a tiny fraction of DMA occupancy; IPCQ copies grow with users and
-PEs.}
-\label{tab:gqa}
-\small
-\begin{tabular}{@{}lrrrr@{}}
-\toprule
-\textbf{Panel} & \textbf{Lat.\ (ns)} & \textbf{GEMM} & \textbf{IPCQ} & \textbf{DMA rd} \\
-\midrule
-prefill C=1        & 445   & 2  & 0  & 3  \\
-prefill C=4 (Ring) & 4630  & 32 & 24 & 12 \\
-decode C=1, P=8    & 3632  & 16 & 21 & 24 \\
-decode C=4, P=8    & 6693  & 64 & 93 & 96 \\
-\bottomrule
-\end{tabular}
-\end{table}
-
-\begin{figure}[t]
-\centering
-\includegraphics[width=\linewidth]{gqa_latency_by_panel.png}
-\caption{Fused GQA end-to-end latency. Latency grows from
-\SI{445}{\nano\second} (single-user prefill) to \SI{6693}{\nano\second}
-(four-user decode) as the KV history and the number of participating
-devices grow.}
-\label{fig:gqa-lat}
-\end{figure}
-
-\begin{figure}[t]
-\centering
-\includegraphics[width=\linewidth]{gqa_op_engine_breakdown.png}
-\caption{Where the work goes. Left: operation counts---GEMM and IPCQ-copy
-volume both scale with users and PEs. Right: summed engine occupancy on a
-log scale---the DMA engine dominates by two to three orders of magnitude
-over the GEMM and MATH engines in every panel.}
-\label{fig:gqa-break}
-\end{figure}
-
-The dominant observation is in Figure~\ref{fig:gqa-break}: the compute
-engines are almost idle. The GEMM engine accumulates only
-\SIrange{2}{33}{\nano\second} of busy time across the panels and the
-vector-math engine \SIrange{5}{688}{\nano\second}, while the DMA engine
-accumulates \SIrange{72}{15920}{\nano\second}. Fused GQA, as modeled here,
-is overwhelmingly data-movement bound. The operation mix shows why the
-collective machinery matters: IPCQ-copy count rises from zero (single-user
-prefill) to 93 (four-user decode) as the kernel reduces partial outputs
-across more PEs and CUBEs, and DMA-read count rises in step as more KV
-shards are streamed. The PE control-processor dispatch cost registered as
-zero in this configuration---command issue is simply not on the critical
-path when data movement is this dominant.
-
-\subsection{Long-context decode: parallelism strategies}
 
 The four headline panels above stress the kernel at moderate context
 lengths. Long-context decode---the regime where KV cache size, not