\section{Discussion: which hardware changes are meaningful}
\label{sec:discussion}

Read together, the three studies point to a consistent ranking of where
hardware investment pays off for attention-centric decoding.

\paragraph{The composite command is foundational, but indirectly.} Its
direct effect---driving compute-rich GEMMs to
\textasciitilde\SI{78}{\percent} of peak---matters most for the
compute-bound parts of a model (the large feed-forward and projection
matrices). For attention itself, its more important effect is
\emph{diagnostic}: by making issue and compute nearly free, it removes the
overhead that would otherwise mask the true bottleneck, and the fused GQA
results then show unambiguously that the kernel is data-movement bound.
Without a cheap, self-routing issue mechanism we would not be able to tell
whether attention is slow because of compute or because of data movement;
with it, the answer is clear.

\paragraph{The communication path is where attention latency actually
lives.} Every all-reduce and fused-GQA measurement says the same thing:
the limiting resource is moving and reducing data, not multiplying it. That
makes the PE\_IPCQ design and the choices around it the highest-value
hardware levers for this workload:
\begin{itemize}
  \item \textbf{On-device collectives with a compute/communication
  virtual-channel split.} Performing the reduction on the device, with
  \texttt{vc\_comm} separated from \texttt{vc\_compute} so the reduction
  does not stall the compute DMA, is what lets attention overlap KV
  movement with score computation at all.
  \item \textbf{Fast on-PE staging memory.} Keeping the IPCQ buffer in TCM
  rather than HBM or SRAM is worth \SI{14}{}--\SI{38}{\percent} of
  collective latency at large payloads---a pure placement decision with a
  first-order effect.
  \item \textbf{Wrap-around (torus) inter-device links.} A torus fabric
  buys \SI{20}{}--\SI{25}{\percent} over a mesh or ring at scale by
  shortening the worst-case reduction path.
\end{itemize}

\paragraph{Raw MAC throughput is not the constraint for attention.} The
GQA panels leave the GEMM and vector-math engines two to three orders of
magnitude below the DMA engine in busy time. Adding MAC area would not move
decode latency; the workload cannot use it. This is the single most
actionable finding for an attention-dominated roadmap. The implication is
that future hardware investment should prioritize communication and
memory-system efficiency over additional compute throughput for
attention-dominated inference workloads.

\paragraph{Caveats.} These conclusions are achievable-kernel results from a
deterministic model, not E2E measurements; absolute numbers carry the
model's idealizations (\S\ref{sec:latency}), though the relative rankings
that drive the recommendations are robust to them. The headline GQA panels
are also at modest scale (up to four users, single SIP), and one designed
refinement---the two-composite \textsf{softmax\_merge} decode---is not yet
in the measured path. Larger-scale and multi-SIP headline runs are 2H work.