mukesh
fc4747668e
Three new subsections at the top of §5, motivated by the deployment-sizing questions the analytical visualization tool exposes: - 5.1 Roofline Analysis — B*, L*, short vs long context batch effect (measured: 19× per-token cost reduction at short context, 28% at long) - 5.2 Capacity Planning — 3-axis sizing (weights / KV / SLO), SLO targets, regime rules, deployment templates, per-axis playbook - 5.3 Parallelism Selection — TP × CP × PP × DP × EP comparison, symptom-driven axis selection, add/stop criteria, misconceptions Restructure: - 05-gqa.tex trimmed to section header + intro - Existing 5.4-5.7 content (placement, short/long ctx, composite) moved to 05x-fused-kernel.tex - Summary extracted to 05z-summary.tex, cross-refs updated - main.tex \input order sets the requested subsection sequence - lmodern loaded to satisfy microtype font-expansion Two new figures generated from tests/analytical_visualization/chip_roofline: - roofline_short_context.png (S_kv=8K, batch wins) - roofline_long_context.png (S_kv=1M, batch stalls at KV floor) - Generator: tests/analytical_visualization/_gen_roofline_paper_figs.py 4 tables (regime rules, deployment templates, playbook, parallelism criteria) use table* so they span both columns without overflow. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
23 lines
1.3 KiB
TeX
23 lines
1.3 KiB
TeX
\subsection{Summary}
|
|
\label{sec:gqa-analysis}
|
|
|
|
The section's results line up into one picture of the fused kernel.
|
|
Placement is decided by memory first and communication second: only the
|
|
64-way splits fit production context, and among them the both-axes
|
|
sequence shard (Case~6) minimizes the per-token collective by merging
|
|
softmax state rather than partial scores. Command form is decided by
|
|
roofline position: the composite command removes the per-tile issue work
|
|
in both regimes, but converts to wall-clock only in compute-bound prefill,
|
|
while memory-bound decode stays bound by KV streaming regardless of command
|
|
form. The thread connecting the two is that, once composite issue makes
|
|
GEMM cheap and leaves the MAC array idle, the fused kernel's latency is set
|
|
almost entirely by data movement---streaming the KV cache and reducing
|
|
partials across PEs---which is exactly the cost the communication-side
|
|
work (on-device reduction, lazy load/compute overlap, fast TCM staging and
|
|
torus links) is built to attack. Roofline framing
|
|
(\S\ref{sec:roofline}) turns this observation into a sizing and sharding
|
|
programme in the next two subsections: capacity planning
|
|
(\S\ref{sec:capacity-planning}) and parallelism selection
|
|
(\S\ref{sec:parallelism-selection}). What this implies for hardware
|
|
investment across the report as a whole is taken up in the discussion.
|