paper(1H): reflect D8 single-op cost model in §2/§3.4 + figure/diagram regen

Two strands bundled as the 1H-codesign-paper refresh unit:

(A) This session — single-op cost-model reflection (depends on 2d8271c):
  - §2 Table 2 (tab:hw): split "FIXED per command" into "FIXED per
    single-op command" (8 cycles) and "FIXED per composite command"
    (40 cycles); §2 dispatch-overhead prose updated to the two-class split.
  - §3.4 (sec:gemm-vs-async): rename paragraph headers + prose to
    async-full / async-tiled; "atomic" -> "single-op" throughout; reframe
    mechanism #3 from the old DMA-only fast-path to the single-op
    fast-path. Headline narrative now: even with EVERY single-op cmd
    (96 DMA + 48 dot + 47 add) charged the light 8-cycle FIXED, composite
    still wins ~2.8x at K=3072 purely on command-count structure (1 vs
    192 commands) -- down from the pre-D8 ~6.3x, and explicitly NOT a
    modelling artifact. Numbers refreshed from the regenerated sweep:
    async-full 3.83->3.91, async-tiled 1.14->~2.53, under-tile corner
    1.06->1.21, depth-2 vs depth-inf spread <1%. New figure wired in.
  - build/main.pdf rebuilt (tectonic); pdftotext-verified (no broken
    refs; Table 2 split, single-op terms, 2.8x/2.53/192-host-commands
    all present).

(B) Prior-session paper work riding along uncommitted: §4 all-reduce
    deep-edit, §5 GQA, §6 discussion trims; milestone_1h_ccl.py plot
    label "FSIM" -> "H2 2025 SW queue baseline"; regenerated diagrams
    under docs/diagrams/** and gemm output PNGs under
    1H_milestone_output/gemm/. (Composite-window gemm plots are
    unaffected by D8 — D8 only changes single-op dispatch FIXED, which
    the composite window excludes.)

All TODO items for the D8 single-op extension are now complete and
pushed across 3 commits (2d8271c cost-model+ADR+tests, 821bbf2 bench
harness, this paper refresh). Full regression green (826 passed, 1
skipped). No remaining work.

NOTE for review (carried from 2d8271c): ADR-0065's "2x CPU-offload win"
headline for GQA decode opt2 may want a refresh to the post-D8 ~1.87x.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

docs/report/1H-codesign-paper/sections/05-gqa.tex

@@ -1,8 +1,6 @@
 \section{Fused Grouped-Query Attention}
 \label{sec:gqa}
 
-\subsection{Why it is needed}
-
 Attention is the 1H focus, and it is where the two preceding optimizations
 have to come together. Grouped-Query Attention (GQA) shrinks the KV cache
 by sharing each KV head across a group of query heads (here $h_q=8$ query
@@ -16,12 +14,31 @@ realizing it as a fast \emph{fused} kernel needs both building blocks from
 this report: efficient GEMM issue (\S\ref{sec:gemm}) for the
 $Q\!\cdot\!K^{\top}$ and $P\!\cdot\!V$ products, and an efficient on-device
 reduction (\S\ref{sec:allreduce}) for the multi-user and
-sequence-parallel KV reductions. This section is the capstone: the fused
+sequence-parallel KV reductions. \emph{Fused} here is meant in the
+FlashAttention sense---$Q\!\cdot\!K^{\top}$, the online softmax, and
+$P\!\cdot\!V$ collapse into a single kernel that never materializes the
+score matrix---and, beyond that, the cross-device KV reduction is absorbed
+into the same kernel (on PE\_IPCQ) rather than issued as a separate
+all-reduce. This section is the capstone: the fused
 kernel that uses the composite command and PE\_IPCQ at the same time.
 Multi-head attention (MHA) was studied in prior work and serves here as
 the established baseline rather than being re-derived.
 
-\subsection{Design}
+\subsection{Data Placement Policy}
+\label{sec:gqa-placement}
+
+% TODO: compare data-placement options that apply across both the
+% short- and long-context regimes. Candidate axes:
+%   - KV cache:  per-CUBE shard vs. replicate; per-PE shard vs. replicate
+%   - Q / W_qkv / W_o weights:  static partition across CUBEs and PEs
+%   - Workspace (m, l, O softmax state):  scratch arena placement
+% The 4-case taxonomy used for long-context decode in
+% \S\ref{sec:gqa-long} (Cube-{SP,Repl} x PE-{TP,SP}) is one instantiation
+% of this framework; the short-context mapping in \S\ref{sec:gqa-short}
+% is another.
+
+\subsection{Inference with Short-Context Length}
+\label{sec:gqa-short}
 
 The fused GQA kernel issues its matrix products as scheduler-managed
 composite commands and keeps the online-softmax merge and the cross-device
@@ -41,7 +58,22 @@ that restructures the decode step into two stateful composites (a named
 \textsf{softmax\_merge} recipe) is designed but not yet wired into the
 measured path; results below reflect the implemented kernel only.
 
-\subsection{Results: long-context decode and parallelism strategies}
+% TODO: CUBE <-> KV-head mapping diagram for the short-context regime
+% (h_kv=8 KV heads -> 8 CUBEs, 1:1; intra-CUBE PE usage).
+% Bench code: src/kernbench/benches/gqa_helpers/short_ctx/
+
+% TODO: prefill performance figure (latency, stage breakdown).
+% TODO: decode performance figure (latency, stage breakdown).
+%       Bench output for short_ctx to be generated.
+
+\subsection{Inference with Long-Context Length}
+\label{sec:gqa-long}
+
+% TODO: prefill long-context kernel implementation description
+% (Sequence-Parallel partition of S_kv, per-case mechanics).
+% Bench code: src/kernbench/benches/gqa_helpers/long_ctx/
+
+% TODO: prefill long-context performance figure.
 
 The four headline panels above stress the kernel at moderate context
 lengths. Long-context decode---the regime where KV cache size, not
@@ -116,7 +148,8 @@ on-device collective traffic that PE\_IPCQ and the torus links of
 ``slower'' strategy is in fact the one that fully cashes in the
 communication-side codesign work of this report.
 
-\subsection{Analysis and meaning}
+\subsection{Comprehensive Analysis}
+\label{sec:gqa-analysis}
 
 These panels are the clearest statement of the codesign thesis in the
 report. Because the composite command keeps GEMM issue cheap and the MAC
@@ -133,3 +166,9 @@ optimization is what \emph{attacks} it. For an attention-dominated decoder
 the meaningful hardware investments are therefore the ones that move data
 faster and reduce it on-device---not additional MAC throughput, which this
 workload cannot use.
+
+% TODO: cross-regime DP (data parallelism) applicability:
+%   - Does Case-4 long-context placement compose with batch-level DP
+%     without further changes?
+%   - Does the short-context placement compose the same way?
+%   - Implications for multi-user serving (single vs. mixed regimes).