paper(1H): reflect D8 single-op cost model in §2/§3.4 + figure/diagram regen
Two strands bundled as the 1H-codesign-paper refresh unit: (A) This session — single-op cost-model reflection (depends on2d8271c): - §2 Table 2 (tab:hw): split "FIXED per command" into "FIXED per single-op command" (8 cycles) and "FIXED per composite command" (40 cycles); §2 dispatch-overhead prose updated to the two-class split. - §3.4 (sec:gemm-vs-async): rename paragraph headers + prose to async-full / async-tiled; "atomic" -> "single-op" throughout; reframe mechanism #3 from the old DMA-only fast-path to the single-op fast-path. Headline narrative now: even with EVERY single-op cmd (96 DMA + 48 dot + 47 add) charged the light 8-cycle FIXED, composite still wins ~2.8x at K=3072 purely on command-count structure (1 vs 192 commands) -- down from the pre-D8 ~6.3x, and explicitly NOT a modelling artifact. Numbers refreshed from the regenerated sweep: async-full 3.83->3.91, async-tiled 1.14->~2.53, under-tile corner 1.06->1.21, depth-2 vs depth-inf spread <1%. New figure wired in. - build/main.pdf rebuilt (tectonic); pdftotext-verified (no broken refs; Table 2 split, single-op terms, 2.8x/2.53/192-host-commands all present). (B) Prior-session paper work riding along uncommitted: §4 all-reduce deep-edit, §5 GQA, §6 discussion trims; milestone_1h_ccl.py plot label "FSIM" -> "H2 2025 SW queue baseline"; regenerated diagrams under docs/diagrams/** and gemm output PNGs under 1H_milestone_output/gemm/. (Composite-window gemm plots are unaffected by D8 — D8 only changes single-op dispatch FIXED, which the composite window excludes.) All TODO items for the D8 single-op extension are now complete and pushed across 3 commits (2d8271ccost-model+ADR+tests,821bbf2bench harness, this paper refresh). Full regression green (826 passed, 1 skipped). No remaining work. NOTE for review (carried from2d8271c): ADR-0065's "2x CPU-offload win" headline for GQA decode opt2 may want a refresh to the post-D8 ~1.87x. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -243,14 +243,18 @@ serializing the command descriptor into the scheduler queue. This
|
||||
\SI{2}{\nano\second} / \SI{1}{\nano\second} \textsf{PE\_CPU} /
|
||||
\textsf{PE\_SCHED} fixed costs: the latter are per-component
|
||||
traversal overheads each command pays when it transits those nodes,
|
||||
whereas the 40-cycle \textsf{FIXED} term is the command-descriptor
|
||||
issue cost. The
|
||||
default anchoring (\textsf{FIXED} $= 40$ cycles, $R = 0.0625$
|
||||
cycles/byte, i.e.\ \SI{16}{\byte\per\cycle}, at \SI{1}{\giga\hertz})
|
||||
places a typical composite at roughly \SI{43}{\nano\second}, and a
|
||||
hard cap on a composite's descriptor size prevents the model from
|
||||
rewarding arbitrarily large fused commands beyond what real descriptor
|
||||
queues accept. In the configurations measured here, command issue is
|
||||
whereas the \textsf{FIXED} term here is the command-descriptor
|
||||
issue cost. \textsf{FIXED} depends on the command class: a single-op
|
||||
command --- one engine operation, i.e.\ a single DMA descriptor, GEMM,
|
||||
or elementwise issue --- carries a lighter 8-cycle \textsf{FIXED},
|
||||
while the composite command, which the scheduler expands into a
|
||||
multi-stage tile-feeder plan, carries 40 cycles
|
||||
(\S\ref{sec:gemm-vs-async}). With the composite \textsf{FIXED} $= 40$
|
||||
cycles and $R = 0.0625$ cycles/byte (i.e.\ \SI{16}{\byte\per\cycle},
|
||||
at \SI{1}{\giga\hertz}) a typical composite lands at roughly
|
||||
\SI{43}{\nano\second}, and a hard cap on a composite's descriptor size
|
||||
prevents the model from rewarding arbitrarily large fused commands
|
||||
beyond what real descriptor queues accept. In the configurations measured here, command issue is
|
||||
not the bottleneck---data movement is---so this term stays small
|
||||
relative to DMA and collective time.
|
||||
|
||||
@@ -452,7 +456,8 @@ Inter-CUBE (UCIe PHY) & \SI{512}{\giga\byte\per\second}, \SI{8}{\nano\second}, X
|
||||
Inter-SIP (PCIe) & \SI{768}{\giga\byte\per\second} per endpoint \\
|
||||
\midrule
|
||||
\multicolumn{2}{@{}l}{\emph{Command-issue cost model (defaults)}} \\
|
||||
FIXED per command & 40 cycles \\
|
||||
FIXED per single-op command & 8 cycles \\
|
||||
FIXED per composite command & 40 cycles \\
|
||||
per-byte rate $R$ & 0.0625 cycles/byte (\SI{16}{\byte\per\cycle}) \\
|
||||
composite size cap & \SI{1024}{\byte} \\
|
||||
\bottomrule
|
||||
|
||||
Reference in New Issue
Block a user