gqa-decode-composite: measure primitive hand-tiled (16×16×16) latency

Browse Source

Switch primitive hand-tiled Q·Kᵀ / P·V from a deferred-K-sum tl.dot
chain to per-block GemmCmd writes into a shared (M, N) out handle
(implicit MAC-side accumulation). Full-shape coarse Q / K_T / V loads
carry data-mode correctness; per-block DMAs remain for the streaming
architecture dispatch story. The kernel now runs in engine mode
end-to-end, so its 128K latency is measured instead of derived as
primitive + Δdispatch. Drop the coarse-primitive baseline from the
sweep and plots; keep the kernel file for reference.

At S_kv=128K: primitive hand-tiled = 959.5 µs vs composite = 460.7 µs
(2.08× from 5 235 vs 94 PE_CPU commands).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

...

This commit is contained in:

Mukesh Garg

2026-07-13 21:49:57 -07:00

parent eec61838b5

commit 23a2bc3fb3

8 changed files with 95 additions and 195 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

docs/report/1H-codesign-paper/figures/gqa_decode_long_ctx_composite.png

BIN

Binary file not shown.

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 151 KiB After Width:  |  Height:  |  Size: 149 KiB

docs/report/1H-codesign-paper/figures/gqa_decode_long_ctx_composite_breakdown.png

BIN

Binary file not shown.

Side by Side Swipe Overlay Before Width:  |  Height:  |  Size: 75 KiB After Width:  |  Height:  |  Size: 73 KiB