gqa(adr/ddd): iteration-2 corrections — GQA reuse via M-fold, transpose caveat
Verified against sim_engine data path (memory_store, data_executor): - GQA reuse does NOT need a broadcast op. The baseline's h_q==h_kv limit is its head-packing reshape hack, not a missing primitive. Correct fix: per-KV-head loop with G folded into matmul M dim (byte-conserving reshape) — runs today, timing correct (m=G*T_q), data mode runs. - ADR-0061 broadcast demoted from 'the blocker' to optional convenience. - Surfaced tl.trans = reshape-not-transpose (memory_store reshapes; data_executor np.matmul on reshaped operands) -> numeric parity is bounded; verification is structural/timing/determinism-first (matches SPEC perf-model purpose). Optional tl.transpose deferred. - Reordered DDD phase plan (P1 GQA needs no new feature; P3 scratch_scope is the key scale feature); added open decisions 10.10 (transpose) and 10.11 (GQA-via-M-fold finding). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -4,11 +4,21 @@
|
||||
|
||||
Proposed
|
||||
|
||||
> Supporting ADR for **ADR-0060** (AHBM GQA Fused Attention). This ADR
|
||||
> removes the single hard blocker that keeps the existing attention
|
||||
> kernels at `h_q == h_kv == 1`: there is no way to expand a KV head to
|
||||
> the `G` query heads that share it without violating the MemoryStore
|
||||
> byte-conservation check.
|
||||
> Supporting ADR for **ADR-0060** (AHBM GQA Fused Attention).
|
||||
>
|
||||
> **⚠ Re-evaluated during design iteration — this is now OPTIONAL, not a
|
||||
> GQA blocker.** The original framing (below) held that GQA `h_q > h_kv`
|
||||
> is blocked by the MemoryStore byte-conservation check on a broadcast
|
||||
> view. That is true *only of the baseline's head-packing hack*
|
||||
> (`_view(K, (h_q·d, S_kv))`). The correct GQA design (ADR-0060 §5.2)
|
||||
> processes **one KV head at a time** and folds the `G` group rows into
|
||||
> the matmul **M** dimension — using only byte-conserving reshapes — so it
|
||||
> needs **no broadcast op at all**, and `np.matmul` already broadcasts a
|
||||
> shared operand in Phase-2 data mode. This ADR therefore drops to a
|
||||
> *convenience* primitive: it cleans up additive-mask construction across
|
||||
> the `G·T_q` rows and serves general kernels. **Lowest priority of the
|
||||
> three supporting ADRs.** The rest of this document is retained as the
|
||||
> design for that convenience op.
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
Reference in New Issue
Block a user