gqa(adr): ADR-0064/0065 second-pass review — 1024B rationale, FIFO as ordering source, robust formula-based tests, D5/D6 auto-bind alignment

ADR-0064 Revision 2 second-pass review fixes:
- D7 added 1024-byte cap rationale: explicitly framed as a *safe
  engineering limit* (intentionally above all known composites at ~322
  bytes, finite descriptor capacity placeholder), not a measured HW
  number. Default is the discipline; topology override for real HW.
- D7 segment ordering: strict FIFO is the *ordering source*; rw_handles
  is dependency metadata, not the ordering primitive. Explicit note for
  the future RW-aware reorder migration path.
- Tests rewritten around the formula (D1) instead of specific numbers:
  - Test #1 → "formula preservation" (parametrized across composites,
    asserts dispatch == FIXED + bytes × R, not anchor 43 ns).
  - Test #2 → robust "opt3 > 2 × opt2" qualitative gate; ≈ 4.0× is
    informative only, not the gate, so calibration changes don't break.
  - Test #9 → qualitative (opt2 < opt3 at all R), not a numeric ratio.
- Anchor description in D3 stays as *informative* (shows the model
  produces ≈ 43 ns at defaults), but is no longer a test gate.

ADR-0065 second-pass review:
- D5 step 5 aligned with D6.6 invariant: explicit conflict check before
  auto-binding (previously written as if auto-bind always happens).
  Mirrors D6.6 wording: error if both kernel explicit `a` and prologue
  primary_out are present.
- Test #7 made robust: `opt3 > 2 × opt2` gate (was ≈ 4.0×).
  Default-calibration model expectation (≈ 4.0×) recorded as informative
  reference, not the gate.

DDD-0065 follow-on robust thresholds:
- §1.4 success criteria: ratio condition rewritten as `opt3 > 2 × opt2`
  with model-expected ≈ 4.0× as informative.
- §9 P6 phase gate: `opt3 > 2 × opt2` (was 4.0× ± 15%).
- §10 dispatch ratio test: assert `opt3 > 2 × opt2`; record observed
  ratio for performance tracking but do not gate on it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 16:34:34 -07:00
parent a8d04750e6
commit 3647e2d8f5
5 changed files with 130 additions and 47 deletions
@@ -234,14 +234,31 @@ issue cost 를 0 → non-zero 로 바꾸면 **모든** bench 의 latency 변화.
- segmentation 알고리즘 결정적 (emit 순서로 op 인덱스 greedy); host
emitter 의 일부, PE_SCHEDULER 가 아님.
필요한 이유. 실제 하드웨어는 descriptor queue entry 크기, scheduler
parser buffer, command SRAM, firmware 입력의 하드 제한 보유. cap 없으면
`FIXED + bytes × R` 모델이 하드웨어가 받지 못할 정도로 큰 fused composite
를 보상 (예: primitive 100 개를 composite 1 개로 fuse, FIXED 1 회만 지불).
~30 bytes 짜리 op 100 개의 composite = ~3000 bytes > 1024 cap → 3 개로
segment → 3 × FIXED, 정직한 회계 회복.
**cap 이 필요한 이유.** 실제 하드웨어는 descriptor queue entry 크기,
scheduler parser buffer, command SRAM, firmware 입력의 하드 제한 보유.
cap 없으면 `FIXED + bytes × R` 모델이 하드웨어가 받지 못할 정도로 큰
fused composite 를 보상 (예: primitive 100 개를 composite 1 개로 fuse,
FIXED 1 회만 지불).
Decode opt2 의 `#2` composite (10 ops, ~310 bytes) 는 1024 cap 안에
**1024 bytes 라는 *특정* 숫자의 근거.** 이는 측정된 HW 숫자가 아니라
**safe engineering limit** — kernbench 코드베이스의 현재 모든 알려진
composite (가장 큰 것이 decode opt2 의 `#2` ~322 bytes) 보다 훨씬 위이면서
미래 recipe 도 지켜야 할 *유한* descriptor capacity 를 의도적으로 표현.
값은 topology 별 override (D4); 실제 HW reference 등장 시 재캘리브레이션.
이 default 의 역할은 cap 을 *원칙으로 존재시키는 것*, 특정 HW 에 맞추는
것이 아님.
**segment 간 순서 — strict FIFO 가 주체, `rw_handles` 아님.** segment 들은
emit 순서로 PE_CPU → PE_SCHEDULER 큐에 들어감. strict-FIFO dispatch
(ADR-0065 D6.3) 가 *순서의 주체* — segment 가 큐 진입 순서로 실행. 각
segment 의 `rw_handles` block 은 cross-composite 해저드 추적기의 **의존성
메타데이터** — 그 자체로 inter-segment 순서를 보장하지 **않음**. 미래
scheduler 가 FIFO 를 완화 (예: RW-aware reorder, ADR-0065 A4) 한다면
segmenter 는 (a) segment 를 단일 CompositeCmd 로 merge 또는
(b) 명시적 completion-handle 의존 체인 도입 필요. Phase 1 에선 무관 —
strict FIFO 가 유효.
Decode opt2 의 `#2` composite (10 ops, ~322 bytes) 는 1024 cap 안에
편안히 — GQA workload 에 segmentation 없음.
## Alternatives
@@ -307,12 +324,24 @@ fold. calibration 이 분리 필요성을 보이면 PE_DMA fixed setup 으로
## Test Requirements
1. **Anchor 보존.** DMA-staged GEMM 경로용 단일-OpSpec composite
(`logical_bytes ≈ 54`) 가 default 값에서 ≈43 ns 에 dispatch (±2 ns).
2. **구조적 ratio.** opt3 vs opt2 dispatch (ADR-0065 §verification 의):
default calibration 에서 `opt3 / opt2 ≈ 4.0×`.
테스트는 specific 숫자 anchor 가 아닌 **공식 (D1)** 에 대해 작성 —
calibration 이 바뀌거나 OpSpec/CompositeCmd 필드가 추가되어도 유효.
1. **공식 보존.** 임의 `CompositeCmd` `c` 에 대해 PE_CPU 의 기록된
dispatch overhead 가 `FIXED_PER_CMD + c.logical_bytes × R`
(floor/round-off 의 ±1 cycle 내). 여러 composite parametrize:
1-OpSpec GEMM composite, 5-op MATH chain, 10-op recipe composite.
Default-calibration 숫자 (1-OpSpec composite 의 anchor ≈43 ns) 는
*informative* 참고, 테스트 gate 가 아님 — gate 는 공식 등가.
2. **정성적 ratio (robust).** opt3 per-tile PE_CPU dispatch 가 opt2
per-tile dispatch 를 최소 2× 차이로 엄격히 초과 — `opt3 > 2 × opt2`.
Default-calibration 모델은 ≈4× 예측; gate 는 느슨한 2× 한계라
calibration 이동 (예: HW reference 가 default 대체) 시 깨지지 않음.
Informative 숫자 — ADR-0065 §verification 와 DDD-0065 §11 의 모델
기대치 참조.
3. **Override 경로.** topology yaml 의 `pe_cost_model:` block 이 per-PE
dispatch cost 변경; block 누락 시 default 복귀.
dispatch cost 변경; block 누락 시 default 복귀. #1 의 공식 등가가
override 값에서도 성립.
4. **`PeCpuOverheadCmd` 우회.** 수동 `tl.cycles(n)` 는 정확히 `n` cycles,
`n + dispatch_cycles(...)` 아님.
5. **double-count 없음.** PE_DMA `drain_ns`, PE_GEMM/MATH `_compute_ns` 가
@@ -322,12 +351,12 @@ fold. calibration 이 분리 필요성을 보이면 PE_DMA fixed setup 으로
dispatch-cycle 회계.
8. **Composite 크기 cap (D7).** `logical_bytes >
MAX_COMPOSITE_LOGICAL_BYTES` 가 될 recipe 가 N 개 연속 `CompositeCmd`
로 segment; 총 dispatch = segment dispatch 의 합; segment 간 RW 순서
strict FIFO 로 보존.
9. **민감도 sweep.** `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte
(= 4 / 16 / 32 bytes/cycle) 에서 *opt2 per-tile dispatch
< opt3 per-tile dispatch* 결론 유지 (ratio 가 R 감소 시 단조 증가
— FIXED 가 더 지배).
로 segment; `sum(segment.logical_bytes) == original logical_bytes`;
총 dispatch = segment dispatch 의 합 (FIXED 가 segment 당 지불);
inter-segment 순서 strict FIFO 로 보존 (`rw_handles` 단독 아님).
9. **민감도 (정성적).** `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte 에서
3 지점 모두 `opt3 > opt2`. 방향 (R 감소 시 ratio 단조 증가) 도 단언,
단 절대 ratio 값은 *불필요*.
## Migration
@@ -251,15 +251,34 @@ per D4). Oversized commands are deterministically **segmented** by the
- The segmentation algorithm is deterministic (greedy by op index in
emit order); it is part of the host emitter, not PE_SCHEDULER.
Why this is needed. Real hardware imposes hard limits — descriptor
**Why a cap is needed.** Real hardware imposes hard limits — descriptor
queue entry size, scheduler parser buffer, command SRAM, firmware
input. Without a cap, the `FIXED + bytes × R` model would reward
arbitrarily large fused composites beyond what hardware accepts (e.g.,
fusing 100 primitive ops into one composite, paying one `FIXED`).
A composite of (say) 100 ops at ~30 bytes each = ~3000 bytes >
1024 cap → segmented into 3 ≈ 3 × FIXED, restoring honest accounting.
Decode opt2's `#2` composite (10 ops, ~310 bytes) sits comfortably
**Why 1024 bytes specifically.** This is a **safe engineering limit**,
not a measured HW number — intentionally chosen to be well above all
currently known composites (decode opt2's `#2` at ~322 bytes is the
largest in the kernbench codebase) while still representing a *finite*
descriptor capacity that future recipes must respect. The number is
overridable per topology (D4); when a real HW reference appears, the
value should be recalibrated. The role of this default is to make the
cap *exist as a discipline*, not to fit a specific HW.
**Ordering of segments — driven by strict FIFO, not `rw_handles`.**
Segments are emitted into the PE_CPU → PE_SCHEDULER queue in their
emit order. Strict-FIFO dispatch (ADR-0065 D6.3) is the *ordering
source*: segments execute in the order they enter the queue. The
`rw_handles` block on each segment is **dependency metadata** for the
cross-composite hazard tracker — it does **not** by itself guarantee
inter-segment ordering. If a future scheduler relaxed FIFO (e.g., to
RW-aware reorder, ADR-0065 A4), the segmenter would need to either
(a) merge segments into a single CompositeCmd, or (b) introduce an
explicit completion-handle dependency chain. For Phase 1 this does
not arise: strict FIFO is in effect.
Decode opt2's `#2` composite (10 ops, ~322 bytes) sits comfortably
inside the 1024 cap — no segmentation for the GQA workload.
## Alternatives
@@ -331,13 +350,27 @@ calibration shows it matters.
## Test Requirements
1. **Anchor preservation.** A single-OpSpec composite for a DMA-staged
GEMM path (`logical_bytes ≈ 54`) dispatches in ≈43 ns at default
values (within ±2 ns).
2. **Structural ratio.** opt3 vs opt2 dispatch (per ADR-0065 §verification):
`opt3 / opt2 ≈ 4.0×` at default calibration.
Tests are written against the **formula** (D1), not against specific
numeric anchors, so they remain valid when calibration changes or when
OpSpec/CompositeCmd fields are added.
1. **Formula preservation.** For any `CompositeCmd` `c`, PE_CPU's
recorded dispatch overhead equals `FIXED_PER_CMD + c.logical_bytes
× R` (within ±1 cycle for floor/round-off). Parametrized over
several composites: a 1-OpSpec GEMM composite, a 5-op MATH chain,
and a 10-op recipe composite. Default-calibration numbers (anchor
≈43 ns for the 1-OpSpec composite) are informative reference, not
the test gate — the test gate is the formula equality.
2. **Qualitative ratio (robust).** opt3 per-tile PE_CPU dispatch
strictly exceeds opt2 per-tile dispatch by at least a 2× margin —
`opt3 > 2 × opt2`. The default-calibration model predicts ≈4×;
the gate is the loose 2× bound so the test does not break when
calibration is moved (e.g., when a HW reference replaces the
default). Informative numbers — see ADR-0065 §verification and
DDD-0065 §11 for the model expectation.
3. **Override path.** Topology yaml `pe_cost_model:` block changes the
per-PE dispatch cost; default is recovered when block is missing.
The formula identity from #1 must hold with the override values.
4. **`PeCpuOverheadCmd` bypass.** Manual `tl.cycles(n)` issues exactly `n`
cycles, not `n + dispatch_cycles(...)`.
5. **No double-count.** PE_DMA `drain_ns`, PE_GEMM/MATH `_compute_ns`
@@ -348,12 +381,14 @@ calibration shows it matters.
dispatch-cycle accounting for the same kernel.
8. **Composite size cap (D7).** A recipe that would emit `logical_bytes
> MAX_COMPOSITE_LOGICAL_BYTES` is segmented into N consecutive
`CompositeCmd`s; total dispatch = sum of segment dispatches; RW
ordering across segments preserved by strict FIFO.
9. **Sensitivity sweep.** At `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte
(= 4 / 16 / 32 bytes/cycle), the conclusion *opt2 per-tile dispatch
< opt3 per-tile dispatch* must hold (ratio monotonically increases
as R decreases — FIXED dominates more).
`CompositeCmd`s; `sum(segment.logical_bytes) == original
logical_bytes`; total dispatch = sum of segment dispatches (FIXED
paid per segment); inter-segment ordering preserved by strict FIFO
(not by `rw_handles` alone).
9. **Sensitivity (qualitative).** At `R ∈ {0.25, 0.0625, 0.03125}`
cycles/byte, `opt3 > opt2` at *all* three points. Direction (ratio
monotonically increases as R decreases) is also asserted, but
absolute ratio values are *not* required.
## Migration
@@ -179,7 +179,10 @@ op="gemm", ...)` 에서:
3. primary output P 의 shape 을 `s.shape` 에서 derive (identity).
4. `engine_seq` 를 8 개의 평평한 MATH OpSpec 으로 펼침 (모든 주소/크기 채움).
각 OpSpec 의 `scope = KERNEL`.
5. head GEMM `operands["a"] = P_handle` auto-bind.
5. **Auto-bind (충돌 검사, D6.6).** head GEMM `operands["a"]`
*명시하지 않은* 경우에만 `operands["a"] = P_handle` auto-bind.
커널이 `a` 를 명시적으로 제공했다면 validation error — 모호한
primary-output 대체.
6. `CompositeCmd(ops=(8 MATH + 1 GEMM + epilogue), rw_handles=(m, l, O))` emit.
**RECIPE_DESCRIPTORS 는 HW 경로 어디에도 안 나타남**. PE_SCHEDULER 는
@@ -331,9 +334,12 @@ scratch 에서 읽는 형태 — 순환 tile-loop 의존성. **기각 (incorrect
emit 안 함.
6. **opt2 가 opt3 와 수치 동등.** data mode 에서 opt2 의 최종
`(m, l, O)` 가 opt3 와 fp tolerance 안.
7. **opt2 dispatch ratio (ADR-0064 Rev2 이후).** opt3 vs opt2 per-tile
PE_CPU dispatch cycles ratio 가 default calibration (FIXED=40 cycles,
R=0.0625 cycles/byte) 에서 ≈ 4.0×. Ratio 가 FIXED-dominated —
7. **opt2 dispatch ratio (ADR-0064 Rev2 이후) — robust.** opt3 vs opt2
per-tile PE_CPU dispatch cycles 가 `opt3 > 2 × opt2` 만족
(정성적 gate, calibration-독립). Default-calibration 모델 기대치는
≈ 4.0× (FIXED=40 cycles, R=0.0625 cycles/byte) — DDD-0065 §11 의
모델-유도 숫자 참조; 테스트 gate 는 느슨한 `> 2×` 한계라
calibration 변경에도 살아남음. Ratio 가 FIXED-dominated —
command-count 감소가 1차 신호임을 반영.
8. **GEMM-count invariant.** GEMM OpSpec 두 개 가진 composite 가
TLContext emit 시 validation error.
@@ -200,7 +200,11 @@ op="gemm", ...)`:
3. Derives the primary output P's shape from `s.shape` (identity).
4. Expands `engine_seq` into 8 flat MATH OpSpecs with all addresses /
sizes filled. Each OpSpec gets `scope=KERNEL`.
5. Auto-binds head GEMM's `operands["a"] = P_handle`.
5. **Auto-binds (with conflict check, D6.6).** If the head GEMM does
*not* explicitly provide `operands["a"]`, auto-bind
`operands["a"] = P_handle`. If the kernel *did* provide `a`
explicitly, emit a validation error — ambiguous primary-output
replacement.
6. Emits `CompositeCmd(ops=(8 MATH + 1 GEMM + epilogue), rw_handles=
(m, l, O))`.
@@ -370,10 +374,14 @@ input from #1's epilogue scratch — circular tile-loop dependency.
`space=hbm` emits DMA_WRITE; `space=tcm` does not.
6. **opt2 numeric parity with opt3.** In data mode, opt2's final
`(m, l, O)` matches opt3's within fp tolerance.
7. **opt2 dispatch ratio (after ADR-0064 Rev2).** opt3 vs opt2 per-tile
PE_CPU dispatch cycles ratio ≈ 4.0× at default calibration
(FIXED=40 cycles, R=0.0625 cycles/byte). Ratio is FIXED-dominated,
reflecting command-count reduction as the primary signal.
7. **opt2 dispatch ratio (after ADR-0064 Rev2) — robust.** opt3 vs
opt2 per-tile PE_CPU dispatch cycles satisfies `opt3 > 2 × opt2`
(qualitative gate; calibration-independent). The default-calibration
model expectation is ≈ 4.0× (FIXED=40 cycles, R=0.0625 cycles/byte)
— see DDD-0065 §11 for the model-derived numbers; only the loose
`> 2×` bound is the test gate so it survives calibration changes.
Ratio is FIXED-dominated, reflecting command-count reduction as the
primary signal.
8. **GEMM-count invariant.** A composite with two GEMM OpSpecs raises a
validation error at TLContext emit.
9. **Within composite size cap (ADR-0064 D7).** Decode opt2's `#2`
@@ -26,9 +26,11 @@ unchanged.
3. **Boundary preservation.** PE_SCHEDULER imports nothing recipe-related;
PE_MATH/PE_GEMM/PE_DMA see only `Stage.params["op_kind"]` (no new
engine code).
4. **Dispatch ratio.** Per-tile PE_CPU dispatch cycles
`ratio(opt3 / opt2) ≈ 4.0×` at default ADR-0064 Rev2 calibration
(FIXED=40 cycles, R=0.0625 cycles/byte).
4. **Dispatch ratio (robust).** Per-tile PE_CPU dispatch cycles satisfy
`opt3 > 2 × opt2` (qualitative invariant, calibration-independent).
Default ADR-0064 Rev2 calibration (FIXED=40 cycles, R=0.0625
cycles/byte) gives model-expected ≈ 4.0× — informative only, not the
test gate.
5. **K-before-V invariant.** op_log of decode opt2 shows zero V-related
DMA during #2's MATH prologue.
@@ -437,7 +439,7 @@ Each phase is an independent Phase-1 → Phase-2 cycle per CLAUDE.md Part 1.
| **P3** | PE_SCHEDULER `_generate_plan` flat-ops + `PipelinePlan.prologue_stages` + DMA auto-insertion + `_feed_loop` extension | all existing bench Stage sequences unchanged; new MATH chain visible in op_log for opt2 mock |
| **P4** | Strict-FIFO RW hazard tracker + `_RwHazardTracker` integration | two-composite RW conflict test serializes correctly |
| **P5** | `_gqa_decode_long.py` opt2 variant; data-mode numeric parity check; K-before-V invariant check | opt2 matches opt3 within fp tolerance; no V DMA during prologue |
| **P6** | Dispatch-ratio measurement: opt3 vs opt2 per-tile PE_CPU cycles + R sensitivity sweep | ratio ≈ 4.0× ± 15% at default calibration; opt2 < opt3 at all `R ∈ {0.25, 0.0625, 0.03125}` (ADR-0064 Test #9) |
| **P6** | Dispatch-ratio measurement: opt3 vs opt2 per-tile PE_CPU cycles + R sensitivity sweep | `opt3 > 2 × opt2` at default calibration (gate); model-expected ≈ 4.0× (informative); `opt2 < opt3` at all `R ∈ {0.25, 0.0625, 0.03125}` (ADR-0064 Test #9) |
P0 must land first (separate ADR). P1 is pure refactor (safest). P2P4
build the new path without touching opt3. P5 enables opt2. P6 closes
@@ -488,7 +490,10 @@ Mirrors ADR-0065 §Test Requirements; grounded in SPEC R2/R5, ADR-0023/
**Dispatch ratio (P6):**
- For `S_kv=64, n_tiles=16`, measure total PE_CPU dispatch cycles for
opt3 vs opt2 paths.
- Assert ratio in `[3.4, 4.6]` (centre 4.0× at ADR-0064 Rev2 defaults).
- Assert `opt3 > 2 × opt2` (robust gate — calibration-independent).
- Record observed ratio; the default-calibration model expects ≈ 4.0×
(DDD §11). The recorded number is informative for performance
tracking, not a test gate.
- **Sensitivity sweep.** Repeat with `R ∈ {0.25, 0.0625, 0.03125}`
cycles/byte; assert opt2 < opt3 in all three. (ADR-0064 Test #9.)