gqa(adr): ADR-0064/0065 review fixes — composite size cap, type-aware logical_bytes, R recalibration

ADR-0064 Revision 2 review fixes:
- D7 NEW: composite size cap (MAX_COMPOSITE_LOGICAL_BYTES default 1024
  bytes). Oversized recipes deterministically segmented into N
  CompositeCmds; each segment incurs its own dispatch cost. Models real
  HW limits (descriptor queue entry, scheduler parser buffer, command
  SRAM) and prevents the model from rewarding pathologically-large
  fused composites.
- D2: type-aware extra-field byte counting (int/float=4, bool=1,
  tuple/list=1+4N, str=1) — replaces uniform 4 bytes per extra.
- D3: recalibrated defaults to FIXED=40 cycles, R=0.0625 cycles/byte
  (16 B/cycle — typical on-die descriptor queue width); anchor stays at
  ~43 ns for typical 1-OpSpec composite. Clarified anchor description:
  DMA stages do not appear in logical_bytes (auto-inserted by
  PE_SCHEDULER from operand.space per ADR-0065 D4).
- D4: removed clock_freq_ghz from pe_cost_model: override block;
  conversion uses the PE node's existing clock_freq_ghz attr. Added
  max_composite_logical_bytes knob.
- Context: emphasized command-count reduction (FIXED) as the primary
  signal; byte term as secondary refinement.
- Open review: added large-composite scheduler-cost stress test.
- Test req: added composite-size-cap (#8) and R-sensitivity sweep (#9).

ADR-0065 + DDD-0065 follow-on updates:
- opt2 vs opt3 dispatch ratio updated 2.4× → ≈4.0× under new defaults
  (FIXED-dominated, reflecting the corrected framing).
- Test req #9: decode opt2 composite fits within 1024-byte cap; no
  segmentation needed for the GQA workload.
- DDD §6: TLContext lowering checks logical_bytes against cap (step 8).
- DDD §11: performance model recomputed with new defaults + sensitivity
  table across R ∈ {0.25, 0.0625, 0.03125} confirming opt2 < opt3 holds.
- DDD §9 P6 gate: ratio band 2.4×±10% → 4.0×±15%; sensitivity sweep added.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 16:20:58 -07:00
parent 7fad0371c5
commit 95cecccd8e
5 changed files with 296 additions and 58 deletions
@@ -50,6 +50,18 @@ cmd 의 *byte 발자국* 이 자연 proxy: N OpSpec composite 는 1 OpSpec primi
*고정* 부분 (큐 tail 업데이트, completion 등록, MMIO-급 latency) 은 cmd 마다. *고정* 부분 (큐 tail 업데이트, completion 등록, MMIO-급 latency) 은 cmd 마다.
둘 합치면: `FIXED + bytes × R`. 둘 합치면: `FIXED + bytes × R`.
### 이 모델이 실제로 노출하는 것
**1차** 신호는 per-cmd FIXED 비용에 의한 **command-count 감소**.
**byte** 항은 *비정상적으로 큰* composite 가 공짜로 보이지 않도록 막는
2차 보정. 현실적 on-die 큐 대역폭 (16 B/cycle, D3) 에서 FIXED 가
decode opt2 의 opt3 (≈10 cmds/tile) vs opt2 (≈2 cmds/tile) 의 dispatch
cost 차이 중 ≥85% 차지.
이 framing 이 모델의 한계도 정의: 단일 composite 가 무한히 커지면 byte
항만으로는 HW 가 받지 못할 정도의 fused command 를 모델이 보상하는 것을
막지 못함 — 그래서 **D7** 의 descriptor 크기 cap 이 필요.
## Decision ## Decision
### D1. 구조적 dispatch cost 공식 ### D1. 구조적 dispatch cost 공식
@@ -104,39 +116,75 @@ def logical_bytes(self) -> int:
1 + 1 # opcode + scope 1 + 1 # opcode + scope
+ 1 + 8 * len(self.operands) # named operand handles + 1 + 8 * len(self.operands) # named operand handles
+ (8 if self.out is not None else 0) # out handle + (8 if self.out is not None else 0) # out handle
+ 1 + sum(4 for _ in self.extra.values()) # extra scalars + 1 + sum(_extra_bytes(v) for v in self.extra.values())
) )
def _extra_bytes(v) -> int:
"""OpSpec.extra 값의 타입-aware byte 카운트."""
if isinstance(v, bool): return 1
if isinstance(v, (int, float)): return 4
if isinstance(v, (tuple, list)): return 1 + 4 * len(v) # shape, axes, …
if isinstance(v, str): return 1 # opcode-like tag
return 4 # default scalar
# extra 의 예시 타입:
# m, k, n int → 4 each
# reduce_axis int → 4
# shape=(64, 64) tuple → 1 + 8 = 9
# factor=1.0 float → 4
``` ```
(`DmaReadCmd`, `MathCmd` 등도 같은 룰 — dataclass 당 property 1 개, (`DmaReadCmd`, `MathCmd` 등도 같은 룰 — dataclass 당 property 1 개,
약 3 줄.) 약 3 줄.)
### D3. Default — 일반 composite ≈ 45 ns 기준 ### D3. Default — 일반 composite ≈ 43 ns 기준
anchor: 일반 1-op DMA→GEMM→DMA composite Anchor: **DMA-staged GEMM 경로용 단일-OpSpec composite**
`logical_bytes ≈ 52` (framing 4 + GEMM OpSpec 39 + rw_handles 9). dispatch `OpSpec(kind="gemm", ...)` 1 개; DMA stage 는 PE_SCHEDULER 가 operand
목표 45 ns. on-die producer→consumer 큐 4 bytes/cycle 가정. `space` 에서 자동 삽입 (ADR-0065 D4), **`logical_bytes` 에 등장하지 않음**
(커널이 별도 cmd 로 emit 안 함). 분해:
``` ```
clock = 1 GHz # 1 cycle = 1 ns framing 4
FIXED_PER_CMD = 32 cycles ops tuple length 1
R = 0.25 cycles/byte GEMM OpSpec 40 (opcode 1 + scope 1 + 1 + 2 handles 16
+ out 8 + 1 + extra m/k/n 12)
rw_handles tuple length 1
rw_handles content 8 (출력 RW 핸들 1 개)
─────────────────────────────
total ~54 bytes
``` ```
검증: `32 + 52 × 0.25 = 45 cycles ≈ 45 ns` Dispatch 목표 ≈43 ns. on-die producer→consumer 큐 16 bytes/cycle
(일반 on-die descriptor queue 폭) 가정.
```
FIXED_PER_CMD = 40 cycles
R = 0.0625 cycles/byte (= 16 bytes/cycle)
```
검증: `40 + 54 × 0.0625 = 43.375 cycles ≈ 43 ns`
Cycle→ns 변환은 PE 노드의 기존 `clock_freq_ghz` attr (PE_MATH `_compute_ns`
가 쓰는 같은 것) 사용. cost-model knob 은 **cycle-domain 만** — clock
설정 중복 안 함.
### D4. topology config override ### D4. topology config override
default 는 `pe_cpu.py` 에 내장. topology yaml 의 PE 노드 attrs 아래 default 는 `pe_cpu.py` 에 내장. topology yaml 의 PE 노드 attrs 아래
`pe_cost_model:` 절로 override: `pe_cost_model:` 절로 override (cycle-domain knob 만; clock 은 PE 의
기존 `clock_freq_ghz` 사용):
```yaml ```yaml
pe: pe:
attrs: attrs:
clock_freq_ghz: 1.0 # 기존, cycle→ns 변환용
pe_cost_model: pe_cost_model:
fixed_per_cmd_cycles: 32 fixed_per_cmd_cycles: 40
byte_cycles_recip: 0.25 byte_cycles_recip: 0.0625 # = 16 bytes/cycle
clock_freq_ghz: 1.0 max_composite_logical_bytes: 1024 # D7 — descriptor 크기 cap
``` ```
누락된 키는 default. dispatch 공식은 PE_CPU init 시 누락된 키는 default. dispatch 공식은 PE_CPU init 시
@@ -160,6 +208,31 @@ issue cost 를 0 → non-zero 로 바꾸면 **모든** bench 의 latency 변화.
이 ADR land 시 골든 latency **한 번** 재생성 — ADR-0062 D3 lazy-load 와 이 ADR land 시 골든 latency **한 번** 재생성 — ADR-0062 D3 lazy-load 와
같은 패턴. 재생성 후 동일 calibration 이 ADR-0065 opt2 측정에 적용. 같은 패턴. 재생성 후 동일 calibration 이 ADR-0065 opt2 측정에 적용.
### D7. Composite 크기 cap (deterministic segmentation)
`CompositeCmd``logical_bytes`**`MAX_COMPOSITE_LOGICAL_BYTES`**
(default **1024 bytes**, D4 로 override) 로 제한. 초과하는 cmd 는
*emitter* (host-side TLContext, ADR-0065 D5) 가 N 개 연속 `CompositeCmd`
로 deterministic 하게 **segment** — 각 ≤ cap.
- 각 segment 가 자기 `completion: CompletionHandle` 보유.
- 각 segment 가 자기 dispatch cost — `total = sum(FIXED + bytes_i × R) =
N × FIXED + total_bytes × R`. FIXED 항이 segment 당.
- segment 가 `rw_handles` 공유 — **strict FIFO** (ADR-0065 D6.3) 가
write-after-write 순서 자동 보존.
- segmentation 알고리즘 결정적 (emit 순서로 op 인덱스 greedy); host
emitter 의 일부, PE_SCHEDULER 가 아님.
이게 필요한 이유. 실제 하드웨어는 descriptor queue entry 크기, scheduler
parser buffer, command SRAM, firmware 입력의 하드 제한 보유. cap 없으면
`FIXED + bytes × R` 모델이 하드웨어가 받지 못할 정도로 큰 fused composite
를 보상 (예: primitive 100 개를 composite 1 개로 fuse, FIXED 1 회만 지불).
~30 bytes 짜리 op 100 개의 composite = ~3000 bytes > 1024 cap → 3 개로
segment → 3 × FIXED, 정직한 회계 회복.
Decode opt2 의 `#2` composite (10 ops, ~310 bytes) 는 1024 cap 안에
편안히 — GQA workload 에 segmentation 없음.
## Alternatives ## Alternatives
### A1. Revision 1 의 op-type calibration 표 유지 ### A1. Revision 1 의 op-type calibration 표 유지
@@ -203,23 +276,30 @@ fold. calibration 이 분리 필요성을 보이면 PE_DMA fixed setup 으로
## Open review items ## Open review items
1. **FIXED 와 R 의 calibration 출처.** default 는 "일반 composite = 45 ns 1. **FIXED 와 R 의 calibration 출처.** default 는 "일반 composite 43 ns
+ on-die 큐 4 bytes/cycle"; on-die producer→consumer 큐로 합리. + on-die 큐 16 bytes/cycle"; on-die descriptor 큐로 합리.
HW reference 등장 시 재방문. HW reference 등장 시 재방문.
2. **Scheduler plan-gen 비용.** 0 유지 — D5 가 PE_SCHEDULER 의 plan 2. **대형 composite 에서 Scheduler plan-gen 비용.** 0 유지 — D5 가
생성을 dispatch 공식 밖에 둠. scheduler-bound 행동이 보이면 기존 PE_SCHEDULER 의 plan 생성을 dispatch 공식 밖에 둠. D7 cap (1024 bytes
`overhead_ns` 로 노출. ≈ 3035 OpSpec) 이 최악을 묶지만, cap 근처 composite 도 현재
zero-cost 모델에서 1-op composite 와 같은 scheduler 비용. stress test
(대형 composite microbench) 가 scheduler-bound 행동을 보이면
per-op-count `overhead_ns` 노출.
3. **Override 위치.** topology yaml 의 PE 노드 attrs 아래 `pe_cost_model:` 3. **Override 위치.** topology yaml 의 PE 노드 attrs 아래 `pe_cost_model:`
block — 모든 knob 을 한 곳에, 리뷰 가능. block — 모든 knob 을 한 곳에, 리뷰 가능. clock 은 PE 의 기존
`clock_freq_ghz` 에서 — 여기 중복 안 함.
4. **경로 parity.** greenlet (`_execute_legacy`, `kernel_runner`) 와 4. **경로 parity.** greenlet (`_execute_legacy`, `kernel_runner`) 와
replay 둘 다 같은 cost model 읽어야 함. 검증. replay 둘 다 같은 cost model 읽어야 함. 검증.
5. **R 에 대한 결론의 민감도.** opt2 < opt3 가 합리적 `R` 범위 (큐 대역폭
가정) 에서 유지되어야 함. 민감도 sweep 이 Test Requirements (#9) 의
일부.
## Test Requirements ## Test Requirements
1. **Anchor 보존.** 일반 DMA→GEMM→DMA composite (1 op, 1. **Anchor 보존.** DMA-staged GEMM 경로용 단일-OpSpec composite
`logical_bytes ≈ 52`) 가 default 값에서 45 ns 에 dispatch. (`logical_bytes ≈ 54`) 가 default 값에서 ≈43 ns 에 dispatch (±2 ns).
2. **구조적 ratio.** opt3 vs opt2 dispatch (ADR-0065 §verification 의): 2. **구조적 ratio.** opt3 vs opt2 dispatch (ADR-0065 §verification 의):
default calibration 에서 `opt3 / opt2 ≈ 2.4×`. default calibration 에서 `opt3 / opt2 ≈ 4.0×`.
3. **Override 경로.** topology yaml 의 `pe_cost_model:` block 이 per-PE 3. **Override 경로.** topology yaml 의 `pe_cost_model:` block 이 per-PE
dispatch cost 변경; block 누락 시 default 복귀. dispatch cost 변경; block 누락 시 default 복귀.
4. **`PeCpuOverheadCmd` 우회.** 수동 `tl.cycles(n)` 는 정확히 `n` cycles, 4. **`PeCpuOverheadCmd` 우회.** 수동 `tl.cycles(n)` 는 정확히 `n` cycles,
@@ -229,13 +309,28 @@ fold. calibration 이 분리 필요성을 보이면 PE_DMA fixed setup 으로
6. **결정성.** 동일 입력 → 동일 op_log + latency (SPEC §0.1). 6. **결정성.** 동일 입력 → 동일 op_log + latency (SPEC §0.1).
7. **경로 parity.** greenlet 과 replay 가 동일 커널에 대해 동일 7. **경로 parity.** greenlet 과 replay 가 동일 커널에 대해 동일
dispatch-cycle 회계. dispatch-cycle 회계.
8. **Composite 크기 cap (D7).** `logical_bytes >
MAX_COMPOSITE_LOGICAL_BYTES` 가 될 recipe 가 N 개 연속 `CompositeCmd`
로 segment; 총 dispatch = segment dispatch 의 합; segment 간 RW 순서
strict FIFO 로 보존.
9. **민감도 sweep.** `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte
(= 4 / 16 / 32 bytes/cycle) 에서 *opt2 per-tile dispatch
< opt3 per-tile dispatch* 결론 유지 (ratio 가 R 감소 시 단조 증가
— FIXED 가 더 지배).
## Migration ## Migration
ADR-0064 Revision 2 는 단일 PR 로 land: ADR-0064 Revision 2 는 단일 PR 로 land:
-`PeCommand` dataclass 의 `logical_bytes` property - 각 `PeCommand` dataclass 의 `logical_bytes` property (D2 의 타입-aware
extra 카운트)
- `pe_cpu.py` dispatch 경로의 공식 적용 - `pe_cpu.py` dispatch 경로의 공식 적용
- PE_CPU init 의 `pe_cost_model:` override read - PE_CPU init 의 `pe_cost_model:` override read (cycle-domain knob +
`max_composite_logical_bytes`)
- **composite 크기 cap (D7)** — TLContext-side segmentation 로직,
`MAX_COMPOSITE_LOGICAL_BYTES` default 1024; 기존 bench 에 필요 없음
(현재 최대 composite 가 200 bytes 훨씬 아래), 하지만 ADR-0065 의 10-op
decode-opt2 composite (~310 bytes) + 더 큰 recipe 가 나타날 때를 위해
메커니즘 land.
- 일회성 골든 재생성 - 일회성 골든 재생성
land 후 ADR-0065 가 위에 쌓임 — 기존 bench 추가 골든 churn 없음 land 후 ADR-0065 가 위에 쌓임 — 기존 bench 추가 골든 churn 없음
@@ -56,6 +56,20 @@ N OpSpecs has ~N× the bytes of a primitive op with one OpSpec. The
*fixed* part (queue head update, completion register, MMIO-class *fixed* part (queue head update, completion register, MMIO-class
latency) is per-command. The two together compose: `FIXED + bytes × R`. latency) is per-command. The two together compose: `FIXED + bytes × R`.
### What this model actually exposes
The **primary** signal is **command-count reduction** through the
per-command FIXED cost. The **byte** term is a secondary refinement
that prevents pathologically-large composites from looking free. With
realistic on-die queue bandwidth (16 B/cycle, D3), FIXED accounts for
≥85% of the dispatch cost differential between opt3 (≈10 cmds/tile)
and opt2 (≈2 cmds/tile) for decode opt2.
This framing also bounds the model: if a single composite were allowed
to grow unboundedly large, the byte term alone would not stop the
formula from rewarding ever-bigger fused commands beyond what real HW
supports — hence the descriptor-size cap in **D7**.
## Decision ## Decision
### D1. Structural dispatch cost formula ### D1. Structural dispatch cost formula
@@ -113,39 +127,76 @@ def logical_bytes(self) -> int:
1 + 1 # opcode + scope 1 + 1 # opcode + scope
+ 1 + 8 * len(self.operands) # named operand handles + 1 + 8 * len(self.operands) # named operand handles
+ (8 if self.out is not None else 0) # out handle + (8 if self.out is not None else 0) # out handle
+ 1 + sum(4 for _ in self.extra.values()) # extra scalars + 1 + sum(_extra_bytes(v) for v in self.extra.values())
) )
def _extra_bytes(v) -> int:
"""Type-aware byte count for OpSpec.extra values."""
if isinstance(v, bool): return 1
if isinstance(v, (int, float)): return 4
if isinstance(v, (tuple, list)): return 1 + 4 * len(v) # shape, axes, …
if isinstance(v, str): return 1 # opcode-like tag
return 4 # default scalar
# Example types in extra:
# m, k, n int → 4 each
# reduce_axis int → 4
# shape=(64, 64) tuple → 1 + 8 = 9
# factor=1.0 float → 4
``` ```
(Identical rule for `DmaReadCmd`, `MathCmd`, etc. — one property per (Identical rule for `DmaReadCmd`, `MathCmd`, etc. — one property per
dataclass, ~3 lines each.) dataclass, ~3 lines each.)
### D3. Defaults — anchored on a typical composite ≈ 45 ns ### D3. Defaults — anchored on a typical composite ≈ 43 ns
Anchor: a typical 1-op DMA→GEMM→DMA composite has `logical_bytes ≈ 52` Anchor: a **single-OpSpec composite for a DMA-staged GEMM path**
(framing 4 + GEMM OpSpec 39 + rw_handles 9). Target dispatch = 45 ns. one `OpSpec(kind="gemm", ...)`; DMA stages are auto-inserted by
On-die producer→consumer queue assumed at 4 bytes/cycle. PE_SCHEDULER from operand `space` (ADR-0065 D4) and **do not appear in
`logical_bytes`** (the kernel does not issue them as separate cmds).
Breakdown:
``` ```
clock = 1 GHz # 1 cycle = 1 ns framing 4
FIXED_PER_CMD = 32 cycles ops tuple length 1
R = 0.25 cycles/byte GEMM OpSpec 40 (opcode 1 + scope 1 + 1 + 2 handles 16
+ out 8 + 1 + extra m/k/n 12)
rw_handles tuple length 1
rw_handles content 8 (one RW handle for the output)
─────────────────────────────
total ~54 bytes
``` ```
Verification: `32 + 52 × 0.25 = 45 cycles ≈ 45 ns` Target dispatch = ~43 ns. On-die producer→consumer queue at 16 bytes/cycle
(typical on-die descriptor queue width).
```
FIXED_PER_CMD = 40 cycles
R = 0.0625 cycles/byte (= 16 bytes/cycle)
```
Verification: `40 + 54 × 0.0625 = 43.375 cycles ≈ 43 ns`
Cycle→ns conversion uses the PE node's existing `clock_freq_ghz` attr
(the same one used by PE_MATH `_compute_ns`). The cost-model knobs are
**cycle-domain only** — they do not duplicate the clock setting.
### D4. Topology config override ### D4. Topology config override
Defaults are baked into `pe_cpu.py`. Topology yaml may override under a Defaults are baked into `pe_cpu.py`. Topology yaml may override under a
`pe_cost_model:` section at the PE node attrs: `pe_cost_model:` section at the PE node attrs (cycle-domain knobs only;
clock comes from the PE's existing `clock_freq_ghz`):
```yaml ```yaml
pe: pe:
attrs: attrs:
clock_freq_ghz: 1.0 # existing, used for cycle→ns
pe_cost_model: pe_cost_model:
fixed_per_cmd_cycles: 32 fixed_per_cmd_cycles: 40
byte_cycles_recip: 0.25 byte_cycles_recip: 0.0625 # = 16 bytes/cycle
clock_freq_ghz: 1.0 max_composite_logical_bytes: 1024 # D7 — descriptor size cap
``` ```
Missing keys fall back to defaults. The dispatch formula reads from Missing keys fall back to defaults. The dispatch formula reads from
@@ -170,6 +221,34 @@ latencies are **regenerated once** when this ADR lands — same posture as
ADR-0062 D3 lazy-load. After regeneration, the same calibration is in ADR-0062 D3 lazy-load. After regeneration, the same calibration is in
effect for ADR-0065 opt2 measurement. effect for ADR-0065 opt2 measurement.
### D7. Composite size cap (deterministic segmentation)
Each `CompositeCmd`'s `logical_bytes` is capped at
**`MAX_COMPOSITE_LOGICAL_BYTES`** (default **1024 bytes**, overridable
per D4). Oversized commands are deterministically **segmented** by the
*emitter* (host-side TLContext, ADR-0065 D5) into N consecutive
`CompositeCmd`s, each ≤ cap.
- Each segment carries its own `completion: CompletionHandle`.
- Each segment incurs its own dispatch cost — `total =
sum(FIXED + bytes_i × R) = N × FIXED + total_bytes × R`. The FIXED
term is paid per segment.
- Segments share `rw_handles` where applicable; **strict FIFO**
(ADR-0065 D6.3) preserves write-after-write ordering automatically.
- The segmentation algorithm is deterministic (greedy by op index in
emit order); it is part of the host emitter, not PE_SCHEDULER.
Why this is needed. Real hardware imposes hard limits — descriptor
queue entry size, scheduler parser buffer, command SRAM, firmware
input. Without a cap, the `FIXED + bytes × R` model would reward
arbitrarily large fused composites beyond what hardware accepts (e.g.,
fusing 100 primitive ops into one composite, paying one `FIXED`).
A composite of (say) 100 ops at ~30 bytes each = ~3000 bytes >
1024 cap → segmented into 3 ≈ 3 × FIXED, restoring honest accounting.
Decode opt2's `#2` composite (10 ops, ~310 bytes) sits comfortably
inside the 1024 cap — no segmentation for the GQA workload.
## Alternatives ## Alternatives
### A1. Keep Revision 1's op-type calibration table ### A1. Keep Revision 1's op-type calibration table
@@ -218,22 +297,32 @@ calibration shows it matters.
## Open review items ## Open review items
1. **Calibration source for FIXED and R.** Defaults from "typical 1. **Calibration source for FIXED and R.** Defaults from "typical
composite = 45 ns + on-die queue 4 bytes/cycle"; reasonable for an composite 43 ns + on-die queue 16 bytes/cycle"; reasonable for an
on-die producer→consumer queue. Revisit when a HW reference appears. on-die descriptor queue. Revisit when a HW reference appears.
2. **Scheduler plan-gen cost.** Stays 0 — D5 keeps PE_SCHEDULER's 2. **Scheduler plan-gen cost vs large composites.** Stays 0 — D5 keeps
plan-generation outside the dispatch formula. Expose via existing PE_SCHEDULER's plan-generation outside the dispatch formula. The
`overhead_ns` if a workload shows scheduler-bound behaviour. D7 cap (1024 bytes ≈ 3035 OpSpecs) bounds the worst case, but a
composite near the cap still costs the scheduler the same as a 1-op
composite under the current zero-cost model. If a stress test
(large-composite microbench) shows scheduler-bound behaviour,
expose `overhead_ns` per-op-count.
3. **Where the override lives.** `pe_cost_model:` block under PE node 3. **Where the override lives.** `pe_cost_model:` block under PE node
attrs in topology yaml — keeps all knobs in one place, reviewable. attrs in topology yaml — keeps all knobs in one place, reviewable.
Clock comes from the PE's existing `clock_freq_ghz` attr, not
duplicated here.
4. **Path parity.** Both greenlet (`_execute_legacy` and `kernel_runner`) 4. **Path parity.** Both greenlet (`_execute_legacy` and `kernel_runner`)
and replay paths must read the same cost model. Verify. and replay paths must read the same cost model. Verify.
5. **Sensitivity of conclusions to R.** opt2 < opt3 must hold across a
reasonable range of `R` (queue-bandwidth assumption). Sensitivity
sweep is part of Test Requirements (#9).
## Test Requirements ## Test Requirements
1. **Anchor preservation.** A typical DMA→GEMM→DMA composite (1 op, 1. **Anchor preservation.** A single-OpSpec composite for a DMA-staged
`logical_bytes ≈ 52`) dispatches in 45 ns at the default values. GEMM path (`logical_bytes ≈ 54`) dispatches in ≈43 ns at default
values (within ±2 ns).
2. **Structural ratio.** opt3 vs opt2 dispatch (per ADR-0065 §verification): 2. **Structural ratio.** opt3 vs opt2 dispatch (per ADR-0065 §verification):
`opt3 / opt2 ≈ 2.4×` at default calibration. `opt3 / opt2 ≈ 4.0×` at default calibration.
3. **Override path.** Topology yaml `pe_cost_model:` block changes the 3. **Override path.** Topology yaml `pe_cost_model:` block changes the
per-PE dispatch cost; default is recovered when block is missing. per-PE dispatch cost; default is recovered when block is missing.
4. **`PeCpuOverheadCmd` bypass.** Manual `tl.cycles(n)` issues exactly `n` 4. **`PeCpuOverheadCmd` bypass.** Manual `tl.cycles(n)` issues exactly `n`
@@ -244,13 +333,28 @@ calibration shows it matters.
(SPEC §0.1). (SPEC §0.1).
7. **Path parity.** Greenlet and replay paths produce identical 7. **Path parity.** Greenlet and replay paths produce identical
dispatch-cycle accounting for the same kernel. dispatch-cycle accounting for the same kernel.
8. **Composite size cap (D7).** A recipe that would emit `logical_bytes
> MAX_COMPOSITE_LOGICAL_BYTES` is segmented into N consecutive
`CompositeCmd`s; total dispatch = sum of segment dispatches; RW
ordering across segments preserved by strict FIFO.
9. **Sensitivity sweep.** At `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte
(= 4 / 16 / 32 bytes/cycle), the conclusion *opt2 per-tile dispatch
< opt3 per-tile dispatch* must hold (ratio monotonically increases
as R decreases — FIXED dominates more).
## Migration ## Migration
ADR-0064 Revision 2 lands as a single PR with: ADR-0064 Revision 2 lands as a single PR with:
- `logical_bytes` property on each `PeCommand` dataclass - `logical_bytes` property on each `PeCommand` dataclass (type-aware
extra-field counting per D2)
- formula application in `pe_cpu.py` dispatch path - formula application in `pe_cpu.py` dispatch path
- `pe_cost_model:` override read at PE_CPU init - `pe_cost_model:` override read at PE_CPU init (cycle-domain knobs +
`max_composite_logical_bytes`)
- **composite size cap (D7)** — TLContext-side segmentation logic with
`MAX_COMPOSITE_LOGICAL_BYTES` default 1024; not needed for any
existing bench (largest current composite is well under 200 bytes),
but the mechanism lands so it is in place when ADR-0065's
10-op decode-opt2 composite (~310 bytes) and larger recipes appear.
- one-time goldens regeneration - one-time goldens regeneration
After this lands, ADR-0065 builds on top with no further goldens churn After this lands, ADR-0065 builds on top with no further goldens churn
@@ -305,9 +305,15 @@ scratch 에서 읽는 형태 — 순환 tile-loop 의존성. **기각 (incorrect
6. **opt2 가 opt3 와 수치 동등.** data mode 에서 opt2 의 최종 6. **opt2 가 opt3 와 수치 동등.** data mode 에서 opt2 의 최종
`(m, l, O)` 가 opt3 와 fp tolerance 안. `(m, l, O)` 가 opt3 와 fp tolerance 안.
7. **opt2 dispatch ratio (ADR-0064 Rev2 이후).** opt3 vs opt2 per-tile 7. **opt2 dispatch ratio (ADR-0064 Rev2 이후).** opt3 vs opt2 per-tile
PE_CPU dispatch cycles ratio 가 default calibration 에서 ≈ 2.4×. PE_CPU dispatch cycles ratio 가 default calibration (FIXED=40 cycles,
R=0.0625 cycles/byte) 에서 ≈ 4.0×. Ratio 가 FIXED-dominated —
command-count 감소가 1차 신호임을 반영.
8. **GEMM-count invariant.** GEMM OpSpec 두 개 가진 composite 가 8. **GEMM-count invariant.** GEMM OpSpec 두 개 가진 composite 가
TLContext emit 시 validation error. TLContext emit 시 validation error.
9. **Composite 크기 cap 안에 (ADR-0064 D7).** Decode opt2 의 `#2`
composite (10 ops, ~310 logical bytes) 가 default
`MAX_COMPOSITE_LOGICAL_BYTES=1024` 안에 편안히 — segmentation 없음.
recipe lowering 테스트가 `cmd.logical_bytes < 1024` 단언.
## Dependencies ## Dependencies
@@ -338,9 +338,15 @@ input from #1's epilogue scratch — circular tile-loop dependency.
6. **opt2 numeric parity with opt3.** In data mode, opt2's final 6. **opt2 numeric parity with opt3.** In data mode, opt2's final
`(m, l, O)` matches opt3's within fp tolerance. `(m, l, O)` matches opt3's within fp tolerance.
7. **opt2 dispatch ratio (after ADR-0064 Rev2).** opt3 vs opt2 per-tile 7. **opt2 dispatch ratio (after ADR-0064 Rev2).** opt3 vs opt2 per-tile
PE_CPU dispatch cycles ratio ≈ 2.4× at default calibration. PE_CPU dispatch cycles ratio ≈ 4.0× at default calibration
(FIXED=40 cycles, R=0.0625 cycles/byte). Ratio is FIXED-dominated,
reflecting command-count reduction as the primary signal.
8. **GEMM-count invariant.** A composite with two GEMM OpSpecs raises a 8. **GEMM-count invariant.** A composite with two GEMM OpSpecs raises a
validation error at TLContext emit. validation error at TLContext emit.
9. **Within composite size cap (ADR-0064 D7).** Decode opt2's `#2`
composite (10 ops, ~310 logical bytes) fits comfortably under the
default `MAX_COMPOSITE_LOGICAL_BYTES=1024` — no segmentation. The
recipe lowering test asserts `cmd.logical_bytes < 1024`.
## Dependencies ## Dependencies
@@ -27,7 +27,8 @@ unchanged.
PE_MATH/PE_GEMM/PE_DMA see only `Stage.params["op_kind"]` (no new PE_MATH/PE_GEMM/PE_DMA see only `Stage.params["op_kind"]` (no new
engine code). engine code).
4. **Dispatch ratio.** Per-tile PE_CPU dispatch cycles 4. **Dispatch ratio.** Per-tile PE_CPU dispatch cycles
`ratio(opt3 / opt2) ≈ 2.4×` at default ADR-0064 calibration. `ratio(opt3 / opt2) ≈ 4.0×` at default ADR-0064 Rev2 calibration
(FIXED=40 cycles, R=0.0625 cycles/byte).
5. **K-before-V invariant.** op_log of decode opt2 shows zero V-related 5. **K-before-V invariant.** op_log of decode opt2 shows zero V-related
DMA during #2's MATH prologue. DMA during #2's MATH prologue.
@@ -267,7 +268,12 @@ Lowering steps in `TLContext.composite()`:
7. **Compute `rw_handles`.** Collect all handles tagged RW in prologue 7. **Compute `rw_handles`.** Collect all handles tagged RW in prologue
recipes + the head GEMM's `out` (if accumulating) + any RW epilogue recipes + the head GEMM's `out` (if accumulating) + any RW epilogue
targets. For softmax_merge: `rw_handles = (m, l, O)`. targets. For softmax_merge: `rw_handles = (m, l, O)`.
8. **Emit `CompositeCmd`.** `ops = (8 MATH + 1 GEMM + 1 MATH(add))`, 8. **Check composite size cap (ADR-0064 D7).** Compute
`cmd.logical_bytes`; if > `MAX_COMPOSITE_LOGICAL_BYTES` (default
1024), segment deterministically by op index. softmax_merge's 10-op
composite (~310 bytes) is well under the cap → single segment, no
split for decode opt2.
9. **Emit `CompositeCmd`.** `ops = (8 MATH + 1 GEMM + 1 MATH(add))`,
`rw_handles = (m, l, O)`, `completion = new`. `rw_handles = (m, l, O)`, `completion = new`.
**Legacy lowering.** Pre-existing **Legacy lowering.** Pre-existing
@@ -392,7 +398,7 @@ Each phase is an independent Phase-1 → Phase-2 cycle per CLAUDE.md Part 1.
| **P3** | PE_SCHEDULER `_generate_plan` flat-ops + `PipelinePlan.prologue_stages` + DMA auto-insertion + `_feed_loop` extension | all existing bench Stage sequences unchanged; new MATH chain visible in op_log for opt2 mock | | **P3** | PE_SCHEDULER `_generate_plan` flat-ops + `PipelinePlan.prologue_stages` + DMA auto-insertion + `_feed_loop` extension | all existing bench Stage sequences unchanged; new MATH chain visible in op_log for opt2 mock |
| **P4** | Strict-FIFO RW hazard tracker + `_RwHazardTracker` integration | two-composite RW conflict test serializes correctly | | **P4** | Strict-FIFO RW hazard tracker + `_RwHazardTracker` integration | two-composite RW conflict test serializes correctly |
| **P5** | `_gqa_decode_long.py` opt2 variant; data-mode numeric parity check; K-before-V invariant check | opt2 matches opt3 within fp tolerance; no V DMA during prologue | | **P5** | `_gqa_decode_long.py` opt2 variant; data-mode numeric parity check; K-before-V invariant check | opt2 matches opt3 within fp tolerance; no V DMA during prologue |
| **P6** | Dispatch-ratio measurement: opt3 vs opt2 per-tile PE_CPU cycles | ratio ≈ 2.4× ± 10% at default calibration | | **P6** | Dispatch-ratio measurement: opt3 vs opt2 per-tile PE_CPU cycles + R sensitivity sweep | ratio ≈ 4.0× ± 15% at default calibration; opt2 < opt3 at all `R ∈ {0.25, 0.0625, 0.03125}` (ADR-0064 Test #9) |
P0 must land first (separate ADR). P1 is pure refactor (safest). P2P4 P0 must land first (separate ADR). P1 is pure refactor (safest). P2P4
build the new path without touching opt3. P5 enables opt2. P6 closes build the new path without touching opt3. P5 enables opt2. P6 closes
@@ -443,7 +449,9 @@ Mirrors ADR-0065 §Test Requirements; grounded in SPEC R2/R5, ADR-0023/
**Dispatch ratio (P6):** **Dispatch ratio (P6):**
- For `S_kv=64, n_tiles=16`, measure total PE_CPU dispatch cycles for - For `S_kv=64, n_tiles=16`, measure total PE_CPU dispatch cycles for
opt3 vs opt2 paths. opt3 vs opt2 paths.
- Assert ratio in `[2.0, 2.8]` (centre 2.4×). - Assert ratio in `[3.4, 4.6]` (centre 4.0× at ADR-0064 Rev2 defaults).
- **Sensitivity sweep.** Repeat with `R ∈ {0.25, 0.0625, 0.03125}`
cycles/byte; assert opt2 < opt3 in all three. (ADR-0064 Test #9.)
**Determinism (all phases):** **Determinism (all phases):**
- Identical inputs → identical op_log + latency (SPEC §0.1). - Identical inputs → identical op_log + latency (SPEC §0.1).
@@ -452,19 +460,37 @@ Mirrors ADR-0065 §Test Requirements; grounded in SPEC R2/R5, ADR-0023/
## 11. Performance model (expected) ## 11. Performance model (expected)
Per-tile PE_CPU dispatch cycles, ADR-0064 Rev2 default calibration: Per-tile PE_CPU dispatch cycles, ADR-0064 Rev2 default calibration
(FIXED=40 cycles, R=0.0625 cycles/byte, 1 cycle = 1 ns at 1 GHz):
``` ```
opt3: 10 cmds × 32 + total_bytes(≈232) × 0.25 = 320 + 58 ≈ 378 ns opt3: 10 cmds × 40 + total_bytes(≈232) × 0.0625 = 400 + 14.5 ≈ 414.5 ns
opt2: 2 cmds × 32 + total_bytes(≈380) × 0.25 = 64 + 95 ≈ 159 ns opt2: 2 cmds × 40 + total_bytes(≈380) × 0.0625 = 80 + 23.75 ≈ 103.75 ns
ratio = 378 / 159 ≈ 2.4× ratio = 414.5 / 103.75 ≈ 4.0×
``` ```
The FIXED term dominates (~96% of the differential), reflecting
ADR-0064's framing: the primary signal is command-count reduction.
The byte term is a secondary refinement (visible: opt2 carries more
total bytes than opt3, so its byte cost is higher; this is more than
offset by 8× fewer FIXED payments).
Per-tile engine timing dominates total latency; dispatch is one Per-tile engine timing dominates total latency; dispatch is one
component. The ratio above is the *dispatch-only* relative cost. Total component. The ratio above is the *dispatch-only* relative cost. Total
latency improvement depends on whether dispatch was on the critical latency improvement depends on whether dispatch was on the critical
path — measured per workload in P6. path — measured per workload in P6.
**R sensitivity** (ADR-0064 Test #9 — sanity at neighbouring R):
| R (cycles/byte) | opt3 ns | opt2 ns | ratio |
|---|---|---|---|
| 0.25 (4 B/cy) | 458 | 175 | 2.6× |
| 0.0625 (16 B/cy) ✓| 414.5 | 103.75 | 4.0× |
| 0.03125 (32 B/cy) | 407.25 | 91.875 | 4.4× |
`opt2 < opt3` at all three; ratio is monotonic in `1/R` (more
FIXED-dominated as R decreases).
--- ---
## 12. Open items (status) ## 12. Open items (status)
@@ -476,8 +502,9 @@ already documented in §§58. Live remaining items:
|---|---| |---|---|
| Non-identity `PrimaryOutSpec.transform` (e.g., `"trans"`) | Future recipe addition; current code uses string dispatch in step 3 of §6 | | Non-identity `PrimaryOutSpec.transform` (e.g., `"trans"`) | Future recipe addition; current code uses string dispatch in step 3 of §6 |
| `_gqa_decode_long.py` opt2 — TILE size sweep | Open (existing §B-3, §9 of ADR-0060) | | `_gqa_decode_long.py` opt2 — TILE size sweep | Open (existing §B-3, §9 of ADR-0060) |
| Larger composite descriptors (10+ OpSpecs) and `logical_bytes` growth | Monitored in P6; if `R` term dominates, calibration revisit | | Larger composite descriptors approaching ADR-0064 D7 cap (1024 bytes) | Monitored in P6; segmentation kicks in deterministically if exceeded |
| `scratch_scope` budget vs softmax_merge intermediates (G·TILE for P) | ADR-0063 sizing check at S_kv = 256K | | `scratch_scope` budget vs softmax_merge intermediates (G·TILE for P) | ADR-0063 sizing check at S_kv = 256K |
| Scheduler plan-gen cost for large composites (ADR-0064 Open Item 2) | If P6 measurements show scheduler-bound behaviour near the cap, expose per-op-count `overhead_ns` on PE_SCHEDULER |
--- ---