diff --git a/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model-ko.md b/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model-ko.md index 6ce122d..eca6000 100644 --- a/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model-ko.md +++ b/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model-ko.md @@ -234,14 +234,31 @@ issue cost 를 0 → non-zero 로 바꾸면 **모든** bench 의 latency 변화. - segmentation 알고리즘 결정적 (emit 순서로 op 인덱스 greedy); host emitter 의 일부, PE_SCHEDULER 가 아님. -이게 필요한 이유. 실제 하드웨어는 descriptor queue entry 크기, scheduler -parser buffer, command SRAM, firmware 입력의 하드 제한 보유. cap 없으면 -`FIXED + bytes × R` 모델이 하드웨어가 받지 못할 정도로 큰 fused composite -를 보상 (예: primitive 100 개를 composite 1 개로 fuse, FIXED 1 회만 지불). -~30 bytes 짜리 op 100 개의 composite = ~3000 bytes > 1024 cap → 3 개로 -segment → 3 × FIXED, 정직한 회계 회복. +**cap 이 필요한 이유.** 실제 하드웨어는 descriptor queue entry 크기, +scheduler parser buffer, command SRAM, firmware 입력의 하드 제한 보유. +cap 없으면 `FIXED + bytes × R` 모델이 하드웨어가 받지 못할 정도로 큰 +fused composite 를 보상 (예: primitive 100 개를 composite 1 개로 fuse, +FIXED 1 회만 지불). -Decode opt2 의 `#2` composite (10 ops, ~310 bytes) 는 1024 cap 안에 +**1024 bytes 라는 *특정* 숫자의 근거.** 이는 측정된 HW 숫자가 아니라 +**safe engineering limit** — kernbench 코드베이스의 현재 모든 알려진 +composite (가장 큰 것이 decode opt2 의 `#2` ~322 bytes) 보다 훨씬 위이면서 +미래 recipe 도 지켜야 할 *유한* descriptor capacity 를 의도적으로 표현. +값은 topology 별 override (D4); 실제 HW reference 등장 시 재캘리브레이션. +이 default 의 역할은 cap 을 *원칙으로 존재시키는 것*, 특정 HW 에 맞추는 +것이 아님. + +**segment 간 순서 — strict FIFO 가 주체, `rw_handles` 아님.** segment 들은 +emit 순서로 PE_CPU → PE_SCHEDULER 큐에 들어감. strict-FIFO dispatch +(ADR-0065 D6.3) 가 *순서의 주체* — segment 가 큐 진입 순서로 실행. 각 +segment 의 `rw_handles` block 은 cross-composite 해저드 추적기의 **의존성 +메타데이터** — 그 자체로 inter-segment 순서를 보장하지 **않음**. 미래 +scheduler 가 FIFO 를 완화 (예: RW-aware reorder, ADR-0065 A4) 한다면 +segmenter 는 (a) segment 를 단일 CompositeCmd 로 merge 또는 +(b) 명시적 completion-handle 의존 체인 도입 필요. Phase 1 에선 무관 — +strict FIFO 가 유효. + +Decode opt2 의 `#2` composite (10 ops, ~322 bytes) 는 1024 cap 안에 편안히 — GQA workload 에 segmentation 없음. ## Alternatives @@ -307,12 +324,24 @@ fold. calibration 이 분리 필요성을 보이면 PE_DMA fixed setup 으로 ## Test Requirements -1. **Anchor 보존.** DMA-staged GEMM 경로용 단일-OpSpec composite - (`logical_bytes ≈ 54`) 가 default 값에서 ≈43 ns 에 dispatch (±2 ns). -2. **구조적 ratio.** opt3 vs opt2 dispatch (ADR-0065 §verification 의): - default calibration 에서 `opt3 / opt2 ≈ 4.0×`. +테스트는 specific 숫자 anchor 가 아닌 **공식 (D1)** 에 대해 작성 — +calibration 이 바뀌거나 OpSpec/CompositeCmd 필드가 추가되어도 유효. + +1. **공식 보존.** 임의 `CompositeCmd` `c` 에 대해 PE_CPU 의 기록된 + dispatch overhead 가 `FIXED_PER_CMD + c.logical_bytes × R` + (floor/round-off 의 ±1 cycle 내). 여러 composite parametrize: + 1-OpSpec GEMM composite, 5-op MATH chain, 10-op recipe composite. + Default-calibration 숫자 (1-OpSpec composite 의 anchor ≈43 ns) 는 + *informative* 참고, 테스트 gate 가 아님 — gate 는 공식 등가. +2. **정성적 ratio (robust).** opt3 per-tile PE_CPU dispatch 가 opt2 + per-tile dispatch 를 최소 2× 차이로 엄격히 초과 — `opt3 > 2 × opt2`. + Default-calibration 모델은 ≈4× 예측; gate 는 느슨한 2× 한계라 + calibration 이동 (예: HW reference 가 default 대체) 시 깨지지 않음. + Informative 숫자 — ADR-0065 §verification 와 DDD-0065 §11 의 모델 + 기대치 참조. 3. **Override 경로.** topology yaml 의 `pe_cost_model:` block 이 per-PE - dispatch cost 변경; block 누락 시 default 복귀. + dispatch cost 변경; block 누락 시 default 복귀. #1 의 공식 등가가 + override 값에서도 성립. 4. **`PeCpuOverheadCmd` 우회.** 수동 `tl.cycles(n)` 는 정확히 `n` cycles, `n + dispatch_cycles(...)` 아님. 5. **double-count 없음.** PE_DMA `drain_ns`, PE_GEMM/MATH `_compute_ns` 가 @@ -322,12 +351,12 @@ fold. calibration 이 분리 필요성을 보이면 PE_DMA fixed setup 으로 dispatch-cycle 회계. 8. **Composite 크기 cap (D7).** `logical_bytes > MAX_COMPOSITE_LOGICAL_BYTES` 가 될 recipe 가 N 개 연속 `CompositeCmd` - 로 segment; 총 dispatch = segment dispatch 의 합; segment 간 RW 순서 - strict FIFO 로 보존. -9. **민감도 sweep.** `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte - (= 4 / 16 / 32 bytes/cycle) 에서 *opt2 per-tile dispatch - < opt3 per-tile dispatch* 결론 유지 (ratio 가 R 감소 시 단조 증가 - — FIXED 가 더 지배). + 로 segment; `sum(segment.logical_bytes) == original logical_bytes`; + 총 dispatch = segment dispatch 의 합 (FIXED 가 segment 당 지불); + inter-segment 순서 strict FIFO 로 보존 (`rw_handles` 단독 아님). +9. **민감도 (정성적).** `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte 에서 + 3 지점 모두 `opt3 > opt2`. 방향 (R 감소 시 ratio 단조 증가) 도 단언, + 단 절대 ratio 값은 *불필요*. ## Migration diff --git a/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model.md b/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model.md index 28bfd01..47ed095 100644 --- a/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model.md +++ b/docs/adr-proposed/ADR-0064-perf-cpu-issue-cost-model.md @@ -251,15 +251,34 @@ per D4). Oversized commands are deterministically **segmented** by the - The segmentation algorithm is deterministic (greedy by op index in emit order); it is part of the host emitter, not PE_SCHEDULER. -Why this is needed. Real hardware imposes hard limits — descriptor +**Why a cap is needed.** Real hardware imposes hard limits — descriptor queue entry size, scheduler parser buffer, command SRAM, firmware input. Without a cap, the `FIXED + bytes × R` model would reward arbitrarily large fused composites beyond what hardware accepts (e.g., fusing 100 primitive ops into one composite, paying one `FIXED`). -A composite of (say) 100 ops at ~30 bytes each = ~3000 bytes > -1024 cap → segmented into 3 ≈ 3 × FIXED, restoring honest accounting. -Decode opt2's `#2` composite (10 ops, ~310 bytes) sits comfortably +**Why 1024 bytes specifically.** This is a **safe engineering limit**, +not a measured HW number — intentionally chosen to be well above all +currently known composites (decode opt2's `#2` at ~322 bytes is the +largest in the kernbench codebase) while still representing a *finite* +descriptor capacity that future recipes must respect. The number is +overridable per topology (D4); when a real HW reference appears, the +value should be recalibrated. The role of this default is to make the +cap *exist as a discipline*, not to fit a specific HW. + +**Ordering of segments — driven by strict FIFO, not `rw_handles`.** +Segments are emitted into the PE_CPU → PE_SCHEDULER queue in their +emit order. Strict-FIFO dispatch (ADR-0065 D6.3) is the *ordering +source*: segments execute in the order they enter the queue. The +`rw_handles` block on each segment is **dependency metadata** for the +cross-composite hazard tracker — it does **not** by itself guarantee +inter-segment ordering. If a future scheduler relaxed FIFO (e.g., to +RW-aware reorder, ADR-0065 A4), the segmenter would need to either +(a) merge segments into a single CompositeCmd, or (b) introduce an +explicit completion-handle dependency chain. For Phase 1 this does +not arise: strict FIFO is in effect. + +Decode opt2's `#2` composite (10 ops, ~322 bytes) sits comfortably inside the 1024 cap — no segmentation for the GQA workload. ## Alternatives @@ -331,13 +350,27 @@ calibration shows it matters. ## Test Requirements -1. **Anchor preservation.** A single-OpSpec composite for a DMA-staged - GEMM path (`logical_bytes ≈ 54`) dispatches in ≈43 ns at default - values (within ±2 ns). -2. **Structural ratio.** opt3 vs opt2 dispatch (per ADR-0065 §verification): - `opt3 / opt2 ≈ 4.0×` at default calibration. +Tests are written against the **formula** (D1), not against specific +numeric anchors, so they remain valid when calibration changes or when +OpSpec/CompositeCmd fields are added. + +1. **Formula preservation.** For any `CompositeCmd` `c`, PE_CPU's + recorded dispatch overhead equals `FIXED_PER_CMD + c.logical_bytes + × R` (within ±1 cycle for floor/round-off). Parametrized over + several composites: a 1-OpSpec GEMM composite, a 5-op MATH chain, + and a 10-op recipe composite. Default-calibration numbers (anchor + ≈43 ns for the 1-OpSpec composite) are informative reference, not + the test gate — the test gate is the formula equality. +2. **Qualitative ratio (robust).** opt3 per-tile PE_CPU dispatch + strictly exceeds opt2 per-tile dispatch by at least a 2× margin — + `opt3 > 2 × opt2`. The default-calibration model predicts ≈4×; + the gate is the loose 2× bound so the test does not break when + calibration is moved (e.g., when a HW reference replaces the + default). Informative numbers — see ADR-0065 §verification and + DDD-0065 §11 for the model expectation. 3. **Override path.** Topology yaml `pe_cost_model:` block changes the per-PE dispatch cost; default is recovered when block is missing. + The formula identity from #1 must hold with the override values. 4. **`PeCpuOverheadCmd` bypass.** Manual `tl.cycles(n)` issues exactly `n` cycles, not `n + dispatch_cycles(...)`. 5. **No double-count.** PE_DMA `drain_ns`, PE_GEMM/MATH `_compute_ns` @@ -348,12 +381,14 @@ calibration shows it matters. dispatch-cycle accounting for the same kernel. 8. **Composite size cap (D7).** A recipe that would emit `logical_bytes > MAX_COMPOSITE_LOGICAL_BYTES` is segmented into N consecutive - `CompositeCmd`s; total dispatch = sum of segment dispatches; RW - ordering across segments preserved by strict FIFO. -9. **Sensitivity sweep.** At `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte - (= 4 / 16 / 32 bytes/cycle), the conclusion *opt2 per-tile dispatch - < opt3 per-tile dispatch* must hold (ratio monotonically increases - as R decreases — FIXED dominates more). + `CompositeCmd`s; `sum(segment.logical_bytes) == original + logical_bytes`; total dispatch = sum of segment dispatches (FIXED + paid per segment); inter-segment ordering preserved by strict FIFO + (not by `rw_handles` alone). +9. **Sensitivity (qualitative).** At `R ∈ {0.25, 0.0625, 0.03125}` + cycles/byte, `opt3 > opt2` at *all* three points. Direction (ratio + monotonically increases as R decreases) is also asserted, but + absolute ratio values are *not* required. ## Migration diff --git a/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge-ko.md b/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge-ko.md index f67af79..3702146 100644 --- a/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge-ko.md +++ b/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge-ko.md @@ -179,7 +179,10 @@ op="gemm", ...)` 에서: 3. primary output P 의 shape 을 `s.shape` 에서 derive (identity). 4. `engine_seq` 를 8 개의 평평한 MATH OpSpec 으로 펼침 (모든 주소/크기 채움). 각 OpSpec 의 `scope = KERNEL`. -5. head GEMM 의 `operands["a"] = P_handle` auto-bind. +5. **Auto-bind (충돌 검사, D6.6).** head GEMM 이 `operands["a"]` 를 + *명시하지 않은* 경우에만 `operands["a"] = P_handle` auto-bind. + 커널이 `a` 를 명시적으로 제공했다면 validation error — 모호한 + primary-output 대체. 6. `CompositeCmd(ops=(8 MATH + 1 GEMM + epilogue), rw_handles=(m, l, O))` emit. **RECIPE_DESCRIPTORS 는 HW 경로 어디에도 안 나타남**. PE_SCHEDULER 는 @@ -331,9 +334,12 @@ scratch 에서 읽는 형태 — 순환 tile-loop 의존성. **기각 (incorrect emit 안 함. 6. **opt2 가 opt3 와 수치 동등.** data mode 에서 opt2 의 최종 `(m, l, O)` 가 opt3 와 fp tolerance 안. -7. **opt2 dispatch ratio (ADR-0064 Rev2 이후).** opt3 vs opt2 per-tile - PE_CPU dispatch cycles ratio 가 default calibration (FIXED=40 cycles, - R=0.0625 cycles/byte) 에서 ≈ 4.0×. Ratio 가 FIXED-dominated — +7. **opt2 dispatch ratio (ADR-0064 Rev2 이후) — robust.** opt3 vs opt2 + per-tile PE_CPU dispatch cycles 가 `opt3 > 2 × opt2` 만족 + (정성적 gate, calibration-독립). Default-calibration 모델 기대치는 + ≈ 4.0× (FIXED=40 cycles, R=0.0625 cycles/byte) — DDD-0065 §11 의 + 모델-유도 숫자 참조; 테스트 gate 는 느슨한 `> 2×` 한계라 + calibration 변경에도 살아남음. Ratio 가 FIXED-dominated — command-count 감소가 1차 신호임을 반영. 8. **GEMM-count invariant.** GEMM OpSpec 두 개 가진 composite 가 TLContext emit 시 validation error. diff --git a/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge.md b/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge.md index 2e87a5e..f26780c 100644 --- a/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge.md +++ b/docs/adr-proposed/ADR-0065-prog-flat-ops-composite-softmax-merge.md @@ -200,7 +200,11 @@ op="gemm", ...)`: 3. Derives the primary output P's shape from `s.shape` (identity). 4. Expands `engine_seq` into 8 flat MATH OpSpecs with all addresses / sizes filled. Each OpSpec gets `scope=KERNEL`. -5. Auto-binds head GEMM's `operands["a"] = P_handle`. +5. **Auto-binds (with conflict check, D6.6).** If the head GEMM does + *not* explicitly provide `operands["a"]`, auto-bind + `operands["a"] = P_handle`. If the kernel *did* provide `a` + explicitly, emit a validation error — ambiguous primary-output + replacement. 6. Emits `CompositeCmd(ops=(8 MATH + 1 GEMM + epilogue), rw_handles= (m, l, O))`. @@ -370,10 +374,14 @@ input from #1's epilogue scratch — circular tile-loop dependency. `space=hbm` emits DMA_WRITE; `space=tcm` does not. 6. **opt2 numeric parity with opt3.** In data mode, opt2's final `(m, l, O)` matches opt3's within fp tolerance. -7. **opt2 dispatch ratio (after ADR-0064 Rev2).** opt3 vs opt2 per-tile - PE_CPU dispatch cycles ratio ≈ 4.0× at default calibration - (FIXED=40 cycles, R=0.0625 cycles/byte). Ratio is FIXED-dominated, - reflecting command-count reduction as the primary signal. +7. **opt2 dispatch ratio (after ADR-0064 Rev2) — robust.** opt3 vs + opt2 per-tile PE_CPU dispatch cycles satisfies `opt3 > 2 × opt2` + (qualitative gate; calibration-independent). The default-calibration + model expectation is ≈ 4.0× (FIXED=40 cycles, R=0.0625 cycles/byte) + — see DDD-0065 §11 for the model-derived numbers; only the loose + `> 2×` bound is the test gate so it survives calibration changes. + Ratio is FIXED-dominated, reflecting command-count reduction as the + primary signal. 8. **GEMM-count invariant.** A composite with two GEMM OpSpecs raises a validation error at TLContext emit. 9. **Within composite size cap (ADR-0064 D7).** Decode opt2's `#2` diff --git a/docs/adr-proposed/DDD-0065-flat-ops-composite-softmax-merge-detailed-design.md b/docs/adr-proposed/DDD-0065-flat-ops-composite-softmax-merge-detailed-design.md index de79b62..33d349d 100644 --- a/docs/adr-proposed/DDD-0065-flat-ops-composite-softmax-merge-detailed-design.md +++ b/docs/adr-proposed/DDD-0065-flat-ops-composite-softmax-merge-detailed-design.md @@ -26,9 +26,11 @@ unchanged. 3. **Boundary preservation.** PE_SCHEDULER imports nothing recipe-related; PE_MATH/PE_GEMM/PE_DMA see only `Stage.params["op_kind"]` (no new engine code). -4. **Dispatch ratio.** Per-tile PE_CPU dispatch cycles - `ratio(opt3 / opt2) ≈ 4.0×` at default ADR-0064 Rev2 calibration - (FIXED=40 cycles, R=0.0625 cycles/byte). +4. **Dispatch ratio (robust).** Per-tile PE_CPU dispatch cycles satisfy + `opt3 > 2 × opt2` (qualitative invariant, calibration-independent). + Default ADR-0064 Rev2 calibration (FIXED=40 cycles, R=0.0625 + cycles/byte) gives model-expected ≈ 4.0× — informative only, not the + test gate. 5. **K-before-V invariant.** op_log of decode opt2 shows zero V-related DMA during #2's MATH prologue. @@ -437,7 +439,7 @@ Each phase is an independent Phase-1 → Phase-2 cycle per CLAUDE.md Part 1. | **P3** | PE_SCHEDULER `_generate_plan` flat-ops + `PipelinePlan.prologue_stages` + DMA auto-insertion + `_feed_loop` extension | all existing bench Stage sequences unchanged; new MATH chain visible in op_log for opt2 mock | | **P4** | Strict-FIFO RW hazard tracker + `_RwHazardTracker` integration | two-composite RW conflict test serializes correctly | | **P5** | `_gqa_decode_long.py` opt2 variant; data-mode numeric parity check; K-before-V invariant check | opt2 matches opt3 within fp tolerance; no V DMA during prologue | -| **P6** | Dispatch-ratio measurement: opt3 vs opt2 per-tile PE_CPU cycles + R sensitivity sweep | ratio ≈ 4.0× ± 15% at default calibration; opt2 < opt3 at all `R ∈ {0.25, 0.0625, 0.03125}` (ADR-0064 Test #9) | +| **P6** | Dispatch-ratio measurement: opt3 vs opt2 per-tile PE_CPU cycles + R sensitivity sweep | `opt3 > 2 × opt2` at default calibration (gate); model-expected ≈ 4.0× (informative); `opt2 < opt3` at all `R ∈ {0.25, 0.0625, 0.03125}` (ADR-0064 Test #9) | P0 must land first (separate ADR). P1 is pure refactor (safest). P2–P4 build the new path without touching opt3. P5 enables opt2. P6 closes @@ -488,7 +490,10 @@ Mirrors ADR-0065 §Test Requirements; grounded in SPEC R2/R5, ADR-0023/ **Dispatch ratio (P6):** - For `S_kv=64, n_tiles=16`, measure total PE_CPU dispatch cycles for opt3 vs opt2 paths. -- Assert ratio in `[3.4, 4.6]` (centre 4.0× at ADR-0064 Rev2 defaults). +- Assert `opt3 > 2 × opt2` (robust gate — calibration-independent). +- Record observed ratio; the default-calibration model expects ≈ 4.0× + (DDD §11). The recorded number is informative for performance + tracking, not a test gate. - **Sensitivity sweep.** Repeat with `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte; assert opt2 < opt3 in all three. (ADR-0064 Test #9.)