ywkang/kernbench2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gqa(adr-0065): P2 — softmax_merge recipe + TLContext prologue lowering

Browse Source 
New triton_emu/tl_recipes.py: RecipeDescriptor/EngineOp/PrimaryOutSpec + RECIPE_DESCRIPTORS['softmax_merge'] (8-step engine_seq). PE_SCHEDULER does not import it (ADR-0065 D5 boundary).

TLContext.composite(): add prologue=[...] + out=TensorHandle kwargs, a optional. _expand_prologue lowers a recipe into flat MATH OpSpecs (scope=KERNEL), allocates TCM scratch, derives the primary-out slot 'P' and auto-binds it into the head GEMM a (D6.6 conflict check); rw_handles=(m,l,O). decode-opt2 #2 lowers to 10 ops [rmax,max_elem,exp_diff,exp_diff,rsum,fma,mul_bcast,copy,gemm,add].

D6.7 (MATH operand TCM-only) scoped to prologue recipe ops only — the head op (gemm or math) keeps existing DMA-staged-from-HBM behavior. D6.1 (GEMM count <=1) on the whole composite. Host-side lowering only; PE_SCHEDULER position-scan is P3.

Also commit the ADR-0064 Rev2 promotion content that the prior commit's git mv dropped: Status Proposed->Accepted + D7 amended to hard-cap ValueError (no segmentation), EN+KO.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Yangwook
2026-06-10 19:48:29 -07:00
parent 47e2c78c66
commit 184f654295
6 changed files with 508 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/adr-ko/ADR-0064-perf-cpu-issue-cost-model.md
+22 -33
View File
@@ -2,7 +2,7 @@
## Status
Proposed (Revision 2)
Accepted (Revision 2)
> **ADR-0060** (AHBM GQA Fused Attention) 와 **ADR-0065** (flat-ops
> composite + 첫 stateful recipe) 의 보조 ADR. 그 hybrid 의 핵심 — "GEMM 은
@@ -219,20 +219,14 @@ issue cost 를 0 → non-zero 로 바꾸면 **모든** bench 의 latency 변화.
이 ADR land 시 골든 latency **한 번** 재생성 — ADR-0062 D3 lazy-load 와
같은 패턴. 재생성 후 동일 calibration 이 ADR-0065 opt2 측정에 적용.
### D7. Composite 크기 cap (deterministic segmentation)
### D7. Composite 크기 cap (하드 제한 — validation error)
각 `CompositeCmd` 의 `logical_bytes` 가 **`MAX_COMPOSITE_LOGICAL_BYTES`**
(default **1024 bytes**, D4 로 override) 로 제한. 초과하는 cmd 는
*emitter* (host-side TLContext, ADR-0065 D5) 가 N 개 연속 `CompositeCmd`
로 deterministic 하게 **segment** — 각 ≤ cap.
- 각 segment 가 자기 `completion: CompletionHandle` 보유.
- 각 segment 가 자기 dispatch cost — `total = sum(FIXED + bytes_i × R) =
N × FIXED + total_bytes × R`. FIXED 항이 segment 당.
- segment 가 `rw_handles` 공유 — **strict FIFO** (ADR-0065 D6.3) 가
write-after-write 순서 자동 보존.
- segmentation 알고리즘 결정적 (emit 순서로 op 인덱스 greedy); host
emitter 의 일부, PE_SCHEDULER 가 아님.
각 `CompositeCmd` 의 `logical_bytes` 가 **`max_composite_logical_bytes`**
(default **1024 bytes**, D4 로 override) 로 제한. `logical_bytes` 가 cap
을 초과하는 composite 는 host-side TLContext 가 emit 시점에 `ValueError`
로 **거부** — **자동 segmentation 없음**. 커널 작성자가 각 `CompositeCmd`
가 descriptor capacity 에 맞도록 recipe 를 재구성(예: 여러 개의 더 작은
composite 로 명시적으로 분할)해야 함.
**cap 이 필요한 이유.** 실제 하드웨어는 descriptor queue entry 크기,
scheduler parser buffer, command SRAM, firmware 입력의 하드 제한 보유.
@@ -248,18 +242,15 @@ composite (가장 큰 것이 decode opt2 의 `#2` ~322 bytes) 보다 훨씬 위
이 default 의 역할은 cap 을 *원칙으로 존재시키는 것*, 특정 HW 에 맞추는
것이 아님.
**segment 간 순서 — strict FIFO 가 주체, `rw_handles` 아님.** segment 들은
emit 순서로 PE_CPU → PE_SCHEDULER 큐에 들어감. strict-FIFO dispatch
(ADR-0065 D6.3) 가 *순서의 주체* — segment 가 큐 진입 순서로 실행. 각
segment 의 `rw_handles` block 은 cross-composite 해저드 추적기의 **의존성
메타데이터** — 그 자체로 inter-segment 순서를 보장하지 **않음**. 미래
scheduler 가 FIFO 를 완화 (예: RW-aware reorder, ADR-0065 A4) 한다면
segmenter 는 (a) segment 를 단일 CompositeCmd 로 merge 또는
(b) 명시적 completion-handle 의존 체인 도입 필요. Phase 1 에선 무관 —
strict FIFO 가 유효.
**자동 segmentation 대신 하드 에러를 택한 근거.** 초과 composite 를 자동
분할(원래 Revision 2 제안)하는 것은 inter-segment 순서, `rw_handles`
공유, completion 체이닝 등 emitter 복잡도를 추가하면서, 실질적으로는
하드웨어가 허용하는 것보다 큰 descriptor 를 요청한 커널을 덮어주는
것일 뿐. 명시적 에러로 표면화하면 emitter 가 단순해지고, HW 제약이
작업을 어떻게 분할할지 가장 잘 아는 커널 작성자에게 보임.
Decode opt2 의 `#2` composite (10 ops, ~322 bytes) 는 1024 cap 안에
편안히 — GQA workload 에 segmentation 없음.
편안히 — GQA workload 에 에러 없음.
## Alternatives
@@ -350,10 +341,8 @@ calibration 이 바뀌거나 OpSpec/CompositeCmd 필드가 추가되어도 유
7. **경로 parity.** greenlet 과 replay 가 동일 커널에 대해 동일
dispatch-cycle 회계.
8. **Composite 크기 cap (D7).** `logical_bytes >
MAX_COMPOSITE_LOGICAL_BYTES` 가 될 recipe 가 N 개 연속 `CompositeCmd`
로 segment; `sum(segment.logical_bytes) == original logical_bytes`;
총 dispatch = segment dispatch 의 합 (FIXED 가 segment 당 지불);
inter-segment 순서 strict FIFO 로 보존 (`rw_handles` 단독 아님).
max_composite_logical_bytes` 가 될 composite 는 emit 시점에 `ValueError`
(segmentation 없음). cap 이내 composite 는 정상 emit.
9. **민감도 (정성적).** `R ∈ {0.25, 0.0625, 0.03125}` cycles/byte 에서
3 지점 모두 `opt3 > opt2`. 방향 (R 감소 시 ratio 단조 증가) 도 단언,
단 절대 ratio 값은 *불필요*.
@@ -366,11 +355,11 @@ ADR-0064 Revision 2 는 단일 PR 로 land:
- `pe_cpu.py` dispatch 경로의 공식 적용
- PE_CPU init 의 `pe_cost_model:` override read (cycle-domain knob +
`max_composite_logical_bytes`)
- **composite 크기 cap (D7)** — TLContext-side segmentation 로직,
`MAX_COMPOSITE_LOGICAL_BYTES` default 1024; 기존 bench 에 필요 없음
(현재 최대 composite 가 200 bytes 훨씬 아래), 하지만 ADR-0065 의 10-op
decode-opt2 composite (~310 bytes) + 더 큰 recipe 가 나타날 때를 위해
메커니즘 land.
- **composite 크기 cap (D7)** — TLContext-side 하드 cap,
`max_composite_logical_bytes` default 1024: cap 초과 composite 는 emit
시점에 `ValueError` (자동 segmentation 없음). 기존 bench 는 트리거
안 됨 (현재 최대 composite 가 ~322 bytes), 하지만 recipe 가 커질 때
descriptor-capacity 제한이 강제되도록 체크 land.
- 일회성 골든 재생성
land 후 ADR-0065 가 위에 쌓임 — 기존 bench 추가 골든 churn 없음
docs/adr/ADR-0064-perf-cpu-issue-cost-model.md
+25 -38
View File
@@ -2,7 +2,7 @@
## Status
Proposed (Revision 2)
Accepted (Revision 2)
> Supporting ADR for **ADR-0060** (AHBM GQA Fused Attention) and **ADR-0065**
> (flat-ops composite + first stateful recipe). The hybrid decision there
@@ -234,22 +234,15 @@ latencies are **regenerated once** when this ADR lands — same posture as
ADR-0062 D3 lazy-load. After regeneration, the same calibration is in
effect for ADR-0065 opt2 measurement.
### D7. Composite size cap (deterministic segmentation)
### D7. Composite size cap (hard limit — validation error)
Each `CompositeCmd`'s `logical_bytes` is capped at
**`MAX_COMPOSITE_LOGICAL_BYTES`** (default **1024 bytes**, overridable
per D4). Oversized commands are deterministically **segmented** by the
*emitter* (host-side TLContext, ADR-0065 D5) into N consecutive
`CompositeCmd`s, each ≤ cap.
- Each segment carries its own `completion: CompletionHandle`.
- Each segment incurs its own dispatch cost — `total =
sum(FIXED + bytes_i × R) = N × FIXED + total_bytes × R`. The FIXED
term is paid per segment.
- Segments share `rw_handles` where applicable; **strict FIFO**
(ADR-0065 D6.3) preserves write-after-write ordering automatically.
- The segmentation algorithm is deterministic (greedy by op index in
emit order); it is part of the host emitter, not PE_SCHEDULER.
**`max_composite_logical_bytes`** (default **1024 bytes**, overridable
per D4). A composite whose `logical_bytes` exceeds the cap is **rejected
at emit time** by the host-side TLContext with a `ValueError` — there is
**no automatic segmentation**. The kernel author must restructure the
recipe (e.g., split it into multiple smaller composites explicitly) so
each `CompositeCmd` fits within the descriptor capacity.
**Why a cap is needed.** Real hardware imposes hard limits — descriptor
queue entry size, scheduler parser buffer, command SRAM, firmware
@@ -266,20 +259,16 @@ overridable per topology (D4); when a real HW reference appears, the
value should be recalibrated. The role of this default is to make the
cap *exist as a discipline*, not to fit a specific HW.
**Ordering of segments — driven by strict FIFO, not `rw_handles`.**
Segments are emitted into the PE_CPU → PE_SCHEDULER queue in their
emit order. Strict-FIFO dispatch (ADR-0065 D6.3) is the *ordering
source*: segments execute in the order they enter the queue. The
`rw_handles` block on each segment is **dependency metadata** for the
cross-composite hazard tracker — it does **not** by itself guarantee
inter-segment ordering. If a future scheduler relaxed FIFO (e.g., to
RW-aware reorder, ADR-0065 A4), the segmenter would need to either
(a) merge segments into a single CompositeCmd, or (b) introduce an
explicit completion-handle dependency chain. For Phase 1 this does
not arise: strict FIFO is in effect.
**Rationale for a hard error over auto-segmentation.** Auto-splitting an
oversized composite (the original Revision 2 proposal) added emitter
complexity — inter-segment ordering, shared `rw_handles`, completion
chaining — to paper over what is, in practice, a kernel that asked for a
descriptor larger than the hardware allows. Surfacing it as an explicit
error keeps the emitter simple and makes the HW constraint visible to the
kernel author, who is best placed to decide how to split the work.
Decode opt2's `#2` composite (10 ops, ~322 bytes) sits comfortably
inside the 1024 cap — no segmentation for the GQA workload.
inside the 1024 cap — no error for the GQA workload.
## Alternatives
@@ -379,12 +368,9 @@ OpSpec/CompositeCmd fields are added.
(SPEC §0.1).
7. **Path parity.** Greenlet and replay paths produce identical
dispatch-cycle accounting for the same kernel.
8. **Composite size cap (D7).** A recipe that would emit `logical_bytes
> MAX_COMPOSITE_LOGICAL_BYTES` is segmented into N consecutive
`CompositeCmd`s; `sum(segment.logical_bytes) == original
logical_bytes`; total dispatch = sum of segment dispatches (FIXED
paid per segment); inter-segment ordering preserved by strict FIFO
(not by `rw_handles` alone).
8. **Composite size cap (D7).** A composite that would emit `logical_bytes
> max_composite_logical_bytes` raises a `ValueError` at emit time (no
segmentation). A composite within the cap emits normally.
9. **Sensitivity (qualitative).** At `R ∈ {0.25, 0.0625, 0.03125}`
cycles/byte, `opt3 > opt2` at *all* three points. Direction (ratio
monotonically increases as R decreases) is also asserted, but
@@ -398,11 +384,12 @@ ADR-0064 Revision 2 lands as a single PR with:
- formula application in `pe_cpu.py` dispatch path
- `pe_cost_model:` override read at PE_CPU init (cycle-domain knobs +
`max_composite_logical_bytes`)
- **composite size cap (D7)** — TLContext-side segmentation logic with
`MAX_COMPOSITE_LOGICAL_BYTES` default 1024; not needed for any
existing bench (largest current composite is well under 200 bytes),
but the mechanism lands so it is in place when ADR-0065's
10-op decode-opt2 composite (~310 bytes) and larger recipes appear.
- **composite size cap (D7)** — TLContext-side hard cap with
`max_composite_logical_bytes` default 1024: a composite over the cap
raises a `ValueError` at emit time (no auto-segmentation). Not
triggered by any existing bench (largest current composite is ~322
bytes), but the check lands so the descriptor-capacity limit is
enforced as recipes grow.
- one-time goldens regeneration
After this lands, ADR-0065 builds on top with no further goldens churn