gqa(adr): ADR-0065 review fixes — KERNEL phase/repetition split, auto-bind conflict + MATH-TCM invariants, FIFO conservatism note, logical_bytes per-op summation
ADR-0065 review fixes: - D3 semantics clarified: position determines *phase* (pre-loop / head / in-accumulation / per-output-tile / post-loop), scope determines *repetition*. KERNEL = "once per CompositeCmd invocation", not "once per kernel launch" — removes ambiguity around softmax_merge's KERNEL scope at different positions before/after GEMM. - D4 MATH-TCM invariant: Phase 1 limits DMA auto-insertion to GEMM operands and head outputs; MATH operand with space="hbm" raises validation error at TLContext emit. Catches kernel misuse early; a MATH-with-HBM path is future work (explicit tl.load + MATH composite). - D6 #6 NEW: auto-bind conflict — explicit GEMM `a=` + prologue primary_out simultaneously → validation error (prevents "which value wins" ambiguity). - D6 #7 NEW: MATH operand TCM-only restatement. - Consequences/Negative: strict-FIFO conservatism note — safe but may under-expose composite-level overlap when handle-sharing composites could in principle run in parallel. - Test Requirements #10, #11: auto-bind conflict + MATH-HBM validation error tests. ADR-0064 Revision 2 clarification: - D2 counting rule: per-op summation, no deduplication. Same handle appearing in multiple OpSpecs or in rw_handles is counted independently — matches HW reality (each descriptor field is a distinct address slot, rw_handles is separate metadata block). Example walked through (mul_bcast with O in src_a + out + rw_handles). DDD-0065 follow-on: - §3.2: new test file test_tl_composite_validation.py covers auto-bind conflict, MATH-TCM, GEMM-count invariants. - §4.3: phase vs repetition split aligned with ADR-0065 D3. - §6 lowering pipeline: step 5 adds auto-bind conflict check; step 8 NEW — operand space validation; step 9 (size cap); step 10 (emit). - §8 strict-FIFO conservatism note: explains why next-tile #1 cross-tile pipelining still works under FIFO (no rw overlap), and where the conservatism would hurt (future handle-sharing recipes). - §10 verification: invariant-guard tests (auto-bind conflict, MATH TCM, per-op summation accuracy). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -103,19 +103,27 @@ takes `operands["a"]`, `operands["b"]`) without positional convention.
|
||||
|
||||
### D3. Position + scope determines tile-loop placement
|
||||
|
||||
**Semantics split.** *Position* determines the **phase** (where in the
|
||||
tile-loop lifecycle the op fires); *scope* determines the **repetition**
|
||||
within that phase. `Scope.KERNEL` means "**once per `CompositeCmd`
|
||||
invocation**", not "once per kernel launch" — its single firing happens
|
||||
at the op's position in the sequence. The same KERNEL value carries
|
||||
different *phases* depending on whether the OpSpec sits before, at, or
|
||||
after the GEMM op.
|
||||
|
||||
PE_SCHEDULER scans `cmd.ops` for the GEMM op (≤1 per CompositeCmd, see
|
||||
D6). Calling its index `g`:
|
||||
|
||||
| OpSpec position | scope | Placement in plan |
|
||||
|---|---|---|
|
||||
| `0 .. g-1` | KERNEL | pre-loop stages (run once before tile loop) |
|
||||
| `g` | KERNEL | head GEMM (drives tile loop with `extra["m"], extra["k"], extra["n"]`) |
|
||||
| `g+1 .. ` | K_TILE | per K-tile epilogue (inside accumulation loop) |
|
||||
| `g+1 .. ` | OUTPUT_TILE | per (m, n) tile epilogue (after K accumulation) |
|
||||
| `g+1 .. ` | KERNEL | post-loop stages (run once after tile loop) |
|
||||
| OpSpec position | scope | Phase | Placement in plan |
|
||||
|---|---|---|---|
|
||||
| `0 .. g-1` | KERNEL | pre-tile-loop | run once before tile loop |
|
||||
| `g` | KERNEL | head GEMM | drives tile loop with `extra["m"], extra["k"], extra["n"]` |
|
||||
| `g+1 .. ` | K_TILE | in-accumulation | per K-tile epilogue (inside K accumulation loop) |
|
||||
| `g+1 .. ` | OUTPUT_TILE | per-output-tile | per (m, n) tile epilogue (after K accumulation) |
|
||||
| `g+1 .. ` | KERNEL | post-tile-loop | run once after tile loop |
|
||||
|
||||
A composite with no GEMM op (e.g., MATH-only composite) treats all ops
|
||||
as KERNEL-scope sequential.
|
||||
as KERNEL-scope sequential — phase collapses to "single-shot in order".
|
||||
|
||||
### D4. PE_SCHEDULER auto-inserts DMAs from operand `space`
|
||||
|
||||
@@ -130,9 +138,15 @@ DMA cmds for composite operands; PE_SCHEDULER infers them from
|
||||
handles**. `tl.ref(addr, shape)` returns a handle with `space="hbm"`;
|
||||
`tl.zeros` / `tl.full` / `tl.load` outputs return `space="tcm"`.
|
||||
|
||||
Non-GEMM ops (MATH chain in prologue / epilogue) have TCM-resident
|
||||
operands (kernel allocates via `tl.zeros` / `tl.full`); no DMA inserted
|
||||
for them.
|
||||
**Non-GEMM ops are TCM-only (Phase 1 limit, enforced at TLContext emit).**
|
||||
Every operand of every non-GEMM (MATH) OpSpec **must** have
|
||||
`space == "tcm"`. If a kernel passes a handle with `space == "hbm"` as a
|
||||
MATH operand, TLContext raises a validation error at emit time. This
|
||||
keeps DMA auto-insertion bounded to GEMM operands and head output
|
||||
handles in Phase 1; a MATH-with-HBM path would require a future
|
||||
extension (split into explicit `tl.load` + MATH composite, or a new
|
||||
"DMA-as-prologue" recipe variant). This invariant is also restated in
|
||||
D6 #7.
|
||||
|
||||
### D5. RECIPE_DESCRIPTORS — TLContext-internal, NOT in `pe_commands.py`
|
||||
|
||||
@@ -211,6 +225,16 @@ sees only the flat ops list.
|
||||
5. **PE_SCHEDULER recipe-free.** PE_SCHEDULER does not import
|
||||
RECIPE_DESCRIPTORS, does not know that "softmax_merge" exists, and
|
||||
has no recipe-specific code branches.
|
||||
6. **No implicit operand replacement (auto-bind conflict).** If a
|
||||
prologue recipe declares a `primary_out` AND the head op's auto-bind
|
||||
target operand (e.g., GEMM `a`) is *also* provided by the kernel
|
||||
explicitly, TLContext raises a validation error at emit time. The
|
||||
kernel must either let the recipe bind (omit the operand) **or**
|
||||
specify it explicitly (omit the prologue, or use a recipe without
|
||||
`primary_out`). This prevents the ambiguity of "which value wins".
|
||||
7. **MATH operands TCM-only.** Restatement of D4: every operand of a
|
||||
non-GEMM OpSpec must have `space == "tcm"`. Phase 1 enforced at
|
||||
TLContext emit.
|
||||
|
||||
### D7. Boundary summary (compiler vs scheduler vs engine)
|
||||
|
||||
@@ -297,6 +321,15 @@ input from #1's epilogue scratch — circular tile-loop dependency.
|
||||
primary_out auto-bind. ~80–120 LOC.
|
||||
- PE_SCHEDULER's `_generate_plan` gets a position-based scan + DMA
|
||||
auto-insertion + strict-FIFO RW tracker. ~60–100 LOC.
|
||||
- **Strict-FIFO RW hazard tracking is safe but conservative.** When
|
||||
two composites share any `rw_handle`, the new one waits for *all*
|
||||
prior overlapping composites to fully complete — even when their
|
||||
compute could in principle overlap (e.g., next-tile #1 = Q·Kᵀ does
|
||||
not touch m/l/O and could overlap with this-tile's #2, but FIFO
|
||||
still serializes any path that touches the same handles down the
|
||||
line). This **may under-expose composite-level overlap** relative to
|
||||
a smarter scheduler. RW-aware reorder (A4) is the deferred
|
||||
improvement.
|
||||
|
||||
## Open review items
|
||||
|
||||
@@ -344,9 +377,17 @@ input from #1's epilogue scratch — circular tile-loop dependency.
|
||||
8. **GEMM-count invariant.** A composite with two GEMM OpSpecs raises a
|
||||
validation error at TLContext emit.
|
||||
9. **Within composite size cap (ADR-0064 D7).** Decode opt2's `#2`
|
||||
composite (10 ops, ~310 logical bytes) fits comfortably under the
|
||||
default `MAX_COMPOSITE_LOGICAL_BYTES=1024` — no segmentation. The
|
||||
recipe lowering test asserts `cmd.logical_bytes < 1024`.
|
||||
composite (10 ops, ~322 logical bytes — per-op summation of operand
|
||||
handles) fits comfortably under the default
|
||||
`MAX_COMPOSITE_LOGICAL_BYTES=1024` — no segmentation. The recipe
|
||||
lowering test asserts `cmd.logical_bytes < 1024`.
|
||||
10. **Auto-bind conflict.** `tl.composite(op="gemm", a=A_handle,
|
||||
prologue=[{"op": "softmax_merge", ...}], ...)` where softmax_merge
|
||||
declares `primary_out` raises a validation error at emit time
|
||||
(D6.6).
|
||||
11. **MATH operand TCM invariant.** `tl.composite(op="math",
|
||||
a=tl.ref(addr, shape), ...)` (passing an HBM-resident handle as a
|
||||
MATH operand) raises a validation error at emit time (D6.7).
|
||||
|
||||
## Dependencies
|
||||
|
||||
|
||||
Reference in New Issue
Block a user