gqa(adr-0065): P2 — softmax_merge recipe + TLContext prologue lowering

New triton_emu/tl_recipes.py: RecipeDescriptor/EngineOp/PrimaryOutSpec + RECIPE_DESCRIPTORS['softmax_merge'] (8-step engine_seq). PE_SCHEDULER does not import it (ADR-0065 D5 boundary).

TLContext.composite(): add prologue=[...] + out=TensorHandle kwargs, a optional. _expand_prologue lowers a recipe into flat MATH OpSpecs (scope=KERNEL), allocates TCM scratch, derives the primary-out slot 'P' and auto-binds it into the head GEMM a (D6.6 conflict check); rw_handles=(m,l,O). decode-opt2 #2 lowers to 10 ops [rmax,max_elem,exp_diff,exp_diff,rsum,fma,mul_bcast,copy,gemm,add].

D6.7 (MATH operand TCM-only) scoped to prologue recipe ops only — the head op (gemm or math) keeps existing DMA-staged-from-HBM behavior. D6.1 (GEMM count <=1) on the whole composite. Host-side lowering only; PE_SCHEDULER position-scan is P3.

Also commit the ADR-0064 Rev2 promotion content that the prior commit's git mv dropped: Status Proposed->Accepted + D7 amended to hard-cap ValueError (no segmentation), EN+KO.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 19:48:29 -07:00
parent 47e2c78c66
commit 184f654295
6 changed files with 508 additions and 85 deletions
+25 -38
View File
@@ -2,7 +2,7 @@
## Status
Proposed (Revision 2)
Accepted (Revision 2)
> Supporting ADR for **ADR-0060** (AHBM GQA Fused Attention) and **ADR-0065**
> (flat-ops composite + first stateful recipe). The hybrid decision there
@@ -234,22 +234,15 @@ latencies are **regenerated once** when this ADR lands — same posture as
ADR-0062 D3 lazy-load. After regeneration, the same calibration is in
effect for ADR-0065 opt2 measurement.
### D7. Composite size cap (deterministic segmentation)
### D7. Composite size cap (hard limit — validation error)
Each `CompositeCmd`'s `logical_bytes` is capped at
**`MAX_COMPOSITE_LOGICAL_BYTES`** (default **1024 bytes**, overridable
per D4). Oversized commands are deterministically **segmented** by the
*emitter* (host-side TLContext, ADR-0065 D5) into N consecutive
`CompositeCmd`s, each ≤ cap.
- Each segment carries its own `completion: CompletionHandle`.
- Each segment incurs its own dispatch cost — `total =
sum(FIXED + bytes_i × R) = N × FIXED + total_bytes × R`. The FIXED
term is paid per segment.
- Segments share `rw_handles` where applicable; **strict FIFO**
(ADR-0065 D6.3) preserves write-after-write ordering automatically.
- The segmentation algorithm is deterministic (greedy by op index in
emit order); it is part of the host emitter, not PE_SCHEDULER.
**`max_composite_logical_bytes`** (default **1024 bytes**, overridable
per D4). A composite whose `logical_bytes` exceeds the cap is **rejected
at emit time** by the host-side TLContext with a `ValueError` — there is
**no automatic segmentation**. The kernel author must restructure the
recipe (e.g., split it into multiple smaller composites explicitly) so
each `CompositeCmd` fits within the descriptor capacity.
**Why a cap is needed.** Real hardware imposes hard limits — descriptor
queue entry size, scheduler parser buffer, command SRAM, firmware
@@ -266,20 +259,16 @@ overridable per topology (D4); when a real HW reference appears, the
value should be recalibrated. The role of this default is to make the
cap *exist as a discipline*, not to fit a specific HW.
**Ordering of segments — driven by strict FIFO, not `rw_handles`.**
Segments are emitted into the PE_CPU → PE_SCHEDULER queue in their
emit order. Strict-FIFO dispatch (ADR-0065 D6.3) is the *ordering
source*: segments execute in the order they enter the queue. The
`rw_handles` block on each segment is **dependency metadata** for the
cross-composite hazard tracker — it does **not** by itself guarantee
inter-segment ordering. If a future scheduler relaxed FIFO (e.g., to
RW-aware reorder, ADR-0065 A4), the segmenter would need to either
(a) merge segments into a single CompositeCmd, or (b) introduce an
explicit completion-handle dependency chain. For Phase 1 this does
not arise: strict FIFO is in effect.
**Rationale for a hard error over auto-segmentation.** Auto-splitting an
oversized composite (the original Revision 2 proposal) added emitter
complexity — inter-segment ordering, shared `rw_handles`, completion
chaining — to paper over what is, in practice, a kernel that asked for a
descriptor larger than the hardware allows. Surfacing it as an explicit
error keeps the emitter simple and makes the HW constraint visible to the
kernel author, who is best placed to decide how to split the work.
Decode opt2's `#2` composite (10 ops, ~322 bytes) sits comfortably
inside the 1024 cap — no segmentation for the GQA workload.
inside the 1024 cap — no error for the GQA workload.
## Alternatives
@@ -379,12 +368,9 @@ OpSpec/CompositeCmd fields are added.
(SPEC §0.1).
7. **Path parity.** Greenlet and replay paths produce identical
dispatch-cycle accounting for the same kernel.
8. **Composite size cap (D7).** A recipe that would emit `logical_bytes
> MAX_COMPOSITE_LOGICAL_BYTES` is segmented into N consecutive
`CompositeCmd`s; `sum(segment.logical_bytes) == original
logical_bytes`; total dispatch = sum of segment dispatches (FIXED
paid per segment); inter-segment ordering preserved by strict FIFO
(not by `rw_handles` alone).
8. **Composite size cap (D7).** A composite that would emit `logical_bytes
> max_composite_logical_bytes` raises a `ValueError` at emit time (no
segmentation). A composite within the cap emits normally.
9. **Sensitivity (qualitative).** At `R ∈ {0.25, 0.0625, 0.03125}`
cycles/byte, `opt3 > opt2` at *all* three points. Direction (ratio
monotonically increases as R decreases) is also asserted, but
@@ -398,11 +384,12 @@ ADR-0064 Revision 2 lands as a single PR with:
- formula application in `pe_cpu.py` dispatch path
- `pe_cost_model:` override read at PE_CPU init (cycle-domain knobs +
`max_composite_logical_bytes`)
- **composite size cap (D7)** — TLContext-side segmentation logic with
`MAX_COMPOSITE_LOGICAL_BYTES` default 1024; not needed for any
existing bench (largest current composite is well under 200 bytes),
but the mechanism lands so it is in place when ADR-0065's
10-op decode-opt2 composite (~310 bytes) and larger recipes appear.
- **composite size cap (D7)** — TLContext-side hard cap with
`max_composite_logical_bytes` default 1024: a composite over the cap
raises a `ValueError` at emit time (no auto-segmentation). Not
triggered by any existing bench (largest current composite is ~322
bytes), but the check lands so the descriptor-capacity limit is
enforced as recipes grow.
- one-time goldens regeneration
After this lands, ADR-0065 builds on top with no further goldens churn