gqa(adr-0065): P2 — softmax_merge recipe + TLContext prologue lowering
New triton_emu/tl_recipes.py: RecipeDescriptor/EngineOp/PrimaryOutSpec + RECIPE_DESCRIPTORS['softmax_merge'] (8-step engine_seq). PE_SCHEDULER does not import it (ADR-0065 D5 boundary). TLContext.composite(): add prologue=[...] + out=TensorHandle kwargs, a optional. _expand_prologue lowers a recipe into flat MATH OpSpecs (scope=KERNEL), allocates TCM scratch, derives the primary-out slot 'P' and auto-binds it into the head GEMM a (D6.6 conflict check); rw_handles=(m,l,O). decode-opt2 #2 lowers to 10 ops [rmax,max_elem,exp_diff,exp_diff,rsum,fma,mul_bcast,copy,gemm,add]. D6.7 (MATH operand TCM-only) scoped to prologue recipe ops only — the head op (gemm or math) keeps existing DMA-staged-from-HBM behavior. D6.1 (GEMM count <=1) on the whole composite. Host-side lowering only; PE_SCHEDULER position-scan is P3. Also commit the ADR-0064 Rev2 promotion content that the prior commit's git mv dropped: Status Proposed->Accepted + D7 amended to hard-cap ValueError (no segmentation), EN+KO. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
## Status
|
||||
|
||||
Proposed (Revision 2)
|
||||
Accepted (Revision 2)
|
||||
|
||||
> Supporting ADR for **ADR-0060** (AHBM GQA Fused Attention) and **ADR-0065**
|
||||
> (flat-ops composite + first stateful recipe). The hybrid decision there
|
||||
@@ -234,22 +234,15 @@ latencies are **regenerated once** when this ADR lands — same posture as
|
||||
ADR-0062 D3 lazy-load. After regeneration, the same calibration is in
|
||||
effect for ADR-0065 opt2 measurement.
|
||||
|
||||
### D7. Composite size cap (deterministic segmentation)
|
||||
### D7. Composite size cap (hard limit — validation error)
|
||||
|
||||
Each `CompositeCmd`'s `logical_bytes` is capped at
|
||||
**`MAX_COMPOSITE_LOGICAL_BYTES`** (default **1024 bytes**, overridable
|
||||
per D4). Oversized commands are deterministically **segmented** by the
|
||||
*emitter* (host-side TLContext, ADR-0065 D5) into N consecutive
|
||||
`CompositeCmd`s, each ≤ cap.
|
||||
|
||||
- Each segment carries its own `completion: CompletionHandle`.
|
||||
- Each segment incurs its own dispatch cost — `total =
|
||||
sum(FIXED + bytes_i × R) = N × FIXED + total_bytes × R`. The FIXED
|
||||
term is paid per segment.
|
||||
- Segments share `rw_handles` where applicable; **strict FIFO**
|
||||
(ADR-0065 D6.3) preserves write-after-write ordering automatically.
|
||||
- The segmentation algorithm is deterministic (greedy by op index in
|
||||
emit order); it is part of the host emitter, not PE_SCHEDULER.
|
||||
**`max_composite_logical_bytes`** (default **1024 bytes**, overridable
|
||||
per D4). A composite whose `logical_bytes` exceeds the cap is **rejected
|
||||
at emit time** by the host-side TLContext with a `ValueError` — there is
|
||||
**no automatic segmentation**. The kernel author must restructure the
|
||||
recipe (e.g., split it into multiple smaller composites explicitly) so
|
||||
each `CompositeCmd` fits within the descriptor capacity.
|
||||
|
||||
**Why a cap is needed.** Real hardware imposes hard limits — descriptor
|
||||
queue entry size, scheduler parser buffer, command SRAM, firmware
|
||||
@@ -266,20 +259,16 @@ overridable per topology (D4); when a real HW reference appears, the
|
||||
value should be recalibrated. The role of this default is to make the
|
||||
cap *exist as a discipline*, not to fit a specific HW.
|
||||
|
||||
**Ordering of segments — driven by strict FIFO, not `rw_handles`.**
|
||||
Segments are emitted into the PE_CPU → PE_SCHEDULER queue in their
|
||||
emit order. Strict-FIFO dispatch (ADR-0065 D6.3) is the *ordering
|
||||
source*: segments execute in the order they enter the queue. The
|
||||
`rw_handles` block on each segment is **dependency metadata** for the
|
||||
cross-composite hazard tracker — it does **not** by itself guarantee
|
||||
inter-segment ordering. If a future scheduler relaxed FIFO (e.g., to
|
||||
RW-aware reorder, ADR-0065 A4), the segmenter would need to either
|
||||
(a) merge segments into a single CompositeCmd, or (b) introduce an
|
||||
explicit completion-handle dependency chain. For Phase 1 this does
|
||||
not arise: strict FIFO is in effect.
|
||||
**Rationale for a hard error over auto-segmentation.** Auto-splitting an
|
||||
oversized composite (the original Revision 2 proposal) added emitter
|
||||
complexity — inter-segment ordering, shared `rw_handles`, completion
|
||||
chaining — to paper over what is, in practice, a kernel that asked for a
|
||||
descriptor larger than the hardware allows. Surfacing it as an explicit
|
||||
error keeps the emitter simple and makes the HW constraint visible to the
|
||||
kernel author, who is best placed to decide how to split the work.
|
||||
|
||||
Decode opt2's `#2` composite (10 ops, ~322 bytes) sits comfortably
|
||||
inside the 1024 cap — no segmentation for the GQA workload.
|
||||
inside the 1024 cap — no error for the GQA workload.
|
||||
|
||||
## Alternatives
|
||||
|
||||
@@ -379,12 +368,9 @@ OpSpec/CompositeCmd fields are added.
|
||||
(SPEC §0.1).
|
||||
7. **Path parity.** Greenlet and replay paths produce identical
|
||||
dispatch-cycle accounting for the same kernel.
|
||||
8. **Composite size cap (D7).** A recipe that would emit `logical_bytes
|
||||
> MAX_COMPOSITE_LOGICAL_BYTES` is segmented into N consecutive
|
||||
`CompositeCmd`s; `sum(segment.logical_bytes) == original
|
||||
logical_bytes`; total dispatch = sum of segment dispatches (FIXED
|
||||
paid per segment); inter-segment ordering preserved by strict FIFO
|
||||
(not by `rw_handles` alone).
|
||||
8. **Composite size cap (D7).** A composite that would emit `logical_bytes
|
||||
> max_composite_logical_bytes` raises a `ValueError` at emit time (no
|
||||
segmentation). A composite within the cap emits normally.
|
||||
9. **Sensitivity (qualitative).** At `R ∈ {0.25, 0.0625, 0.03125}`
|
||||
cycles/byte, `opt3 > opt2` at *all* three points. Direction (ratio
|
||||
monotonically increases as R decreases) is also asserted, but
|
||||
@@ -398,11 +384,12 @@ ADR-0064 Revision 2 lands as a single PR with:
|
||||
- formula application in `pe_cpu.py` dispatch path
|
||||
- `pe_cost_model:` override read at PE_CPU init (cycle-domain knobs +
|
||||
`max_composite_logical_bytes`)
|
||||
- **composite size cap (D7)** — TLContext-side segmentation logic with
|
||||
`MAX_COMPOSITE_LOGICAL_BYTES` default 1024; not needed for any
|
||||
existing bench (largest current composite is well under 200 bytes),
|
||||
but the mechanism lands so it is in place when ADR-0065's
|
||||
10-op decode-opt2 composite (~310 bytes) and larger recipes appear.
|
||||
- **composite size cap (D7)** — TLContext-side hard cap with
|
||||
`max_composite_logical_bytes` default 1024: a composite over the cap
|
||||
raises a `ValueError` at emit time (no auto-segmentation). Not
|
||||
triggered by any existing bench (largest current composite is ~322
|
||||
bytes), but the check lands so the descriptor-capacity limit is
|
||||
enforced as recipes grow.
|
||||
- one-time goldens regeneration
|
||||
|
||||
After this lands, ADR-0065 builds on top with no further goldens churn
|
||||
|
||||
Reference in New Issue
Block a user