tl.composite now returns the output TensorHandle (not a CompletionHandle) so its result chains like tl.dot's; the handle carries the completion in a CompositeFuture pending so downstream ops and tl.wait auto-await it. out is a handle: out=tl.ref(addr,shape) (HBM, DMA_WRITE inside the composite) or an in-place TCM handle (STORE only); omitted -> TLContext auto-allocates a TCM scratch. out_ptr kept as HBM shorthand (= out=tl.ref(out_ptr, shape)) to avoid churning ~30 existing call sites.
tl.ref now returns space=hbm (it references HBM data; operand-input DMA stays pinned-based per D4 so input streaming is unchanged). tiling: the tile loop's DMA_WRITE is gated on out.space==hbm (out analog of the operand pinned rule) — a TCM output stays on-chip (chainable) and its high-bit scratch address no longer hits the DMA PA decoder.
Fixes the opt2 data-mode crash: the recipe accumulator O is TCM -> no DMA_WRITE -> opt2 now RUNS end-to-end in data mode (enable_data=True). Numeric parity of the recipe MATH ops is the next step. Suite 812 pass / 3 pre-existing fail; existing composite benches use out_ptr->hbm->DMA_WRITE unchanged (byte-equal).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tiling.generate_plan_from_ops: scan flat ops for the GEMM (<=1); pre-GEMM KERNEL ops become single-shot prologue MATH stages, post-GEMM ops split by scope (K_TILE/OUTPUT_TILE epilogue + KERNEL post-loop). MATH-only composite reproduces the legacy math-head plan. Prologue/post-loop stages fold into the first/last tile so the feeder + completion counting are untouched (existing benches have neither -> byte-equal op_log).
PipelinePlan gains prologue_stages/epilogue_stages. pe_scheduler._generate_plan delegates to generate_plan_from_ops. DMA keeps the existing pinned signal (NOT a space flip); recipe scratch/primary-out handles are pinned=True so the head GEMM's auto-bound a (=P) is consumed in place.
ADR-0065 D4/D6.7/Test#5 amended (EN+KO): DMA decision from pinned, not space; prologue recipe ops TCM-only (head op exempt -- it may stream from HBM).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ADR-0064 Revision 2 second-pass review fixes:
- D7 added 1024-byte cap rationale: explicitly framed as a *safe
engineering limit* (intentionally above all known composites at ~322
bytes, finite descriptor capacity placeholder), not a measured HW
number. Default is the discipline; topology override for real HW.
- D7 segment ordering: strict FIFO is the *ordering source*; rw_handles
is dependency metadata, not the ordering primitive. Explicit note for
the future RW-aware reorder migration path.
- Tests rewritten around the formula (D1) instead of specific numbers:
- Test #1 → "formula preservation" (parametrized across composites,
asserts dispatch == FIXED + bytes × R, not anchor 43 ns).
- Test #2 → robust "opt3 > 2 × opt2" qualitative gate; ≈ 4.0× is
informative only, not the gate, so calibration changes don't break.
- Test #9 → qualitative (opt2 < opt3 at all R), not a numeric ratio.
- Anchor description in D3 stays as *informative* (shows the model
produces ≈ 43 ns at defaults), but is no longer a test gate.
ADR-0065 second-pass review:
- D5 step 5 aligned with D6.6 invariant: explicit conflict check before
auto-binding (previously written as if auto-bind always happens).
Mirrors D6.6 wording: error if both kernel explicit `a` and prologue
primary_out are present.
- Test #7 made robust: `opt3 > 2 × opt2` gate (was ≈ 4.0×).
Default-calibration model expectation (≈ 4.0×) recorded as informative
reference, not the gate.
DDD-0065 follow-on robust thresholds:
- §1.4 success criteria: ratio condition rewritten as `opt3 > 2 × opt2`
with model-expected ≈ 4.0× as informative.
- §9 P6 phase gate: `opt3 > 2 × opt2` (was 4.0× ± 15%).
- §10 dispatch ratio test: assert `opt3 > 2 × opt2`; record observed
ratio for performance tracking but do not gate on it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ADR-0065 review fixes:
- D3 semantics clarified: position determines *phase* (pre-loop / head /
in-accumulation / per-output-tile / post-loop), scope determines
*repetition*. KERNEL = "once per CompositeCmd invocation", not "once
per kernel launch" — removes ambiguity around softmax_merge's KERNEL
scope at different positions before/after GEMM.
- D4 MATH-TCM invariant: Phase 1 limits DMA auto-insertion to GEMM
operands and head outputs; MATH operand with space="hbm" raises
validation error at TLContext emit. Catches kernel misuse early; a
MATH-with-HBM path is future work (explicit tl.load + MATH composite).
- D6 #6 NEW: auto-bind conflict — explicit GEMM `a=` + prologue
primary_out simultaneously → validation error (prevents "which value
wins" ambiguity).
- D6 #7 NEW: MATH operand TCM-only restatement.
- Consequences/Negative: strict-FIFO conservatism note — safe but may
under-expose composite-level overlap when handle-sharing composites
could in principle run in parallel.
- Test Requirements #10, #11: auto-bind conflict + MATH-HBM validation
error tests.
ADR-0064 Revision 2 clarification:
- D2 counting rule: per-op summation, no deduplication. Same handle
appearing in multiple OpSpecs or in rw_handles is counted
independently — matches HW reality (each descriptor field is a
distinct address slot, rw_handles is separate metadata block).
Example walked through (mul_bcast with O in src_a + out + rw_handles).
DDD-0065 follow-on:
- §3.2: new test file test_tl_composite_validation.py covers auto-bind
conflict, MATH-TCM, GEMM-count invariants.
- §4.3: phase vs repetition split aligned with ADR-0065 D3.
- §6 lowering pipeline: step 5 adds auto-bind conflict check; step 8
NEW — operand space validation; step 9 (size cap); step 10 (emit).
- §8 strict-FIFO conservatism note: explains why next-tile #1 cross-tile
pipelining still works under FIFO (no rw overlap), and where the
conservatism would hurt (future handle-sharing recipes).
- §10 verification: invariant-guard tests (auto-bind conflict, MATH
TCM, per-op summation accuracy).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ADR-0064 Revision 2 review fixes:
- D7 NEW: composite size cap (MAX_COMPOSITE_LOGICAL_BYTES default 1024
bytes). Oversized recipes deterministically segmented into N
CompositeCmds; each segment incurs its own dispatch cost. Models real
HW limits (descriptor queue entry, scheduler parser buffer, command
SRAM) and prevents the model from rewarding pathologically-large
fused composites.
- D2: type-aware extra-field byte counting (int/float=4, bool=1,
tuple/list=1+4N, str=1) — replaces uniform 4 bytes per extra.
- D3: recalibrated defaults to FIXED=40 cycles, R=0.0625 cycles/byte
(16 B/cycle — typical on-die descriptor queue width); anchor stays at
~43 ns for typical 1-OpSpec composite. Clarified anchor description:
DMA stages do not appear in logical_bytes (auto-inserted by
PE_SCHEDULER from operand.space per ADR-0065 D4).
- D4: removed clock_freq_ghz from pe_cost_model: override block;
conversion uses the PE node's existing clock_freq_ghz attr. Added
max_composite_logical_bytes knob.
- Context: emphasized command-count reduction (FIXED) as the primary
signal; byte term as secondary refinement.
- Open review: added large-composite scheduler-cost stress test.
- Test req: added composite-size-cap (#8) and R-sensitivity sweep (#9).
ADR-0065 + DDD-0065 follow-on updates:
- opt2 vs opt3 dispatch ratio updated 2.4× → ≈4.0× under new defaults
(FIXED-dominated, reflecting the corrected framing).
- Test req #9: decode opt2 composite fits within 1024-byte cap; no
segmentation needed for the GQA workload.
- DDD §6: TLContext lowering checks logical_bytes against cap (step 8).
- DDD §11: performance model recomputed with new defaults + sensitivity
table across R ∈ {0.25, 0.0625, 0.03125} confirming opt2 < opt3 holds.
- DDD §9 P6 gate: ratio band 2.4×±10% → 4.0×±15%; sensitivity sweep added.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ywkang