ywkang 5bbe293bea gqa(adr-0065): N3 — GEMM-epilogue accumulate dataflow (O = O.corr + P.V) ...

Two fixes make the recipe's P.V GEMM fold into the rescaled accumulator: (1) op_log marks the composite gemm record accumulate=True when an 'add' epilogue targets the GEMM's own output; data_executor._execute_gemm then does O = O_existing + a@b instead of overwriting. (2) PE_SCHEDULER records the composite's numeric op at COMPLETION (in _retire_on_done) instead of at dispatch, so the GEMM's t_start sorts AFTER its prologue MATH ops -> it reads the computed P and the rescaled O.

Verified end-to-end (DataExecutor): the full recipe produces O = O_old*corr + P@V matching a numpy flash-attention reference. Suite 816 pass / 3 pre-existing fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-10 23:06:01 -07:00

..

attention

gqa(adr-0065): P6 — close dispatch-ratio loop (ADR-0064 Test #9 direction)

2026-06-10 21:04:45 -07:00

custom

Add SIP-level tensor parallelism, component registry YAML, VA offset verification

2026-03-26 01:13:17 -07:00

gemm

eval: fold GEMM/allreduce harnesses into self-contained milestone benches

2026-05-22 15:19:52 -07:00

sccl

eval: fold GEMM/allreduce harnesses into self-contained milestone benches

2026-05-22 15:19:52 -07:00

conftest.py

sccl: drive allreduce tests via torch.distributed; reorganize into tests/sccl/

2026-05-20 22:24:43 -07:00

test_adr0026_dppolicy_intra_device.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_bench_registry.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_bw_occupancy.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_ccl_framework.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_ccl_helpers.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_ccl_install.py

Intercube allreduce: pe0 cube-mesh reduce + multi-SIP ring/torus/mesh

2026-04-16 17:33:42 -07:00

test_ccl_strict_mode.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_ccl_topologies.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_cli_list.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_cli_verify_data.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_cli.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_component_registry.py

Calibrate 3 tests for ADR-0033 Phase 2c per-flit wire timing

2026-05-14 23:06:33 -07:00

test_composite_epilogue.py

gqa(adr-0064/0065): flat-ops CompositeCmd (P1) + structural dispatch cost (ADR-0064 Rev2); promote ADR-0064

2026-06-10 19:18:04 -07:00

test_composite_flat_ops_refactor.py

gqa(adr-0064/0065): flat-ops CompositeCmd (P1) + structural dispatch cost (ADR-0064 Rev2); promote ADR-0064

2026-06-10 19:18:04 -07:00

test_composite_output_handle.py

gqa(adr-0065): D8 — composite returns output handle + output-space DMA

2026-06-10 22:29:10 -07:00

test_cross_sip_routing.py

ADR-0019 D1/D4: per-PE HBM CTRL partitioning

2026-05-15 01:04:30 -07:00

test_d5_barrier_invariant.py

ADR-0009 D5: chain-aware target_start_ns + zero-byte launch fanout

2026-04-27 15:12:58 -07:00

test_data_executor.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_dppolicy_cube_start.py

policy: add DPPolicy.cube_start for disjoint cube sub-meshes within a SIP

2026-06-04 12:39:30 -07:00

test_e2e_data.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_e2e_pipeline.py

gqa(adr-0065): N1 — composite GEMM computes numerically in data mode

2026-06-10 22:46:39 -07:00

test_emit_ipcq_diagram.py

Add tests/test_emit_ipcq_diagram.py (missed from earlier commit)

2026-04-27 21:42:44 -07:00

test_engine.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_flit_streaming.py

ADR-0033 Phase 2c-3 finish: op_log test + ADR doc reflect chunk-streaming

2026-05-14 23:12:50 -07:00

test_hbm_address_based_pc.py

ADR-0019 D1/D4: per-PE HBM CTRL partitioning

2026-05-15 01:04:30 -07:00

test_hbm_pc_striping.py

ADR housekeeping: category prefixes, lifecycle folders, retroactive 0034-0037

2026-05-20 01:15:55 -07:00

test_intercube_sfr_config.py

CCL allreduce: rename to lrab_hierarchical_allreduce + descriptive plots

2026-05-20 20:50:48 -07:00

test_iochiplet_noc_d2h.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_ipcq_buffer_kind_latency.py

sccl: drive allreduce tests via torch.distributed; reorganize into tests/sccl/

2026-05-20 22:24:43 -07:00

test_ipcq_buffer_kind_locations.py

sccl: drive allreduce tests via torch.distributed; reorganize into tests/sccl/

2026-05-20 22:24:43 -07:00

test_ipcq_types.py

Fix ADR-0025: IPCQ direction addressing via address-based matching

2026-04-14 00:38:41 -07:00

test_kernel_launch_sync.py

Kernel-launch sync (ADR-0009 D5) and IPCQ drain at inbound (ADR-0023)

2026-04-23 15:30:29 -07:00

test_kernel_runner.py

gqa: ADR-0060/0062/0063/0064 unified GQA kernels + CPU cost model

2026-06-09 18:15:59 -07:00

test_launch_dim_translation.py

runtime_api: ctx.launch honors DPPolicy.num_cubes + adds _auto_dim_remap opt-out

2026-06-01 19:33:40 -07:00

test_memory_store.py

Implement ADR-0020: 2-pass data execution with greenlet kernel runner

2026-04-08 00:22:44 -07:00

test_milestone_benches.py

eval: fold GEMM/allreduce harnesses into self-contained milestone benches

2026-05-22 15:19:52 -07:00

test_mmu_component.py

Rename impl names: add builtin. prefix for clear provenance

2026-04-09 00:16:24 -07:00

test_mmu_fabric.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_noc_mesh.py

ADR housekeeping: category prefixes, lifecycle folders, retroactive 0034-0037

2026-05-20 01:15:55 -07:00

test_op_log_input_snapshot_race.py

sim_engine: fix IPCQ slot-wrap snapshot race in Phase 2 replay

2026-06-01 19:14:09 -07:00

test_op_log.py

Implement ADR-0020: 2-pass data execution with greenlet kernel runner

2026-04-08 00:22:44 -07:00

test_pe_components.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_pe_cost_model.py

gqa(adr-0064/0065): flat-ops CompositeCmd (P1) + structural dispatch cost (ADR-0064 Rev2); promote ADR-0064

2026-06-10 19:18:04 -07:00

test_pe_dma_ipcq.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_pe_ipcq.py

Fix ADR-0025: IPCQ direction addressing via address-based matching

2026-04-14 00:38:41 -07:00

test_pe_mmu.py

Add virtual memory support: PE_MMU, VA allocator, fabric MmuMapMsg

2026-03-26 00:01:47 -07:00

test_pe_pipeline.py

ADR housekeeping: category prefixes, lifecycle folders, retroactive 0034-0037

2026-05-20 01:15:55 -07:00

test_pe_scheduler_flat_ops_plan.py

gqa(adr-0065): P3 — flat-ops PE_SCHEDULER plan (position-scan + prologue stages)

2026-06-10 20:24:33 -07:00

test_pe_scheduler_strict_fifo.py

gqa(adr-0065): P4 — strict-FIFO RW hazard tracker

2026-06-10 20:36:31 -07:00

test_pe_to_pe_diagnostic.py

CCL allreduce: rename to lrab_hierarchical_allreduce + descriptive plots

2026-05-20 20:50:48 -07:00

test_pe_to_pe_latency.py

CCL allreduce: rename to lrab_hierarchical_allreduce + descriptive plots

2026-05-20 20:50:48 -07:00

test_per_pe_hbm_partition.py

ADR housekeeping: category prefixes, lifecycle folders, retroactive 0034-0037

2026-05-20 01:15:55 -07:00

test_phase_a_components.py

Latency model: HBM PC striping + chunk-loop drain (ADR-0033)

2026-05-14 21:59:07 -07:00

test_phyaddr.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_probe.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_recipe_data_mode.py

gqa(adr-0065): N3 — GEMM-epilogue accumulate dataflow (O = O.corr + P.V)

2026-06-10 23:06:01 -07:00

test_routing.py

ADR housekeeping: category prefixes, lifecycle folders, retroactive 0034-0037

2026-05-20 01:15:55 -07:00

test_runtime_api_tensor.py

ADR-0026: DPPolicy intra-device only + ShardSpec structural coords

2026-04-14 13:02:19 -07:00

test_sip_parallel.py

ADR-0026: DPPolicy intra-device only + ShardSpec structural coords

2026-04-14 13:02:19 -07:00

test_sip_topology_rectangular.py

Rectangular SIP topology + 6-device allreduce sweep

2026-04-27 15:13:14 -07:00

test_tensor_free.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_tensor.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_tl_composite_validation.py

gqa(adr-0065): P2 — softmax_merge recipe + TLContext prologue lowering

2026-06-10 19:48:29 -07:00

test_tl_copy_to.py

gqa: ADR-0060/0062/0063/0064 unified GQA kernels + CPU cost model

2026-06-09 18:15:59 -07:00

test_tl_ipcq_api.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_tl_load_lazy.py

gqa: ADR-0060/0062/0063/0064 unified GQA kernels + CPU cost model

2026-06-09 18:15:59 -07:00

test_tl_recipe_softmax_merge.py

gqa(adr-0065): P2 — softmax_merge recipe + TLContext prologue lowering

2026-06-10 19:48:29 -07:00

test_tl_scratch_scope.py

gqa: ADR-0060/0062/0063/0064 unified GQA kernels + CPU cost model

2026-06-09 18:15:59 -07:00

test_topology_compile.py

ADR housekeeping: category prefixes, lifecycle folders, retroactive 0034-0037

2026-05-20 01:15:55 -07:00

test_topology_load.py

Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

2026-04-12 19:36:59 -07:00

test_topology_visualize.py

Cube-view SVG: detailed topology validation rendering

2026-04-04 22:03:38 -07:00

test_tp_layers.py

ADR-0027: Megatron TP API + worker-wait generalization + mp.spawn

2026-04-14 16:31:13 -07:00

test_tp_mlp.py

ADR-0027: Megatron TP API + worker-wait generalization + mp.spawn

2026-04-14 16:31:13 -07:00

test_tp_parallel_state.py

ADR housekeeping: category prefixes, lifecycle folders, retroactive 0034-0037

2026-05-20 01:15:55 -07:00

test_triton_emu.py

gqa(adr-0065): D8 — composite returns output handle + output-space DMA

2026-06-10 22:29:10 -07:00

test_va_allocator.py

Add virtual memory support: PE_MMU, VA allocator, fabric MmuMapMsg

2026-03-26 00:01:47 -07:00

test_va_integration.py

ADR-0001 Rev 2: 51-bit PhysAddr layout with concrete sub-unit tables

2026-04-27 15:52:29 -07:00

test_va_offset.py

benches: package as kernbench.benches, add @bench registry + list subcommand

2026-05-20 14:42:10 -07:00

test_verify_adr_lang_pairs.py

ADR: translate adr-ko/ to Korean, fix ADR-0013 slug, refine Status check

2026-05-20 08:17:56 -07:00

test_wire_cut_through.py

Latency model: HBM PC striping + chunk-loop drain (ADR-0033)

2026-05-14 21:59:07 -07:00