gqa(decode-4cases): Case 4 anchor in dedicated bench (5C.D)
First milestone of the decode 4-cases comparative study per GQA_full_deck.pptx slides 11-17. Case 4 (Cube-SP × PE-SP, the optimal case per slide 11) is structurally the existing _gqa_attention_decode_long at sub_w=4 (Increment 2's lrab-adapted center-root reduce). This commit wires it into a dedicated bench so the remaining cases land alongside. Changes: - New bench: src/kernbench/benches/milestone_gqa_decode_4cases.py Houses the 4 case panels under one milestone-gqa-decode-4cases entry (gated by GQA_DECODE_4CASES_RUN=1; output to 1H_milestone_output/gqa_decode_4cases/sweep.json). Cases 1-3 are TBD in subsequent sub-increments (5C.A/B/C). - New panel: single_kv_group_decode_gqa_cube_sp_pe_sp C=8, P=8, sub_w=4, T_q=1, S_kv=131_072, d_head=128, h_q=8, h_kv=1. - src/kernbench/benches/milestone_gqa_headline.py: _run_decode_panel extended with keyword-only sub_w/T_q/d_head/h_q/h_kv overrides (defaults preserve existing-panel behaviour). - tests/attention/test_milestone_gqa_decode_4cases.py: 4 new tests asserting registration, smoke completion, reduce-to-root at the lrab center cube (cube 6), and the predicted 189-ipcq Case-4 traffic pattern (168 intra-CUBE + 21 inter-CUBE lrab Phase 1+2). - tests/attention/test_milestone_gqa_headline.py: rename test_sweep_json_has_four_panels -> test_sweep_json_has_expected_panels and switch hardcoded 4 to len(PANELS) (the panel set grew to 5 with Increment 5's single_kv_group_prefill_gqa_c8_p8). Deviation noted: slide 13 prescribes AllReduce on (m,ℓ,O); our kernel does reduce-to-root (only the lrab center cube has the answer) per ADR-0060 §4. Treated as the kernbench Case-4 baseline. Verification: all 4 new tests pass; 90 regression tests pass; the previously-failing test_sweep_json_has_four_panels now passes under its renamed form. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -37,6 +37,7 @@ BENCH_NAME = "milestone-gqa-headline"
|
||||
PANELS = (
|
||||
"single_user_prefill_gqa",
|
||||
"multi_user_prefill_gqa",
|
||||
"single_kv_group_prefill_gqa_c8_p8",
|
||||
"single_user_decode_gqa",
|
||||
"multi_user_decode_gqa",
|
||||
)
|
||||
@@ -88,12 +89,12 @@ def test_validation_run_completes_ok(monkeypatch):
|
||||
# ── sweep.json shape ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_sweep_json_has_four_panels(monkeypatch):
|
||||
def test_sweep_json_has_expected_panels(monkeypatch):
|
||||
data = _sweep_json(monkeypatch)
|
||||
assert set(data["panels"]) == set(PANELS), (
|
||||
f"panels mismatch: expected {set(PANELS)}, got {set(data['panels'])}"
|
||||
)
|
||||
assert len(data["rows"]) == 4
|
||||
assert len(data["rows"]) == len(PANELS)
|
||||
assert {r["panel"] for r in data["rows"]} == set(PANELS)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user