ywkang/kernbench2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gqa(decode-4cases): rename bench/kernels/panels with long_ctx token

Browse Source 
The 4-cases comparative study is specifically about long-context
decode (LLaMA-3.1-70B, S_kv=128K, T_q=1); the long_ctx token in the
names makes the scope explicit and aligns with the existing
_gqa_attention_decode_long convention.

Renames (mechanical; no semantic change):
  - bench file:   milestone_gqa_decode_4cases.py
                  → milestone_gqa_decode_long_ctx_4cases.py
  - bench name:   milestone-gqa-decode-4cases
                  → milestone-gqa-decode-long-ctx-4cases
  - output dir:   1H_milestone_output/gqa_decode_4cases/
                  → 1H_milestone_output/gqa_decode_long_ctx_4cases/
  - env vars:     GQA_DECODE_4CASES_RUN / _TOPOLOGY
                  → GQA_DECODE_LONG_CTX_4CASES_RUN / _TOPOLOGY
  - 4 kernel files _gqa_attention_decode_<case>.py
                  → _gqa_attention_decode_long_ctx_<case>.py
  - 4 kernel functions gqa_attention_decode_<case>_kernel
                  → gqa_attention_decode_long_ctx_<case>_kernel
  - 4 dispatch kinds  decode_<case> → decode_long_ctx_<case>
  - 4 panel names     single_kv_group_decode_gqa_<case>
                      → single_kv_group_decode_long_ctx_gqa_<case>
  - 4 helper functions _run_decode_panel_<case>
                       → _run_decode_panel_long_ctx_<case>
  - test file renamed in lockstep

Also: Case 4 smoke test now uses its case-specific helper
_run_decode_panel_long_ctx_cube_sp_pe_sp (consistent with Cases 1-3)
instead of the legacy _run_decode_panel from milestone_gqa_headline.
16 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Mukesh Garg
2026-06-15 16:09:21 -07:00
parent 3c155be8e6
commit ddee28a499
6 changed files with 123 additions and 143 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/attention/test_milestone_gqa_decode_4cases.py → tests/attention/test_milestone_gqa_decode_long_ctx_4cases.py
+72 -87
View File
@@ -1,4 +1,4 @@
"""Tests for the decode 4-cases comparative-study bench.
"""Tests for the long-context decode 4-cases comparative-study bench.
Per ``GQA_full_deck.pptx`` slides 11-17, the 4 cases differ in how KV
cache is sharded across the 8 cubes and 8 PEs of a single KV-head
@@ -9,28 +9,15 @@ group on LLaMA-3.1-70B GQA:
Case 3 Cube-Repl / PE-SP → full KV per cube; PEs SP on S_kv (intra-cube AR)
Case 4 Cube-SP / PE-SP → KV split 64-way; 2-phase AR on (m,ℓ,O) ★ optimal
This file grows as each case lands. Phase 1 of 5C.D adds Case 4 first,
which is structurally the existing ``_gqa_attention_decode_long.py`` at
``sub_w=4`` (Increment 2's lrab-adapted center-root reduce). The Phase 2
production change introduces a new bench file
``src/kernbench/benches/milestone_gqa_decode_4cases.py`` housing the 4
case panels under a single ``milestone-gqa-decode-4cases`` entry, and
extends ``_run_decode_panel`` in ``milestone_gqa_headline`` to accept
``sub_w``/``d_head``/``h_q``/``h_kv`` overrides.
Deviation from slide 13: slide prescribes AllReduce on (m,ℓ,O); the
kernel does reduce-to-root (only the lrab center cube has the answer)
per ADR-0060 §4. Treated as the kernbench Case-4 baseline.
Phase 1: tests only. T1, T2, T3, T4 fail today (new bench file does
not yet exist; ``_run_decode_panel`` does not yet accept ``sub_w``).
"""
from __future__ import annotations
import re
from pathlib import Path
from kernbench.benches.milestone_gqa_headline import _run_decode_panel
from kernbench.runtime_api.bench_runner import run_bench
from kernbench.runtime_api.types import resolve_device
from kernbench.sim_engine.engine import GraphEngine
@@ -38,10 +25,10 @@ from kernbench.topology.builder import resolve_topology
TOPOLOGY_DEFAULT = Path(__file__).resolve().parents[2] / "topology.yaml"
_CASE4_PANEL = "single_kv_group_decode_gqa_cube_sp_pe_sp"
_CASE3_PANEL = "single_kv_group_decode_gqa_cube_repl_pe_sp"
_CASE2_PANEL = "single_kv_group_decode_gqa_cube_repl_pe_tp"
_CASE1_PANEL = "single_kv_group_decode_gqa_cube_sp_pe_tp"
_CASE4_PANEL = "single_kv_group_decode_long_ctx_gqa_cube_sp_pe_sp"
_CASE3_PANEL = "single_kv_group_decode_long_ctx_gqa_cube_repl_pe_sp"
_CASE2_PANEL = "single_kv_group_decode_long_ctx_gqa_cube_repl_pe_tp"
_CASE1_PANEL = "single_kv_group_decode_long_ctx_gqa_cube_sp_pe_tp"
_CUBE_RE = re.compile(r"\bcube(\d+)\b")
@@ -64,20 +51,26 @@ def _dma_write_cubes(op_log) -> list[int]:
return cubes
# ── Case 4 (Cube-SP × PE-SP) — ★ optimal ────────────────────────────
def _run_case4_smoke(*, S_kv: int):
"""Drive the Case 4 decode panel via ``_run_decode_panel``.
"""Drive the Case 4 decode panel via the case-specific runner.
Uses ``S_kv=8192`` (smoke) to keep test time bounded; the headline
``S_kv=128K`` runs come from ``kernbench run --bench
milestone-gqa-decode-4cases``, not pytest.
milestone-gqa-decode-long-ctx-4cases``, not pytest.
"""
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_run_decode_panel_long_ctx_cube_sp_pe_sp,
)
topo = resolve_topology(str(TOPOLOGY_DEFAULT))
def _bench_fn(ctx):
_run_decode_panel(
_run_decode_panel_long_ctx_cube_sp_pe_sp(
ctx, panel=_CASE4_PANEL,
C=8, P=8, sub_w=4,
S_kv=S_kv,
C=8, P=8,
T_q=1, S_kv=S_kv,
d_head=128, h_q=8, h_kv=1,
)
@@ -88,11 +81,11 @@ def _run_case4_smoke(*, S_kv: int):
)
# ── T1: Case 4 panel is registered in the new bench ──────────────────
# ── Case 4 — T1: panel is registered in the bench ───────────────────
def test_case4_panel_registered():
"""The Case 4 panel must be in the new bench's ``_PANELS`` +
"""The Case 4 panel must be in the bench's ``_PANELS`` +
``_PANEL_DISPATCH`` with the expected LLaMA-3.1-70B target dims.
Headline config:
@@ -103,10 +96,9 @@ def test_case4_panel_registered():
d_head = 128, h_q = 8, h_kv = 1
The lrab sub_w=4 / sub_h=2 geometry is baked into the Case 4
wrapper kernel (``_gqa_attention_decode_cube_sp_pe_sp.py``); it
is not a panel parameter.
wrapper kernel; it is not a panel parameter.
"""
from kernbench.benches.milestone_gqa_decode_4cases import (
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_PANEL_DISPATCH,
_PANELS,
)
@@ -115,8 +107,8 @@ def test_case4_panel_registered():
)
assert _CASE4_PANEL in _PANEL_DISPATCH
kind, params = _PANEL_DISPATCH[_CASE4_PANEL]
assert kind == "decode_cube_sp_pe_sp", (
f"kind={kind!r}, expected 'decode_cube_sp_pe_sp'"
assert kind == "decode_long_ctx_cube_sp_pe_sp", (
f"kind={kind!r}, expected 'decode_long_ctx_cube_sp_pe_sp'"
)
assert params.get("C") == 8
assert params.get("P") == 8
@@ -127,26 +119,19 @@ def test_case4_panel_registered():
assert params.get("h_kv") == 1
# ── T2: Case 4 runner drives the kernel to completion ───────────────
# ── Case 4 — T2: runner drives the kernel to completion ─────────────
def test_case4_runner_smoke():
"""``_run_decode_panel`` must accept the new ``sub_w``,
``d_head``, ``h_q``, ``h_kv`` kwargs and launch the kernel at
``(C, P, sub_w) = (8, 8, 4)`` (the Case 4 / lrab path).
Smoke uses ``S_kv=8192`` so the simulation completes quickly. The
headline 128K dims run via ``kernbench run --bench
milestone-gqa-decode-4cases``.
"""
"""Case 4 runner drives the new kernel to completion at smoke S_kv."""
result = _run_case4_smoke(S_kv=8192)
assert result.completion.ok, (
f"Case 4 decode smoke at C=8 P=8 sub_w=4 must complete; "
f"Case 4 decode smoke at C=8 P=8 must complete; "
f"got {result.completion}"
)
# ── T3: reduce-to-root lands at the lrab center cube (cube 6) ───────
# ── Case 4 — T3: reduce-to-root at the lrab center cube (cube 6) ────
def test_case4_root_at_center_cube_6():
@@ -165,7 +150,37 @@ def test_case4_root_at_center_cube_6():
)
# ── T4: 2-phase AR ipcq pattern matches the predicted Case-4 traffic ─
# ── Case 4 — T4: 2-phase AR ipcq pattern matches Case-4 traffic ─────
def test_case4_two_level_ar_ipcq_pattern():
"""Total ipcq_copy for the Case 4 reduce at (C, P, sub_w) =
(8, 8, 4):
Intra-CUBE (per CUBE = 8 PEs in a 2×4 grid):
row chain along intra_W: cols 1,2,3 each row × 2 rows ×
3 tensors = 18
col bridge along intra_N: pe4 only × 3 tensors = 3
per-CUBE intra total = 21
× 8 CUBEs = 168
Inter-CUBE lrab (sub_w=4, sub_h=2):
Phase 1 row reduce — 3 sends/row × 3 tensors × 2 rows = 18
Phase 2 col reduce — cube 2 → S × 3 tensors = 3
inter-CUBE total = 21
Grand total: 168 + 21 = 189
"""
result = _run_case4_smoke(S_kv=8192)
assert result.completion.ok
n_copy = _count(result.engine.op_log, "ipcq_copy")
assert n_copy == 189, (
f"Case 4 expected 189 ipcq_copy "
f"(168 intra-CUBE + 21 inter-CUBE lrab); got {n_copy}"
)
# ── Case 2 (Cube-Repl × PE-TP) ──────────────────────────────────────
def _run_case2_smoke(*, S_kv: int):
@@ -175,13 +190,13 @@ def _run_case2_smoke(*, S_kv: int):
slide-11 memory waste); for B=1 only one rank does the work; no
inter-rank comm.
"""
from kernbench.benches.milestone_gqa_decode_4cases import (
_run_decode_panel_cube_repl_pe_tp,
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_run_decode_panel_long_ctx_cube_repl_pe_tp,
)
topo = resolve_topology(str(TOPOLOGY_DEFAULT))
def _bench_fn(ctx):
_run_decode_panel_cube_repl_pe_tp(
_run_decode_panel_long_ctx_cube_repl_pe_tp(
ctx, panel=_CASE2_PANEL,
C=8, P=8,
T_q=1, S_kv=S_kv,
@@ -207,7 +222,7 @@ def test_case2_panel_registered():
For B=1 only one rank works (PEs 1-7 idle — slide-11 calls
out this PE-TP waste).
"""
from kernbench.benches.milestone_gqa_decode_4cases import (
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_PANEL_DISPATCH,
_PANELS,
)
@@ -216,7 +231,7 @@ def test_case2_panel_registered():
)
assert _CASE2_PANEL in _PANEL_DISPATCH
kind, params = _PANEL_DISPATCH[_CASE2_PANEL]
assert kind == "decode_cube_repl_pe_tp"
assert kind == "decode_long_ctx_cube_repl_pe_tp"
assert params.get("C") == 8
assert params.get("P") == 8
assert params.get("T_q") == 1
@@ -281,13 +296,13 @@ def _run_case3_smoke(*, S_kv: int):
8-way across PEs within each cube. Intra-CUBE 8-way reduce on
(m, ℓ, O); no inter-CUBE comm (every cube ends with full answer).
"""
from kernbench.benches.milestone_gqa_decode_4cases import (
_run_decode_panel_cube_repl_pe_sp,
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_run_decode_panel_long_ctx_cube_repl_pe_sp,
)
topo = resolve_topology(str(TOPOLOGY_DEFAULT))
def _bench_fn(ctx):
_run_decode_panel_cube_repl_pe_sp(
_run_decode_panel_long_ctx_cube_repl_pe_sp(
ctx, panel=_CASE3_PANEL,
C=8, P=8,
T_q=1, S_kv=S_kv,
@@ -311,7 +326,7 @@ def test_case3_panel_registered():
Case 3: Cube-Repl × PE-SP. K, V replicated per cube; PEs SP on
S_kv. Intra-CUBE 8-way AR; no inter-CUBE comm.
"""
from kernbench.benches.milestone_gqa_decode_4cases import (
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_PANEL_DISPATCH,
_PANELS,
)
@@ -320,7 +335,7 @@ def test_case3_panel_registered():
)
assert _CASE3_PANEL in _PANEL_DISPATCH
kind, params = _PANEL_DISPATCH[_CASE3_PANEL]
assert kind == "decode_cube_repl_pe_sp"
assert kind == "decode_long_ctx_cube_repl_pe_sp"
assert params.get("C") == 8
assert params.get("P") == 8
assert params.get("T_q") == 1
@@ -401,13 +416,13 @@ def _run_case1_smoke(*, S_kv: int):
each cube has work; PEs 1-7 idle. Inter-CUBE 8-way reduce via the
lrab-adapted center-root pattern (root cube 6); no intra-CUBE comm.
"""
from kernbench.benches.milestone_gqa_decode_4cases import (
_run_decode_panel_cube_sp_pe_tp,
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_run_decode_panel_long_ctx_cube_sp_pe_tp,
)
topo = resolve_topology(str(TOPOLOGY_DEFAULT))
def _bench_fn(ctx):
_run_decode_panel_cube_sp_pe_tp(
_run_decode_panel_long_ctx_cube_sp_pe_tp(
ctx, panel=_CASE1_PANEL,
C=8, P=8,
T_q=1, S_kv=S_kv,
@@ -432,7 +447,7 @@ def test_case1_panel_registered():
batch. At B=1 only PE 0 of each cube works (PE-TP waste).
Inter-CUBE lrab AR; no intra-CUBE comm.
"""
from kernbench.benches.milestone_gqa_decode_4cases import (
from kernbench.benches.milestone_gqa_decode_long_ctx_4cases import (
_PANEL_DISPATCH,
_PANELS,
)
@@ -441,7 +456,7 @@ def test_case1_panel_registered():
)
assert _CASE1_PANEL in _PANEL_DISPATCH
kind, params = _PANEL_DISPATCH[_CASE1_PANEL]
assert kind == "decode_cube_sp_pe_tp"
assert kind == "decode_long_ctx_cube_sp_pe_tp"
assert params.get("C") == 8
assert params.get("P") == 8
assert params.get("T_q") == 1
@@ -507,33 +522,3 @@ def test_case1_root_at_center_cube_6():
f"Case 1 root must be the lrab center cube 6; "
f"got cubes={sorted(distinct)}"
)
# ── Case 4 — T4 (existing, kept) ────────────────────────────────────
def test_case4_two_level_ar_ipcq_pattern():
"""Total ipcq_copy for the Case 4 reduce at (C, P, sub_w) =
(8, 8, 4):
Intra-CUBE (per CUBE = 8 PEs in a 2×4 grid):
row chain along intra_W: cols 1,2,3 each row × 2 rows ×
3 tensors = 18
col bridge along intra_N: pe4 only × 3 tensors = 3
per-CUBE intra total = 21
× 8 CUBEs = 168
Inter-CUBE lrab (sub_w=4, sub_h=2):
Phase 1 row reduce — 3 sends/row × 3 tensors × 2 rows = 18
Phase 2 col reduce — cube 2 → S × 3 tensors = 3
inter-CUBE total = 21
Grand total: 168 + 21 = 189
"""
result = _run_case4_smoke(S_kv=8192)
assert result.completion.ok
n_copy = _count(result.engine.op_log, "ipcq_copy")
assert n_copy == 189, (
f"Case 4 expected 189 ipcq_copy "
f"(168 intra-CUBE + 21 inter-CUBE lrab); got {n_copy}"
)