gqa short-context: exclude KV deploy from wall; correct HBM peak; batch scaffold

Metric fixes (test harnesses, deploy artifact + wrong constant):
- wall = max(t_end) - min(t_start): exclude the one-time KV-cache deploy
  from the measured decode/prefill step (it was 92-99% of wall, mapping-
  invariant, and masked the real per-mapping separation).
- PEAK_PE_HBM_BPS 128 -> 256 B/ns (8 channels/PE x 32 GB/s); the old value
  was half the modeled per-PE HBM BW, so hbm_bw_util read >1.0 once wall
  was corrected. All six short-context sweep CSVs regenerated/repatched.

Result: the four KV mappings now separate along a {64,32,16,8}-active-PE
ladder (decode 8-kv/1-kv = 4.8x at 8K, 7.4x at 64K), not "modest/tied" as
before; decode is bandwidth-bound at 46-76% of the 256 GB/s per-PE ceiling.

Report (S5.2 rewrite):
- Replace the tied-wall / 8x-per-PE-util claims (both deploy artifacts)
  with the corrected separation and a density trade-off (per-CUBE KV
  footprint, Fig 16 top panel switched to a wall-invariant metric).
- Add a projected latency-vs-batched-throughput analysis (marked
  projected, not measured): dense mappings win throughput, 1-kv wins
  latency; converges at long context.
- Regenerate Fig 15/16/17/18; fix plot script hardcoded ROOT path.

Batch experiment (Part 2, cube_base):
- Add backward-compatible cube_base=0 scalar to the decode kernel so a
  batched user placed at DPPolicy.cube_start addresses 0-based shards.
  Default preserves single-user behavior (32 decode tests pass unchanged).
- New batch harness (skipped): concurrent B>=2 launches hit a sim routing
  issue (sip0.cube0.pe8); single-user path verified. Concurrency fix next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-22 16:25:04 -07:00
parent cb6d21f145
commit 711a9a257f
23 changed files with 403 additions and 187 deletions
@@ -25,7 +25,7 @@ from tests.attention.test_gqa_short_context import (
D_HEAD, H_KV, P, TILE_S_KV, _run_decode,
)
PEAK_PE_HBM_BPS = 1024.0 / 8 # 128 bytes/ns
PEAK_PE_HBM_BPS = 8 * 32.0 # 8 channels/PE x 32 GB/s = 256 bytes/ns per PE
MATH_OPS = {"max", "sum", "exp", "sub", "mul_bcast", "add", "div",
"maximum", "minimum", "neg", "rmax", "rsum", "exp_diff",
@@ -46,7 +46,7 @@ CSV_OUT = (Path(__file__).resolve().parents[2]
def _compute_metrics(r, *, mode, kv_per_cube, C, S_kv):
op_log = r.engine.op_log
wall_ns = max(rec.t_end for rec in op_log)
wall_ns = max(rec.t_end for rec in op_log) - min(rec.t_start for rec in op_log)
active_pes = set()
for rec in op_log: