gqa short-context: exclude KV deploy from wall; correct HBM peak; batch scaffold

Metric fixes (test harnesses, deploy artifact + wrong constant):
- wall = max(t_end) - min(t_start): exclude the one-time KV-cache deploy
  from the measured decode/prefill step (it was 92-99% of wall, mapping-
  invariant, and masked the real per-mapping separation).
- PEAK_PE_HBM_BPS 128 -> 256 B/ns (8 channels/PE x 32 GB/s); the old value
  was half the modeled per-PE HBM BW, so hbm_bw_util read >1.0 once wall
  was corrected. All six short-context sweep CSVs regenerated/repatched.

Result: the four KV mappings now separate along a {64,32,16,8}-active-PE
ladder (decode 8-kv/1-kv = 4.8x at 8K, 7.4x at 64K), not "modest/tied" as
before; decode is bandwidth-bound at 46-76% of the 256 GB/s per-PE ceiling.

Report (S5.2 rewrite):
- Replace the tied-wall / 8x-per-PE-util claims (both deploy artifacts)
  with the corrected separation and a density trade-off (per-CUBE KV
  footprint, Fig 16 top panel switched to a wall-invariant metric).
- Add a projected latency-vs-batched-throughput analysis (marked
  projected, not measured): dense mappings win throughput, 1-kv wins
  latency; converges at long context.
- Regenerate Fig 15/16/17/18; fix plot script hardcoded ROOT path.

Batch experiment (Part 2, cube_base):
- Add backward-compatible cube_base=0 scalar to the decode kernel so a
  batched user placed at DPPolicy.cube_start addresses 0-based shards.
  Default preserves single-user behavior (32 decode tests pass unchanged).
- New batch harness (skipped): concurrent B>=2 launches hit a sim routing
  issue (sip0.cube0.pe8); single-user path verified. Concurrency fix next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-22 16:25:04 -07:00
parent cb6d21f145
commit 711a9a257f
23 changed files with 403 additions and 187 deletions
@@ -1,17 +1,17 @@
variant,mode,kv_per_cube,C,S_kv,wall_us,n_pe,gemm_util,math_count,math_pipeline_us,hbm_bw_util,hbm_read_mb,hbm_write_kb,ipcq_kb,kv_cache_per_cube_mb
composite,A1,1,8,8192,1101.224596999844,64,0.006144606666764116,5248,0.0,0.0018742770599413814,16.0625,64.0,114688.0,2.0
composite,A1,1,8,16384,2178.2218819663735,64,0.006212950164859459,10880,0.0,0.0018877782993750494,32.0625,64.0,229376.0,4.0
composite,A1,1,8,32768,4334.841731847132,64,0.006243911466154239,22144,0.0,0.0018934947358510517,64.0625,64.0,458752.0,8.0
composite,A1,1,8,65536,8649.585161613555,64,0.006258419909799432,44672,0.0,0.0018960446881062475,128.0625,64.0,917504.0,16.0
composite,A2,2,4,8192,1094.5704665014161,32,0.007139940496760185,2624,0.0,0.0037713423907684606,16.0625,64.0,49152.0,4.0
composite,A2,2,4,16384,2177.5659939366815,32,0.007177893135813299,5440,0.0,0.0037766938053309506,32.0625,64.0,98304.0,8.0
composite,A2,2,4,32768,4341.645937350417,32,0.007200189155116074,11072,0.0,0.0037810545209999826,64.0625,64.0,196608.0,16.0
composite,A2,2,4,65536,8671.42409207618,32,0.007210043393449057,22336,0.0,0.003782539021470782,128.0625,64.0,393216.0,32.0
composite,A4,4,2,8192,1104.2310135028845,16,0.008976672344028575,1312,0.0,0.007476696360673657,16.0625,64.0,16384.0,8.0
composite,A4,4,2,16384,2197.6415934265688,16,0.009020870400324496,2720,0.0,0.0074843869215062645,32.0625,64.0,32768.0,16.0
composite,A4,4,2,32768,4384.414073368468,16,0.00904323344692278,5536,0.0,0.007488343813013936,64.0625,64.0,65536.0,32.0
composite,A4,4,2,65536,8757.965852885032,16,0.009054449551299243,11168,0.0,0.0074903237922982055,128.0625,64.0,131072.0,64.0
composite,B,8,1,8192,1125.5021760018114,8,0.020408994750147753,656,0.0,0.014670784608037422,16.0625,64.0,0.0,16.0
composite,B,8,1,16384,2242.2356159431683,8,0.02048880843562789,1360,0.0,0.014671071927542597,32.0625,64.0,0.0,32.0
composite,B,8,1,32768,4475.702495825993,8,0.020528949826292484,2768,0.0,0.014671216431663579,64.0625,64.0,0.0,64.0
composite,B,8,1,65536,8942.63625560376,8,0.020549079566244906,5584,0.0,0.014671288896245289,128.0625,64.0,0.0,128.0
composite,A1,1,8,8192,35.312546999435405,64,0.19162004939598534,5248,0.0,0.0292247,16.0625,64.0,114688.0,2.0
composite,A1,1,8,16384,62.80411199463298,64,0.21548245124172694,10880,0.0,0.0327367,32.0625,64.0,229376.0,4.0
composite,A1,1,8,32768,117.78396202135924,64,0.2297967187462748,22144,0.0,0.0348435,64.0625,64.0,458752.0,8.0
composite,A1,1,8,65536,227.7376320068799,64,0.23769780826259174,44672,0.0,0.0360063,128.0625,64.0,917504.0,16.0
composite,A2,2,4,8192,35.873466499721395,32,0.21785371648964683,2624,0.0,0.0575356,16.0625,64.0,49152.0,4.0
composite,A2,2,4,16384,67.67155399355386,32,0.23097350479268527,5440,0.0,0.0607641,32.0625,64.0,98304.0,8.0
composite,A2,2,4,32768,129.35661752446276,32,0.24166271963282046,11072,0.0,0.0634525,64.0625,64.0,196608.0,16.0
composite,A2,2,4,65536,254.34501247150266,32,0.24581313146006306,22336,0.0,0.0644793,128.0625,64.0,393216.0,32.0
composite,A4,4,2,8192,46.88723350110906,16,0.2114076532174909,1312,0.0,0.088041,16.0625,64.0,16384.0,8.0
composite,A4,4,2,16384,89.10037348339473,16,0.22249783278807841,2720,0.0,0.0923004,32.0625,64.0,32768.0,16.0
composite,A4,4,2,32768,173.47797354247328,16,0.22855512537871095,5536,0.0,0.0946287,64.0625,64.0,65536.0,32.0
composite,A4,4,2,65536,342.2399932809714,16,0.23170453934016594,11168,0.0,0.0958392,128.0625,64.0,131072.0,64.0
composite,B,8,1,8192,68.73804599999497,8,0.3341725483628459,656,0.0,0.120108,16.0625,64.0,0.0,16.0
composite,B,8,1,16384,134.27404599999218,8,0.3421415930417734,1360,0.0,0.122496,32.0625,64.0,0.0,32.0
composite,B,8,1,32768,265.34604599999824,8,0.3462703641501576,2768,0.0,0.123733,64.0625,64.0,0.0,64.0
composite,B,8,1,65536,527.4900460000299,8,0.3483723443539211,5584,0.0,0.124363,128.0625,64.0,0.0,128.0
1 variant mode kv_per_cube C S_kv wall_us n_pe gemm_util math_count math_pipeline_us hbm_bw_util hbm_read_mb hbm_write_kb ipcq_kb kv_cache_per_cube_mb
2 composite A1 1 8 8192 1101.224596999844 35.312546999435405 64 0.006144606666764116 0.19162004939598534 5248 0.0 0.0018742770599413814 0.0292247 16.0625 64.0 114688.0 2.0
3 composite A1 1 8 16384 2178.2218819663735 62.80411199463298 64 0.006212950164859459 0.21548245124172694 10880 0.0 0.0018877782993750494 0.0327367 32.0625 64.0 229376.0 4.0
4 composite A1 1 8 32768 4334.841731847132 117.78396202135924 64 0.006243911466154239 0.2297967187462748 22144 0.0 0.0018934947358510517 0.0348435 64.0625 64.0 458752.0 8.0
5 composite A1 1 8 65536 8649.585161613555 227.7376320068799 64 0.006258419909799432 0.23769780826259174 44672 0.0 0.0018960446881062475 0.0360063 128.0625 64.0 917504.0 16.0
6 composite A2 2 4 8192 1094.5704665014161 35.873466499721395 32 0.007139940496760185 0.21785371648964683 2624 0.0 0.0037713423907684606 0.0575356 16.0625 64.0 49152.0 4.0
7 composite A2 2 4 16384 2177.5659939366815 67.67155399355386 32 0.007177893135813299 0.23097350479268527 5440 0.0 0.0037766938053309506 0.0607641 32.0625 64.0 98304.0 8.0
8 composite A2 2 4 32768 4341.645937350417 129.35661752446276 32 0.007200189155116074 0.24166271963282046 11072 0.0 0.0037810545209999826 0.0634525 64.0625 64.0 196608.0 16.0
9 composite A2 2 4 65536 8671.42409207618 254.34501247150266 32 0.007210043393449057 0.24581313146006306 22336 0.0 0.003782539021470782 0.0644793 128.0625 64.0 393216.0 32.0
10 composite A4 4 2 8192 1104.2310135028845 46.88723350110906 16 0.008976672344028575 0.2114076532174909 1312 0.0 0.007476696360673657 0.088041 16.0625 64.0 16384.0 8.0
11 composite A4 4 2 16384 2197.6415934265688 89.10037348339473 16 0.009020870400324496 0.22249783278807841 2720 0.0 0.0074843869215062645 0.0923004 32.0625 64.0 32768.0 16.0
12 composite A4 4 2 32768 4384.414073368468 173.47797354247328 16 0.00904323344692278 0.22855512537871095 5536 0.0 0.007488343813013936 0.0946287 64.0625 64.0 65536.0 32.0
13 composite A4 4 2 65536 8757.965852885032 342.2399932809714 16 0.009054449551299243 0.23170453934016594 11168 0.0 0.0074903237922982055 0.0958392 128.0625 64.0 131072.0 64.0
14 composite B 8 1 8192 1125.5021760018114 68.73804599999497 8 0.020408994750147753 0.3341725483628459 656 0.0 0.014670784608037422 0.120108 16.0625 64.0 0.0 16.0
15 composite B 8 1 16384 2242.2356159431683 134.27404599999218 8 0.02048880843562789 0.3421415930417734 1360 0.0 0.014671071927542597 0.122496 32.0625 64.0 0.0 32.0
16 composite B 8 1 32768 4475.702495825993 265.34604599999824 8 0.020528949826292484 0.3462703641501576 2768 0.0 0.014671216431663579 0.123733 64.0625 64.0 0.0 64.0
17 composite B 8 1 65536 8942.63625560376 527.4900460000299 8 0.020549079566244906 0.3483723443539211 5584 0.0 0.014671288896245289 0.124363 128.0625 64.0 0.0 128.0