gqa short-context: exclude KV deploy from wall; correct HBM peak; batch scaffold

Metric fixes (test harnesses, deploy artifact + wrong constant):
- wall = max(t_end) - min(t_start): exclude the one-time KV-cache deploy
  from the measured decode/prefill step (it was 92-99% of wall, mapping-
  invariant, and masked the real per-mapping separation).
- PEAK_PE_HBM_BPS 128 -> 256 B/ns (8 channels/PE x 32 GB/s); the old value
  was half the modeled per-PE HBM BW, so hbm_bw_util read >1.0 once wall
  was corrected. All six short-context sweep CSVs regenerated/repatched.

Result: the four KV mappings now separate along a {64,32,16,8}-active-PE
ladder (decode 8-kv/1-kv = 4.8x at 8K, 7.4x at 64K), not "modest/tied" as
before; decode is bandwidth-bound at 46-76% of the 256 GB/s per-PE ceiling.

Report (S5.2 rewrite):
- Replace the tied-wall / 8x-per-PE-util claims (both deploy artifacts)
  with the corrected separation and a density trade-off (per-CUBE KV
  footprint, Fig 16 top panel switched to a wall-invariant metric).
- Add a projected latency-vs-batched-throughput analysis (marked
  projected, not measured): dense mappings win throughput, 1-kv wins
  latency; converges at long context.
- Regenerate Fig 15/16/17/18; fix plot script hardcoded ROOT path.

Batch experiment (Part 2, cube_base):
- Add backward-compatible cube_base=0 scalar to the decode kernel so a
  batched user placed at DPPolicy.cube_start addresses 0-based shards.
  Default preserves single-user behavior (32 decode tests pass unchanged).
- New batch harness (skipped): concurrent B>=2 launches hit a sim routing
  issue (sip0.cube0.pe8); single-user path verified. Concurrency fix next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-22 16:25:04 -07:00
parent cb6d21f145
commit 711a9a257f
23 changed files with 403 additions and 187 deletions
@@ -1,17 +1,17 @@
variant,mode,kv_per_cube,C,S_kv,wall_us,n_pe,gemm_util,math_count,math_pipeline_us,hbm_bw_util,hbm_read_mb,hbm_write_kb,ipcq_kb,kv_cache_per_cube_mb
composite,A1,1,8,8192,147.23658450000076,64,0.0057446592018384314,656,0.0,0.013970712557516501,16.0625,8.0,57.75,2.0
composite,A1,1,8,16384,279.9533665000026,64,0.006042606385378475,1360,0.0,0.014663156408229735,32.0625,8.0,57.75,4.0
composite,A1,1,8,32768,545.3869305000062,64,0.006203478321011016,2768,0.0,0.015037030668265908,64.0625,8.0,57.75,8.0
composite,A1,1,8,65536,1076.3614585003581,64,0.006286542449928883,5584,0.0,0.015230013923798023,128.0625,8.0,57.75,16.0
composite,A2,2,4,8192,140.44093275000077,32,0.012045263206857853,656,0.0,0.02923649052736712,16.03125,8.0,24.75,4.0
composite,A2,2,4,16384,274.80249675000204,32,0.012311736756599198,1360,0.0,0.029846890392199246,32.03125,8.0,24.75,8.0
composite,A2,2,4,32768,543.6330247504077,32,0.012446984807344498,2768,0.0,0.030156372504276757,64.03125,8.0,24.75,16.0
composite,A2,2,4,65536,1082.0090007518106,32,0.012507459726542472,5584,0.0,0.030293648183356066,128.03125,8.0,24.75,32.0
composite,A4,4,2,8192,141.0397155000009,16,0.023988250316491475,656,0.0,0.05816801296653173,16.015625,8.0,8.25,8.0
composite,A4,4,2,16384,278.7445435001758,16,0.02427524469220757,1360,0.0,0.05882088235384476,32.015625,8.0,8.25,16.0
composite,A4,4,2,32768,554.51165950104,16,0.024405589617097562,2768,0.0,0.059115077994024615,64.015625,8.0,8.25,32.0
composite,A4,4,2,65536,1106.4268915018213,16,0.024462861675913012,5584,0.0,0.05924295631591859,128.015625,8.0,8.25,64.0
composite,B,8,1,8192,146.3478710000607,8,0.0462363541999099,656,0.0,0.11206176002378058,16.0078125,8.0,0.0,16.0
composite,B,8,1,16384,290.8105570004271,8,0.046536082251474285,1360,0.0,0.1127331838917796,32.0078125,8.0,0.0,32.0
composite,B,8,1,32768,579.9264290010428,8,0.046672071914328234,2768,0.0,0.11303502775846438,64.0078125,8.0,0.0,64.0
composite,B,8,1,65536,1158.158173001712,8,0.04674036523209764,5584,0.0,0.11318661220534877,128.0078125,8.0,0.0,128.0
composite,A1,1,8,8192,2.532214500000584,64,0.33402541530150626,656,0.0,0.406166,16.0625,8.0,57.75,2.0
composite,A1,1,8,16384,4.176996500001463,64,0.40499148132408613,1360,0.0,0.491382,32.0625,8.0,57.75,4.0
composite,A1,1,8,32768,7.466560500002,64,0.4531264428807111,2768,0.0,0.549182,64.0625,8.0,57.75,8.0
composite,A1,1,8,65536,14.045688499950572,64,0.4817558071541797,5584,0.0,0.58356,128.0625,8.0,57.75,16.0
composite,A2,2,4,8192,3.9256127500006404,32,0.4309258471789165,656,0.0,0.522976,16.03125,8.0,24.75,4.0
composite,A2,2,4,16384,7.215176750001498,32,0.4689138072801237,1360,0.0,0.568385,32.03125,8.0,24.75,8.0
composite,A2,2,4,32768,13.794304750002688,32,0.4905351971318266,2768,0.0,0.594231,64.03125,8.0,24.75,16.0
composite,A2,2,4,65536,26.95256074991636,32,0.5021112511805295,5584,0.0,0.608068,128.03125,8.0,24.75,32.0
composite,A4,4,2,8192,6.726335500000743,16,0.5029924540606681,656,0.0,0.609842,16.015625,8.0,8.25,8.0
composite,A4,4,2,16384,13.305463500001585,16,0.5085574057667106,1360,0.0,0.616138,32.015625,8.0,8.25,16.0
composite,A4,4,2,32768,26.463719500003965,16,0.5113863151276257,2768,0.0,0.619338,64.015625,8.0,8.25,32.0
composite,A4,4,2,65536,52.78023149984703,16,0.5128126048745716,5584,0.0,0.620952,128.015625,8.0,8.25,64.0
composite,B,8,1,8192,13.096491000001318,8,0.5166721375947838,656,0.0,0.626122,16.0078125,8.0,0.0,16.0
composite,B,8,1,16384,26.254747000003057,8,0.5154566524738308,1360,0.0,0.624344,32.0078125,8.0,0.0,32.0
composite,B,8,1,32768,52.57125900000788,8,0.5148510519664782,2768,0.0,0.623459,64.0078125,8.0,0.0,64.0
composite,B,8,1,65536,105.20428299969761,8,0.514548785079354,5584,0.0,0.623016,128.0078125,8.0,0.0,128.0
1 variant mode kv_per_cube C S_kv wall_us n_pe gemm_util math_count math_pipeline_us hbm_bw_util hbm_read_mb hbm_write_kb ipcq_kb kv_cache_per_cube_mb
2 composite A1 1 8 8192 147.23658450000076 2.532214500000584 64 0.0057446592018384314 0.33402541530150626 656 0.0 0.013970712557516501 0.406166 16.0625 8.0 57.75 2.0
3 composite A1 1 8 16384 279.9533665000026 4.176996500001463 64 0.006042606385378475 0.40499148132408613 1360 0.0 0.014663156408229735 0.491382 32.0625 8.0 57.75 4.0
4 composite A1 1 8 32768 545.3869305000062 7.466560500002 64 0.006203478321011016 0.4531264428807111 2768 0.0 0.015037030668265908 0.549182 64.0625 8.0 57.75 8.0
5 composite A1 1 8 65536 1076.3614585003581 14.045688499950572 64 0.006286542449928883 0.4817558071541797 5584 0.0 0.015230013923798023 0.58356 128.0625 8.0 57.75 16.0
6 composite A2 2 4 8192 140.44093275000077 3.9256127500006404 32 0.012045263206857853 0.4309258471789165 656 0.0 0.02923649052736712 0.522976 16.03125 8.0 24.75 4.0
7 composite A2 2 4 16384 274.80249675000204 7.215176750001498 32 0.012311736756599198 0.4689138072801237 1360 0.0 0.029846890392199246 0.568385 32.03125 8.0 24.75 8.0
8 composite A2 2 4 32768 543.6330247504077 13.794304750002688 32 0.012446984807344498 0.4905351971318266 2768 0.0 0.030156372504276757 0.594231 64.03125 8.0 24.75 16.0
9 composite A2 2 4 65536 1082.0090007518106 26.95256074991636 32 0.012507459726542472 0.5021112511805295 5584 0.0 0.030293648183356066 0.608068 128.03125 8.0 24.75 32.0
10 composite A4 4 2 8192 141.0397155000009 6.726335500000743 16 0.023988250316491475 0.5029924540606681 656 0.0 0.05816801296653173 0.609842 16.015625 8.0 8.25 8.0
11 composite A4 4 2 16384 278.7445435001758 13.305463500001585 16 0.02427524469220757 0.5085574057667106 1360 0.0 0.05882088235384476 0.616138 32.015625 8.0 8.25 16.0
12 composite A4 4 2 32768 554.51165950104 26.463719500003965 16 0.024405589617097562 0.5113863151276257 2768 0.0 0.059115077994024615 0.619338 64.015625 8.0 8.25 32.0
13 composite A4 4 2 65536 1106.4268915018213 52.78023149984703 16 0.024462861675913012 0.5128126048745716 5584 0.0 0.05924295631591859 0.620952 128.015625 8.0 8.25 64.0
14 composite B 8 1 8192 146.3478710000607 13.096491000001318 8 0.0462363541999099 0.5166721375947838 656 0.0 0.11206176002378058 0.626122 16.0078125 8.0 0.0 16.0
15 composite B 8 1 16384 290.8105570004271 26.254747000003057 8 0.046536082251474285 0.5154566524738308 1360 0.0 0.1127331838917796 0.624344 32.0078125 8.0 0.0 32.0
16 composite B 8 1 32768 579.9264290010428 52.57125900000788 8 0.046672071914328234 0.5148510519664782 2768 0.0 0.11303502775846438 0.623459 64.0078125 8.0 0.0 64.0
17 composite B 8 1 65536 1158.158173001712 105.20428299969761 8 0.04674036523209764 0.514548785079354 5584 0.0 0.11318661220534877 0.623016 128.0078125 8.0 0.0 128.0