kernbench2/docs/sweeps/short_context_decode_composite_sweep.csv at master

Files

T

ywkang 711a9a257f gqa short-context: exclude KV deploy from wall; correct HBM peak; batch scaffold

Metric fixes (test harnesses, deploy artifact + wrong constant):
- wall = max(t_end) - min(t_start): exclude the one-time KV-cache deploy
  from the measured decode/prefill step (it was 92-99% of wall, mapping-
  invariant, and masked the real per-mapping separation).
- PEAK_PE_HBM_BPS 128 -> 256 B/ns (8 channels/PE x 32 GB/s); the old value
  was half the modeled per-PE HBM BW, so hbm_bw_util read >1.0 once wall
  was corrected. All six short-context sweep CSVs regenerated/repatched.

Result: the four KV mappings now separate along a {64,32,16,8}-active-PE
ladder (decode 8-kv/1-kv = 4.8x at 8K, 7.4x at 64K), not "modest/tied" as
before; decode is bandwidth-bound at 46-76% of the 256 GB/s per-PE ceiling.

Report (S5.2 rewrite):
- Replace the tied-wall / 8x-per-PE-util claims (both deploy artifacts)
  with the corrected separation and a density trade-off (per-CUBE KV
  footprint, Fig 16 top panel switched to a wall-invariant metric).
- Add a projected latency-vs-batched-throughput analysis (marked
  projected, not measured): dense mappings win throughput, 1-kv wins
  latency; converges at long context.
- Regenerate Fig 15/16/17/18; fix plot script hardcoded ROOT path.

Batch experiment (Part 2, cube_base):
- Add backward-compatible cube_base=0 scalar to the decode kernel so a
  batched user placed at DPPolicy.cube_start addresses 0-based shards.
  Default preserves single-user behavior (32 decode tests pass unchanged).
- New batch harness (skipped): concurrent B>=2 launches hit a sim routing
  issue (sip0.cube0.pe8); single-user path verified. Concurrency fix next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-22 16:25:04 -07:00

1.8 KiB

Raw Permalink Blame History

1	variant	mode	kv_per_cube	C	S_kv	wall_us	n_pe	gemm_util	math_count	hbm_bw_util	hbm_read_mb	hbm_write_kb	ipcq_kb	kv_cache_per_cube_mb
2	composite	A1	1	8	8192	2.532214500000584	64	0.33402541530150626	656	0.406166	16.0625	8.0	57.75	2.0
3	composite	A1	1	8	16384	4.176996500001463	64	0.40499148132408613	1360	0.491382	32.0625	8.0	57.75	4.0
4	composite	A1	1	8	32768	7.466560500002	64	0.4531264428807111	2768	0.549182	64.0625	8.0	57.75	8.0
5	composite	A1	1	8	65536	14.045688499950572	64	0.4817558071541797	5584	0.58356	128.0625	8.0	57.75	16.0
6	composite	A2	2	4	8192	3.9256127500006404	32	0.4309258471789165	656	0.522976	16.03125	8.0	24.75	4.0
7	composite	A2	2	4	16384	7.215176750001498	32	0.4689138072801237	1360	0.568385	32.03125	8.0	24.75	8.0
8	composite	A2	2	4	32768	13.794304750002688	32	0.4905351971318266	2768	0.594231	64.03125	8.0	24.75	16.0
9	composite	A2	2	4	65536	26.95256074991636	32	0.5021112511805295	5584	0.608068	128.03125	8.0	24.75	32.0
10	composite	A4	4	2	8192	6.726335500000743	16	0.5029924540606681	656	0.609842	16.015625	8.0	8.25	8.0
11	composite	A4	4	2	16384	13.305463500001585	16	0.5085574057667106	1360	0.616138	32.015625	8.0	8.25	16.0
12	composite	A4	4	2	32768	26.463719500003965	16	0.5113863151276257	2768	0.619338	64.015625	8.0	8.25	32.0
13	composite	A4	4	2	65536	52.78023149984703	16	0.5128126048745716	5584	0.620952	128.015625	8.0	8.25	64.0
14	composite	B	8	1	8192	13.096491000001318	8	0.5166721375947838	656	0.626122	16.0078125	8.0	0.0	16.0
15	composite	B	8	1	16384	26.254747000003057	8	0.5154566524738308	1360	0.624344	32.0078125	8.0	0.0	32.0
16	composite	B	8	1	32768	52.57125900000788	8	0.5148510519664782	2768	0.623459	64.0078125	8.0	0.0	64.0
17	composite	B	8	1	65536	105.20428299969761	8	0.514548785079354	5584	0.623016	128.0078125	8.0	0.0	128.0

1.8 KiB Raw Permalink Blame History

1.8 KiB

Raw Permalink Blame History