topology: set flit_bytes=4096 (was implicit 256 default)
Wire per-flit loop in engine._wire (ADR-0033 Phase 2c) iterates
nbytes/flit_bytes times per hop, firing ~2 SimPy events per flit.
At long-context decode, K/V tile loads are hundreds of KB per shard,
producing thousands of flit-iterations per hop. The default 256-byte
flit size made the wire loop the dominant wall-clock cost.
Raising flit_bytes to 4096 (still a multiple of hbm_ctrl.burst_bytes=256
per ADR-0033 D1) cuts flit iterations 16x with no meaningful change to
modeled latency. Wormhole pipelining and the available_at BW-share
chain in _wire behave identically at any flit size.
Measured impact on Case 4 (Cube-SP x PE-SP) decode, C=8, P=8, T_q=1,
LLaMA-3.1-70B single-KV-group, enable_data=False:
S_kv flit=256 wall flit=4096 wall speedup latency drift
128K 233.71 s 18.16 s 12.9x +0.05% (0.25us)
256K 485.75 s 25.31 s 19.2x +0.01% (0.09us)
512K 1045.64 s 48.45 s 21.6x +0.01% (0.17us)
1M ~2250 s (proj) 104.75 s ~21x within extrap
Total sweep at flit=4096: ~3.3 min. Baseline would have been ~65 min.
1M decode iteration is now practical.
op_log records are byte-identical (same counts, same components); the
tiny latency drift (order of a few ns) comes from the last flit's
env.timeout landing on a slightly different granularity when flits are
larger. Well within noise for any test that asserts on structural
invariants (dma_write cubes, ipcq_copy counts).
Verified:
- tests/attention/test_milestone_gqa_decode_long_ctx_4cases.py: 18/18 pass
(also 7x faster: 174s -> 24.83s per full run)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,7 @@
|
|||||||
|
|
||||||
system:
|
system:
|
||||||
ns_per_mm: 0.01 # wire propagation delay: 10 ps/mm (on-chip silicon)
|
ns_per_mm: 0.01 # wire propagation delay: 10 ps/mm (on-chip silicon)
|
||||||
|
flit_bytes: 4096 # wire chunk size (ADR-0033 D1: multiple of hbm_ctrl.burst_bytes=256)
|
||||||
|
|
||||||
sips:
|
sips:
|
||||||
count: 2
|
count: 2
|
||||||
|
|||||||
Reference in New Issue
Block a user