From 09e9331026887123aaa772dd5c7bd0363c793ffb Mon Sep 17 00:00:00 2001 From: Mukesh Garg Date: Tue, 28 Jul 2026 11:35:04 -0700 Subject: [PATCH] topology: set flit_bytes=4096 (was implicit 256 default) Wire per-flit loop in engine._wire (ADR-0033 Phase 2c) iterates nbytes/flit_bytes times per hop, firing ~2 SimPy events per flit. At long-context decode, K/V tile loads are hundreds of KB per shard, producing thousands of flit-iterations per hop. The default 256-byte flit size made the wire loop the dominant wall-clock cost. Raising flit_bytes to 4096 (still a multiple of hbm_ctrl.burst_bytes=256 per ADR-0033 D1) cuts flit iterations 16x with no meaningful change to modeled latency. Wormhole pipelining and the available_at BW-share chain in _wire behave identically at any flit size. Measured impact on Case 4 (Cube-SP x PE-SP) decode, C=8, P=8, T_q=1, LLaMA-3.1-70B single-KV-group, enable_data=False: S_kv flit=256 wall flit=4096 wall speedup latency drift 128K 233.71 s 18.16 s 12.9x +0.05% (0.25us) 256K 485.75 s 25.31 s 19.2x +0.01% (0.09us) 512K 1045.64 s 48.45 s 21.6x +0.01% (0.17us) 1M ~2250 s (proj) 104.75 s ~21x within extrap Total sweep at flit=4096: ~3.3 min. Baseline would have been ~65 min. 1M decode iteration is now practical. op_log records are byte-identical (same counts, same components); the tiny latency drift (order of a few ns) comes from the last flit's env.timeout landing on a slightly different granularity when flits are larger. Well within noise for any test that asserts on structural invariants (dma_write cubes, ipcq_copy counts). Verified: - tests/attention/test_milestone_gqa_decode_long_ctx_4cases.py: 18/18 pass (also 7x faster: 174s -> 24.83s per full run) Co-Authored-By: Claude Opus 4.7 --- topology.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/topology.yaml b/topology.yaml index e1238d4..7e64d45 100644 --- a/topology.yaml +++ b/topology.yaml @@ -1,6 +1,7 @@ system: ns_per_mm: 0.01 # wire propagation delay: 10 ps/mm (on-chip silicon) + flit_bytes: 4096 # wire chunk size (ADR-0033 D1: multiple of hbm_ctrl.burst_bytes=256) sips: count: 2