topology: set flit_bytes=4096 (was implicit 256 default)

Wire per-flit loop in engine._wire (ADR-0033 Phase 2c) iterates
nbytes/flit_bytes times per hop, firing ~2 SimPy events per flit.
At long-context decode, K/V tile loads are hundreds of KB per shard,
producing thousands of flit-iterations per hop. The default 256-byte
flit size made the wire loop the dominant wall-clock cost.

Raising flit_bytes to 4096 (still a multiple of hbm_ctrl.burst_bytes=256
per ADR-0033 D1) cuts flit iterations 16x with no meaningful change to
modeled latency. Wormhole pipelining and the available_at BW-share
chain in _wire behave identically at any flit size.

Measured impact on Case 4 (Cube-SP x PE-SP) decode, C=8, P=8, T_q=1,
LLaMA-3.1-70B single-KV-group, enable_data=False:

  S_kv     flit=256 wall    flit=4096 wall    speedup    latency drift
   128K       233.71 s          18.16 s        12.9x     +0.05% (0.25us)
   256K       485.75 s          25.31 s        19.2x     +0.01% (0.09us)
   512K      1045.64 s          48.45 s        21.6x     +0.01% (0.17us)
    1M      ~2250 s (proj)     104.75 s        ~21x      within extrap

Total sweep at flit=4096: ~3.3 min. Baseline would have been ~65 min.
1M decode iteration is now practical.

op_log records are byte-identical (same counts, same components); the
tiny latency drift (order of a few ns) comes from the last flit's
env.timeout landing on a slightly different granularity when flits are
larger. Well within noise for any test that asserts on structural
invariants (dma_write cubes, ipcq_copy counts).

Verified:
- tests/attention/test_milestone_gqa_decode_long_ctx_4cases.py: 18/18 pass
  (also 7x faster: 174s -> 24.83s per full run)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-07-28 11:35:04 -07:00
parent f9d0077472
commit 09e9331026
+1
View File
@@ -1,6 +1,7 @@
system: system:
ns_per_mm: 0.01 # wire propagation delay: 10 ps/mm (on-chip silicon) ns_per_mm: 0.01 # wire propagation delay: 10 ps/mm (on-chip silicon)
flit_bytes: 4096 # wire chunk size (ADR-0033 D1: multiple of hbm_ctrl.burst_bytes=256)
sips: sips:
count: 2 count: 2