WIP toward batch-size support. This first commit is behavior-preserving
at the default B=1 (every new multiplication is by max(1, cfg.topo.b) =
1 today) so all existing tests pass. Follow-up commits will:
- scale the remaining stages (S6/S7/S8/S9/S10 + C1/C2/C3 + FFN + FFN AR)
- add a batch selectbox to the sidebar
- forward b through auto_suggest / auto_explore / auto_hardware
Changes so far:
- TopologyConfig: new b: int = 1 field (batch size).
- memory_layout.per_pe_kv_cache_bytes: * B (each concurrent request
keeps its own KV cache slice).
- stage_rmsnorm / stage_wq / stage_wkv / stage_kv_append /
_per_hop_qkT_pv: FLOPs and activation memory scaled by B; weight
bytes stay fixed (weights shared across the batch).
Verified: 24 pytest tests still pass at default B=1.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Was 512 GB/s in the analytical viz MachineParams default. Bringing the
default down to 128 GB/s to match what the modelled physical link now
represents (parity with inter-cube D2D and topology.yaml).
Files touched (all three needed to keep defaults coherent):
- model_config.py: MachineParams.bw_intra_gbs = 128.0 (was 512.0)
- app.py: sidebar selectbox default index = 0 (128 GB/s) instead of 2 (512)
- auto_hardware.py: _HW_KNOB_DEFAULTS + BALANCED + COARSE bw_intra_gbs
baselines shifted so cost_score at defaults remains 6.0 and the
sensitivity sweep starts from the new baseline.
Verified: 24 pytest tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New tests/analytical_visualization/ module - an interactive dashboard
for exploring memory / latency tradeoffs of transformer inference on
the SIP architecture.
Highlights:
- 30+ model presets (Qwen, Llama 2/3/3.1, Mistral, Gemma 2, Phi 3,
DeepSeek MLA, Mixtral/Qwen 3 MoE, Grok-1, ...)
- Placement toggles: TP and CP each on PE-level vs cube-level
- CP ring variant: K/V ring vs Q+O/m/l ring (prefill); in decode the
O/m/l all-reduce is folded into S8 (no separate C1 row)
- SIP interconnect: ring / mesh2d / torus2d with matching link drawing
- Per-stage latency table with compute + memory + comm formulas,
auto-scaled ns/us/ms, colored by dominant bound
- Ring attention loop indicator on the pipeline diagram (purple arc
over S5-S8 with 'xN hops' badge)
- Tensor sharding view with optional physical PE/cube annotations
- Replication-waste + optimization-hints panel
- Save & compare configurations (config1, config2, ...): summary table
plus side-by-side per-stage attention and FFN latency, best-in-row
highlighting
- Symbol glossary with current values for every symbol used in formulas
Not tied to production sim_engine or runtime API; purely analytical
tooling for design-space exploration.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
mukesh