analytical-viz: add sidebar Batch B selectbox + wire through all suggesters
New sidebar widget in the Workload expander:
Batch B (concurrent requests) — selectbox 1/2/4/…/256, default 1.
Wired end-to-end:
- app.py sidebar → forwards b_batch when constructing the main topo
- auto_suggest(..., b=1) — passes b to _score_candidate, which builds
TopologyConfig with b=b
- run_auto_explore(..., b=1) — sets topo.b = b for every enumerated
candidate before scoring
- joint_explore(..., b=1) — forwards b to _best_parallelism_for_hw
and _best_parallelism_two_stage; both set topo.b = b before scoring
All button handlers (sidebar Apply-scope, Physical Layout scope,
Auto Suggest tab sweep, Auto Hardware tab sweep) now pass b_batch.
Combined with the earlier partial-scaffold commits (memory_layout
scales KV by B; stage_rmsnorm / stage_wq / stage_wkv / stage_kv_append /
_per_hop_qkT_pv scale flops + activation memory by B), changing B in
the sidebar now affects reported latency and per-PE memory footprint
in the visible parts of the pipeline. The remaining FFN + comm-AR
stages still ignore B (they'll be next); their contribution is small
for the memory-bound decode case that matters most, but latency for
FFN-heavy configs at high B will be slightly under-reported until
those are scaled too.
Verified: 24 pytest tests pass at default B=1 (backward compat).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -60,7 +60,8 @@ def _iter_candidates(model: ModelConfig) -> Iterable[tuple[int, int, int]]:
|
||||
def _score_candidate(cp: int, tp: int, pp: int,
|
||||
model: ModelConfig, machine: MachineParams,
|
||||
s_kv: int, mode: str,
|
||||
slack_frac: float = 0.10) -> Suggestion:
|
||||
slack_frac: float = 0.10,
|
||||
b: int = 1) -> Suggestion:
|
||||
# For each triple, try both cp_placement options and keep the one
|
||||
# with fewer cubes (breaks the historical "CP always across cubes"
|
||||
# default when a smaller pack is possible). The pe placement is only
|
||||
@@ -73,7 +74,7 @@ def _score_candidate(cp: int, tp: int, pp: int,
|
||||
_placements_to_try.append("pe")
|
||||
for _place in _placements_to_try:
|
||||
_t = TopologyConfig(
|
||||
cp=cp, tp=tp, pp=pp, s_kv=s_kv, mode=mode,
|
||||
cp=cp, tp=tp, pp=pp, s_kv=s_kv, mode=mode, b=b,
|
||||
cp_placement=_place,
|
||||
)
|
||||
if best_topo is None or _t.cubes_used < best_topo.cubes_used:
|
||||
@@ -108,7 +109,8 @@ def _score_candidate(cp: int, tp: int, pp: int,
|
||||
|
||||
def auto_suggest(model: ModelConfig, machine: MachineParams,
|
||||
s_kv: int, mode: str = "decode",
|
||||
slack_frac: float = 0.10) -> Suggestion:
|
||||
slack_frac: float = 0.10,
|
||||
b: int = 1) -> Suggestion:
|
||||
"""Return the smallest deployment (fewer cubes, then fewer PEs)
|
||||
that fits.
|
||||
|
||||
@@ -124,7 +126,8 @@ def auto_suggest(model: ModelConfig, machine: MachineParams,
|
||||
scored: list[Suggestion] = []
|
||||
for cp, tp, pp in _iter_candidates(model):
|
||||
scored.append(
|
||||
_score_candidate(cp, tp, pp, model, machine, s_kv, mode, slack_frac)
|
||||
_score_candidate(cp, tp, pp, model, machine, s_kv, mode,
|
||||
slack_frac, b=b)
|
||||
)
|
||||
scored.sort(key=lambda s: (s.cubes_used, s.pes_used, s.pp, s.tp, s.cp))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user