analytical-viz: add sidebar Batch B selectbox + wire through all suggesters
New sidebar widget in the Workload expander:
Batch B (concurrent requests) — selectbox 1/2/4/…/256, default 1.
Wired end-to-end:
- app.py sidebar → forwards b_batch when constructing the main topo
- auto_suggest(..., b=1) — passes b to _score_candidate, which builds
TopologyConfig with b=b
- run_auto_explore(..., b=1) — sets topo.b = b for every enumerated
candidate before scoring
- joint_explore(..., b=1) — forwards b to _best_parallelism_for_hw
and _best_parallelism_two_stage; both set topo.b = b before scoring
All button handlers (sidebar Apply-scope, Physical Layout scope,
Auto Suggest tab sweep, Auto Hardware tab sweep) now pass b_batch.
Combined with the earlier partial-scaffold commits (memory_layout
scales KV by B; stage_rmsnorm / stage_wq / stage_wkv / stage_kv_append /
_per_hop_qkT_pv scale flops + activation memory by B), changing B in
the sidebar now affects reported latency and per-PE memory footprint
in the visible parts of the pipeline. The remaining FFN + comm-AR
stages still ignore B (they'll be next); their contribution is small
for the memory-bound decode case that matters most, but latency for
FFN-heavy configs at high B will be slightly under-reported until
those are scaled too.
Verified: 24 pytest tests pass at default B=1 (backward compat).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -419,6 +419,7 @@ def run_auto_explore(
|
||||
mode: str = "decode",
|
||||
include_attention: bool = True,
|
||||
include_ffn: bool = True,
|
||||
b: int = 1,
|
||||
) -> AutoExploreResult:
|
||||
"""Enumerate all configs, score each, extract Pareto frontier.
|
||||
|
||||
@@ -433,6 +434,8 @@ def run_auto_explore(
|
||||
all_scores: list[ConfigScore] = []
|
||||
total_enumerated = 0
|
||||
for topo in enumerate_configs(model, s_kv, mode):
|
||||
# Inject the caller's batch B into every candidate.
|
||||
topo.b = b
|
||||
total_enumerated += 1
|
||||
cfg = FullConfig(model=model, topo=topo, machine=machine)
|
||||
all_scores.append(score_config(
|
||||
|
||||
Reference in New Issue
Block a user