analytical-viz: parallelism sensitivity chart in Auto Explore tab

Adds compute_parallelism_sensitivity() + ParallelismSensitivityRow to
auto_explore.py. For a baseline ConfigScore (picked from the Pareto set),
sweeps each parallelism knob (CP, TP, PP, DP, EP) individually while
holding others fixed. Reports latency + memory-fit per value.

Sweep values start at 1 and step by 2 (multiples of 2 rather than only
powers of 2), giving finer granularity for the visual than the enumerator's
sparser set. CP goes up to 256 (per real-deployment scale), TP to 64,
PP to 32.

New UI panel in Auto Explore tab: 5 subplots (log/log), one per knob:
  - Solid line = latency where the config fits memory
  - Red X markers = infeasible (out of budget)
  - Dotted vertical line = baseline value
User picks which Pareto row is the "baseline" via a number input; the
sensitivity chart re-computes around it.

Behaviour caveat noted during verification: for large PP the FFN AR can
cross a SIP boundary (uses stage_latencies.py:730 sips_used tier
selection), producing a step-up in latency. This is the existing model's
choice, honestly reflected in the chart. Whether the FFN AR should span
only one PP stage's ranks (thus not cross SIPs) is a separate discussion
about stage_latencies.py.

Verified:
- 11 pytest tests pass (added 2 new for parallelism sensitivity)
- Smoke: at Llama 70B decode 128K with baseline CP=8/TP=16/PP=1/DP=1:
  * CP sweep: 4 fits, 8 = baseline optimum, 16+ overshoots memory
  * TP sweep: 8/16 fit, 16 = baseline optimum, 32 slower (spans SIPs)
  * PP sweep: 1 = baseline, 2+ slower due to sips_used tier drop for FFN AR
  * DP sweep: same shape as PP
  * EP sweep: monotone decreasing (bigger EP = smaller per-PE FFN)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-07-28 13:39:51 -07:00
parent 55c20bd1a6
commit 91b63eb9f6
3 changed files with 184 additions and 1 deletions
@@ -140,3 +140,41 @@ def test_pp_does_not_reduce_single_request_latency():
assert fastest.pp == 1, (
f"expected PP=1 for the fastest decode config; got PP={fastest.pp}"
)
# ── Parallelism sensitivity ─────────────────────────────────────────
def test_parallelism_sensitivity_has_all_five_knobs():
"""compute_parallelism_sensitivity returns one row per knob."""
from tests.analytical_visualization.auto_explore import (
compute_parallelism_sensitivity,
)
model = PRESETS["Llama 3.1 70B"].model
machine = MachineParams()
res = run_auto_explore(model, machine, s_kv=8192, mode="decode")
baseline = res.pareto_scores[0]
rows = compute_parallelism_sensitivity(
baseline, model, machine, s_kv=8192, mode="decode",
)
knobs = {r.knob for r in rows}
assert knobs == {"cp", "tp", "pp", "dp", "ep"}
def test_parallelism_sensitivity_includes_baseline_value():
"""Each row's values contain the baseline_value."""
from tests.analytical_visualization.auto_explore import (
compute_parallelism_sensitivity,
)
model = PRESETS["Llama 3.1 70B"].model
machine = MachineParams()
res = run_auto_explore(model, machine, s_kv=8192, mode="decode")
baseline = res.pareto_scores[0]
rows = compute_parallelism_sensitivity(
baseline, model, machine, s_kv=8192, mode="decode",
)
for r in rows:
# Baseline value must be sweepable OR mapped to something in values.
assert r.baseline_value in r.values or r.baseline_value == 1, (
f"knob {r.knob}: baseline_value={r.baseline_value} not in {r.values}"
)