analytical-viz: parallelism sensitivity chart in Auto Explore tab
Adds compute_parallelism_sensitivity() + ParallelismSensitivityRow to auto_explore.py. For a baseline ConfigScore (picked from the Pareto set), sweeps each parallelism knob (CP, TP, PP, DP, EP) individually while holding others fixed. Reports latency + memory-fit per value. Sweep values start at 1 and step by 2 (multiples of 2 rather than only powers of 2), giving finer granularity for the visual than the enumerator's sparser set. CP goes up to 256 (per real-deployment scale), TP to 64, PP to 32. New UI panel in Auto Explore tab: 5 subplots (log/log), one per knob: - Solid line = latency where the config fits memory - Red X markers = infeasible (out of budget) - Dotted vertical line = baseline value User picks which Pareto row is the "baseline" via a number input; the sensitivity chart re-computes around it. Behaviour caveat noted during verification: for large PP the FFN AR can cross a SIP boundary (uses stage_latencies.py:730 sips_used tier selection), producing a step-up in latency. This is the existing model's choice, honestly reflected in the chart. Whether the FFN AR should span only one PP stage's ranks (thus not cross SIPs) is a separate discussion about stage_latencies.py. Verified: - 11 pytest tests pass (added 2 new for parallelism sensitivity) - Smoke: at Llama 70B decode 128K with baseline CP=8/TP=16/PP=1/DP=1: * CP sweep: 4 fits, 8 = baseline optimum, 16+ overshoots memory * TP sweep: 8/16 fit, 16 = baseline optimum, 32 slower (spans SIPs) * PP sweep: 1 = baseline, 2+ slower due to sips_used tier drop for FFN AR * DP sweep: same shape as PP * EP sweep: monotone decreasing (bigger EP = smaller per-PE FFN) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -140,3 +140,41 @@ def test_pp_does_not_reduce_single_request_latency():
|
||||
assert fastest.pp == 1, (
|
||||
f"expected PP=1 for the fastest decode config; got PP={fastest.pp}"
|
||||
)
|
||||
|
||||
|
||||
# ── Parallelism sensitivity ─────────────────────────────────────────
|
||||
|
||||
|
||||
def test_parallelism_sensitivity_has_all_five_knobs():
|
||||
"""compute_parallelism_sensitivity returns one row per knob."""
|
||||
from tests.analytical_visualization.auto_explore import (
|
||||
compute_parallelism_sensitivity,
|
||||
)
|
||||
model = PRESETS["Llama 3.1 70B"].model
|
||||
machine = MachineParams()
|
||||
res = run_auto_explore(model, machine, s_kv=8192, mode="decode")
|
||||
baseline = res.pareto_scores[0]
|
||||
rows = compute_parallelism_sensitivity(
|
||||
baseline, model, machine, s_kv=8192, mode="decode",
|
||||
)
|
||||
knobs = {r.knob for r in rows}
|
||||
assert knobs == {"cp", "tp", "pp", "dp", "ep"}
|
||||
|
||||
|
||||
def test_parallelism_sensitivity_includes_baseline_value():
|
||||
"""Each row's values contain the baseline_value."""
|
||||
from tests.analytical_visualization.auto_explore import (
|
||||
compute_parallelism_sensitivity,
|
||||
)
|
||||
model = PRESETS["Llama 3.1 70B"].model
|
||||
machine = MachineParams()
|
||||
res = run_auto_explore(model, machine, s_kv=8192, mode="decode")
|
||||
baseline = res.pareto_scores[0]
|
||||
rows = compute_parallelism_sensitivity(
|
||||
baseline, model, machine, s_kv=8192, mode="decode",
|
||||
)
|
||||
for r in rows:
|
||||
# Baseline value must be sweepable OR mapped to something in values.
|
||||
assert r.baseline_value in r.values or r.baseline_value == 1, (
|
||||
f"knob {r.knob}: baseline_value={r.baseline_value} not in {r.values}"
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user