analytical-viz: parallelism sensitivity chart in Auto Explore tab

Adds compute_parallelism_sensitivity() + ParallelismSensitivityRow to
auto_explore.py. For a baseline ConfigScore (picked from the Pareto set),
sweeps each parallelism knob (CP, TP, PP, DP, EP) individually while
holding others fixed. Reports latency + memory-fit per value.

Sweep values start at 1 and step by 2 (multiples of 2 rather than only
powers of 2), giving finer granularity for the visual than the enumerator's
sparser set. CP goes up to 256 (per real-deployment scale), TP to 64,
PP to 32.

New UI panel in Auto Explore tab: 5 subplots (log/log), one per knob:
  - Solid line = latency where the config fits memory
  - Red X markers = infeasible (out of budget)
  - Dotted vertical line = baseline value
User picks which Pareto row is the "baseline" via a number input; the
sensitivity chart re-computes around it.

Behaviour caveat noted during verification: for large PP the FFN AR can
cross a SIP boundary (uses stage_latencies.py:730 sips_used tier
selection), producing a step-up in latency. This is the existing model's
choice, honestly reflected in the chart. Whether the FFN AR should span
only one PP stage's ranks (thus not cross SIPs) is a separate discussion
about stage_latencies.py.

Verified:
- 11 pytest tests pass (added 2 new for parallelism sensitivity)
- Smoke: at Llama 70B decode 128K with baseline CP=8/TP=16/PP=1/DP=1:
  * CP sweep: 4 fits, 8 = baseline optimum, 16+ overshoots memory
  * TP sweep: 8/16 fit, 16 = baseline optimum, 32 slower (spans SIPs)
  * PP sweep: 1 = baseline, 2+ slower due to sips_used tier drop for FFN AR
  * DP sweep: same shape as PP
  * EP sweep: monotone decreasing (bigger EP = smaller per-PE FFN)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-07-28 13:39:51 -07:00
parent 55c20bd1a6
commit 91b63eb9f6
3 changed files with 184 additions and 1 deletions
+66 -1
View File
@@ -1286,7 +1286,10 @@ with tab_compare:
# ── TAB 5: Auto Explore ─────────────────────────────────────────
with tab_auto:
from tests.analytical_visualization.auto_explore import run_auto_explore
from tests.analytical_visualization.auto_explore import (
compute_parallelism_sensitivity,
run_auto_explore,
)
st.markdown(
"Sweep every valid parallelism config (CP, TP, PP, DP, kv_shard, "
@@ -1416,6 +1419,68 @@ with tab_auto:
_df = pd.DataFrame(_rows)
st.dataframe(_df, width='stretch', hide_index=True)
# ── Parallelism sensitivity subplots ──────────────────────
st.markdown(
"**Parallelism sensitivity** — for the *baseline* config "
"picked below, vary each knob one at a time (holding others "
"fixed). Answers 'if I only tweak CP / TP / PP / DP / EP, "
"what happens?' Solid line = fits memory; red X = infeasible."
)
_sens_c1, _sens_c2 = st.columns([1, 4])
with _sens_c1:
_sens_row = st.number_input(
"Baseline row #",
min_value=0, max_value=len(_pareto_by_lat) - 1,
value=0, step=1, key="_auto_sens_row",
help="Which Pareto row is the sensitivity computed around?",
)
_baseline_for_sens = _pareto_by_lat[int(_sens_row)]
_sens_rows = compute_parallelism_sensitivity(
_baseline_for_sens, model, machine, s_kv=s_kv, mode=mode,
)
import matplotlib.pyplot as _plt
_fig_sens, _axes_sens = _plt.subplots(1, 5, figsize=(14, 3.2))
for _ax, _row in zip(_axes_sens, _sens_rows):
_fit_vals, _fit_lats = [], []
_bad_vals = []
for _v, _lat_ns, _fit in zip(_row.values, _row.latencies_ns,
_row.fits_flags):
if _lat_ns != _lat_ns: # NaN — skip out-of-bounds
continue
if _fit:
_fit_vals.append(_v)
_fit_lats.append(_lat_ns / 1e6) # ms
else:
_bad_vals.append((_v, _lat_ns / 1e6))
if _fit_vals:
_ax.plot(_fit_vals, _fit_lats, "o-", color="#2c3e50",
linewidth=1.4, markersize=4)
if _bad_vals:
_ax.scatter(
[v for v, _ in _bad_vals],
[l for _, l in _bad_vals],
marker="x", color="#c0392b", s=40, alpha=0.6,
label="no fit",
)
# Baseline marker
_ax.axvline(_row.baseline_value, color="#3498db",
linestyle=":", linewidth=1.0)
_ax.set_xscale("log")
_ax.set_yscale("log")
_ax.set_title(
f"{_row.knob.upper()} (baseline={_row.baseline_value})",
fontsize=10,
)
_ax.set_xlabel("value")
_ax.set_ylabel("lat (ms)")
_ax.grid(alpha=0.3)
if _bad_vals:
_ax.legend(loc="best", fontsize=7)
_fig_sens.tight_layout()
st.pyplot(_fig_sens, width='stretch')
_plt.close(_fig_sens)
# ── Load-into-sidebar ─────────────────────────────────────
st.markdown("**Load a Pareto config into the main sidebar sliders**")
_lc1, _lc2 = st.columns([1, 3])