Adds compute_parallelism_sensitivity() + ParallelismSensitivityRow to
auto_explore.py. For a baseline ConfigScore (picked from the Pareto set),
sweeps each parallelism knob (CP, TP, PP, DP, EP) individually while
holding others fixed. Reports latency + memory-fit per value.
Sweep values start at 1 and step by 2 (multiples of 2 rather than only
powers of 2), giving finer granularity for the visual than the enumerator's
sparser set. CP goes up to 256 (per real-deployment scale), TP to 64,
PP to 32.
New UI panel in Auto Explore tab: 5 subplots (log/log), one per knob:
- Solid line = latency where the config fits memory
- Red X markers = infeasible (out of budget)
- Dotted vertical line = baseline value
User picks which Pareto row is the "baseline" via a number input; the
sensitivity chart re-computes around it.
Behaviour caveat noted during verification: for large PP the FFN AR can
cross a SIP boundary (uses stage_latencies.py:730 sips_used tier
selection), producing a step-up in latency. This is the existing model's
choice, honestly reflected in the chart. Whether the FFN AR should span
only one PP stage's ranks (thus not cross SIPs) is a separate discussion
about stage_latencies.py.
Verified:
- 11 pytest tests pass (added 2 new for parallelism sensitivity)
- Smoke: at Llama 70B decode 128K with baseline CP=8/TP=16/PP=1/DP=1:
* CP sweep: 4 fits, 8 = baseline optimum, 16+ overshoots memory
* TP sweep: 8/16 fit, 16 = baseline optimum, 32 slower (spans SIPs)
* PP sweep: 1 = baseline, 2+ slower due to sips_used tier drop for FFN AR
* DP sweep: same shape as PP
* EP sweep: monotone decreasing (bigger EP = smaller per-PE FFN)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
mukesh
91b63eb9f6