analytical-viz: autosuggest prefers fewer cubes, tries cp_placement=pe
Two changes to the memory-only auto_suggest so a "fits" that packs into
one cube is preferred over a "fits" that spreads across multiple cubes.
Suggestion dataclass gains cubes_used and cp_placement fields.
_score_candidate: for each (CP,TP,PP) triple, try cp_placement="cube"
(historical default: CP spans cubes) and, when CP·TP fits in one cube's
PE count, also cp_placement="pe" (pack CP into intra-cube PEs). Keep
the placement with fewer cubes; break ties toward "cube".
auto_suggest: switch sort key from (pes_used ↑, pp ↑, tp ↑, cp ↑) to
(cubes_used ↑, pes_used ↑, pp ↑, tp ↑, cp ↑). Fewer cubes wins first
because a cube is the physical die-level unit; PE count is the
tiebreaker.
Sidebar caption now also displays cubes_used + cp_placement so the
user can see the packed layout at a glance. Preset-change auto-reset
also applies the picked cp_placement.
Observed effect (verified):
Qwen 3 8B / 128K decode: CP=4 TP=2, "pe" → 1 cube, 8 PEs
(was 4 cubes with default "cube")
Llama 3.1 70B / 128K: CP=4 TP=16, "cube" → 8 cubes (unchanged;
TP=16 > 8 PEs/cube, can't pack)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -32,6 +32,8 @@ class Suggestion:
|
||||
fits: bool
|
||||
pes_used: int
|
||||
sips_used: int
|
||||
cubes_used: int = 0 # picked to minimize this (see _score_candidate)
|
||||
cp_placement: str = "cube" # "pe" packs CP into intra-cube PEs when it fits
|
||||
reason: str = ""
|
||||
|
||||
|
||||
@@ -59,7 +61,25 @@ def _score_candidate(cp: int, tp: int, pp: int,
|
||||
model: ModelConfig, machine: MachineParams,
|
||||
s_kv: int, mode: str,
|
||||
slack_frac: float = 0.10) -> Suggestion:
|
||||
topo = TopologyConfig(cp=cp, tp=tp, pp=pp, s_kv=s_kv, mode=mode)
|
||||
# For each triple, try both cp_placement options and keep the one
|
||||
# with fewer cubes (breaks the historical "CP always across cubes"
|
||||
# default when a smaller pack is possible). The pe placement is only
|
||||
# valid when CP·TP fits within a single cube's PE count.
|
||||
best_topo: TopologyConfig | None = None
|
||||
best_placement = "cube"
|
||||
_pe_per_cube_hw = TopologyConfig().pes_per_cube_hw # instance-invariant HW const
|
||||
_placements_to_try = ["cube"]
|
||||
if cp * tp <= _pe_per_cube_hw:
|
||||
_placements_to_try.append("pe")
|
||||
for _place in _placements_to_try:
|
||||
_t = TopologyConfig(
|
||||
cp=cp, tp=tp, pp=pp, s_kv=s_kv, mode=mode,
|
||||
cp_placement=_place,
|
||||
)
|
||||
if best_topo is None or _t.cubes_used < best_topo.cubes_used:
|
||||
best_topo = _t
|
||||
best_placement = _place
|
||||
topo = best_topo
|
||||
cfg = FullConfig(model=model, topo=topo, machine=machine)
|
||||
mem = compute_memory(cfg)
|
||||
|
||||
@@ -80,6 +100,8 @@ def _score_candidate(cp: int, tp: int, pp: int,
|
||||
fits=fits,
|
||||
pes_used=topo.total_pes,
|
||||
sips_used=topo.sips_used,
|
||||
cubes_used=topo.cubes_used,
|
||||
cp_placement=best_placement,
|
||||
reason=reason,
|
||||
)
|
||||
|
||||
@@ -87,20 +109,27 @@ def _score_candidate(cp: int, tp: int, pp: int,
|
||||
def auto_suggest(model: ModelConfig, machine: MachineParams,
|
||||
s_kv: int, mode: str = "decode",
|
||||
slack_frac: float = 0.10) -> Suggestion:
|
||||
"""Return the smallest-PE-count (CP, TP, PP) that fits.
|
||||
"""Return the smallest deployment (fewer cubes, then fewer PEs)
|
||||
that fits.
|
||||
|
||||
Sort key: (cubes_used ↑, pes_used ↑, pp ↑, tp ↑, cp ↑).
|
||||
Fewer cubes wins first because a cube is the physical die-level
|
||||
hardware unit; PE count is the tiebreaker. Preferring fewer PP then
|
||||
TP then CP keeps the earlier historical bias (avoid pipeline
|
||||
bubbles / head-dim splits) among equal-cube-and-PE ties.
|
||||
|
||||
If no candidate fits, returns the best-effort one (highest slack,
|
||||
even if negative) with fits=False.
|
||||
"""
|
||||
best_fit: Suggestion | None = None
|
||||
best_effort: Suggestion | None = None
|
||||
|
||||
scored: list[Suggestion] = []
|
||||
for cp, tp, pp in _iter_candidates(model):
|
||||
s = _score_candidate(cp, tp, pp, model, machine, s_kv, mode, slack_frac)
|
||||
if s.fits and best_fit is None:
|
||||
best_fit = s
|
||||
break # candidates are pre-sorted by PE count
|
||||
if best_effort is None or s.slack_gb > best_effort.slack_gb:
|
||||
best_effort = s
|
||||
scored.append(
|
||||
_score_candidate(cp, tp, pp, model, machine, s_kv, mode, slack_frac)
|
||||
)
|
||||
scored.sort(key=lambda s: (s.cubes_used, s.pes_used, s.pp, s.tp, s.cp))
|
||||
|
||||
return best_fit or best_effort # type: ignore
|
||||
for s in scored:
|
||||
if s.fits:
|
||||
return s
|
||||
# No fit — return the best-effort (largest slack, closest to fitting).
|
||||
return max(scored, key=lambda s: s.slack_gb)
|
||||
|
||||
Reference in New Issue
Block a user