analytical-viz: capacity planning tab — add rules of thumb + deployment templates
Two more table sections on the Capacity planning tab: **Section 4 — Practical rules of thumb (by context regime)** - Short (S_kv < L*): pack HBM, B ~ 2·B*, high util, cheap tier - Long (S_kv > L*): fewer users/replica, low B, util drops as 1/(1 + S_kv/L*), pricier - Extreme (S_kv >> L*): dedicated pool, heavy CP, disaggregated prefill, very low util without sparse attention Columns: Regime, Batch strategy, Utilization, Cost/token, Deployment. **Section 5 — Sample deployment templates** Same base model, three different sharding recipes routed to by the API gateway based on request context length: - Config_small : CP=1, TP=8, PP=1 → 8 GPUs, up to 32k, B=64 - Config_medium: CP=4, TP=8, PP=1 → 32 GPUs, up to 128k, B=32 - Config_large : CP=32, TP=8, PP=1 → 256 GPUs, up to 1M, B=4 Columns: Tier, CP, TP, PP, Total GPUs/replica, Max context, Typical B, Best for. Playbook section renumbered to #6. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -2828,8 +2828,68 @@ with tab_planning:
|
|||||||
|
|
||||||
st.divider()
|
st.divider()
|
||||||
|
|
||||||
# ── Section 4: Binding-axis playbook ──────────────────────────
|
# ── Section 4: Practical rules of thumb by context regime ─────
|
||||||
st.markdown("### 4. What to do when each axis binds")
|
st.markdown("### 4. Practical rules of thumb (by context regime)")
|
||||||
|
_rules_rows = [
|
||||||
|
{"Regime": "Short context (S_kv < L*)",
|
||||||
|
"Batch strategy": "Pack as many users as HBM allows; aim for "
|
||||||
|
"B ≈ 2·B*",
|
||||||
|
"Utilization": "High (near compute floor)",
|
||||||
|
"Cost / token": "Low — cheap tier",
|
||||||
|
"Deployment": "Standard elastic pool"},
|
||||||
|
{"Regime": "Long context (S_kv > L*)",
|
||||||
|
"Batch strategy": "Fewer users per replica; more HBM per user; "
|
||||||
|
"lower B",
|
||||||
|
"Utilization": "Drops as 1 / (1 + S_kv/L*)",
|
||||||
|
"Cost / token": "Higher — accept lower utilization",
|
||||||
|
"Deployment": "Length-tier replica with more CP sharding"},
|
||||||
|
{"Regime": "Extreme long context (S_kv ≫ L*)",
|
||||||
|
"Batch strategy": "Dedicated pool, heavy CP; often "
|
||||||
|
"disaggregated prefill",
|
||||||
|
"Utilization": "Very low without sparse attention / MLA",
|
||||||
|
"Cost / token": "Much higher — priced accordingly",
|
||||||
|
"Deployment": "Long-context tier + disaggregated prefill/decode"},
|
||||||
|
]
|
||||||
|
st.dataframe(pd.DataFrame(_rules_rows), width='stretch',
|
||||||
|
hide_index=True)
|
||||||
|
|
||||||
|
st.divider()
|
||||||
|
|
||||||
|
# ── Section 5: Sample deployment templates ────────────────────
|
||||||
|
st.markdown("### 5. Sample deployment templates (same model, different sharding)")
|
||||||
|
st.caption(
|
||||||
|
"One base model runs on all three configs. What changes is the "
|
||||||
|
"sharding (CP/TP/PP) that shapes the effective 'small' vs "
|
||||||
|
"'large' setup. The API gateway routes each request to the "
|
||||||
|
"right tier based on its context length."
|
||||||
|
)
|
||||||
|
_tmpl_rows = [
|
||||||
|
{"Tier": "Config_small",
|
||||||
|
"CP": 1, "TP": 8, "PP": 1,
|
||||||
|
"Total GPUs / replica": 8,
|
||||||
|
"Max context": "up to 32k",
|
||||||
|
"Typical B": 64,
|
||||||
|
"Best for": "Chat, RAG, short prompts — high utilization"},
|
||||||
|
{"Tier": "Config_medium",
|
||||||
|
"CP": 4, "TP": 8, "PP": 1,
|
||||||
|
"Total GPUs / replica": 32,
|
||||||
|
"Max context": "up to 128k",
|
||||||
|
"Typical B": 32,
|
||||||
|
"Best for": "Long-doc analysis, code review — standard tier"},
|
||||||
|
{"Tier": "Config_large",
|
||||||
|
"CP": 32, "TP": 8, "PP": 1,
|
||||||
|
"Total GPUs / replica": 256,
|
||||||
|
"Max context": "up to 1M",
|
||||||
|
"Typical B": 4,
|
||||||
|
"Best for": "Whole-repo / long-video / massive-doc — premium tier"},
|
||||||
|
]
|
||||||
|
st.dataframe(pd.DataFrame(_tmpl_rows), width='stretch',
|
||||||
|
hide_index=True)
|
||||||
|
|
||||||
|
st.divider()
|
||||||
|
|
||||||
|
# ── Section 6: Binding-axis playbook ──────────────────────────
|
||||||
|
st.markdown("### 6. What to do when each axis binds")
|
||||||
_playbook_rows = [
|
_playbook_rows = [
|
||||||
{"Binding axis": "A. Capacity (weights)",
|
{"Binding axis": "A. Capacity (weights)",
|
||||||
"Symptom": "Weights alone are close to per-PE HBM budget",
|
"Symptom": "Weights alone are close to per-PE HBM budget",
|
||||||
|
|||||||
Reference in New Issue
Block a user