analytical-viz: capacity planning tab — add rules of thumb + deployment templates

Two more table sections on the Capacity planning tab:

**Section 4 — Practical rules of thumb (by context regime)**
- Short (S_kv < L*): pack HBM, B ~ 2·B*, high util, cheap tier
- Long (S_kv > L*): fewer users/replica, low B, util drops as
  1/(1 + S_kv/L*), pricier
- Extreme (S_kv >> L*): dedicated pool, heavy CP, disaggregated
  prefill, very low util without sparse attention

Columns: Regime, Batch strategy, Utilization, Cost/token, Deployment.

**Section 5 — Sample deployment templates**
Same base model, three different sharding recipes routed to by the
API gateway based on request context length:
- Config_small : CP=1,  TP=8, PP=1 → 8 GPUs, up to 32k, B=64
- Config_medium: CP=4,  TP=8, PP=1 → 32 GPUs, up to 128k, B=32
- Config_large : CP=32, TP=8, PP=1 → 256 GPUs, up to 1M, B=4

Columns: Tier, CP, TP, PP, Total GPUs/replica, Max context, Typical
B, Best for.

Playbook section renumbered to #6.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-07-29 15:13:08 -07:00
parent f54822d247
commit 77abc95d78
+62 -2
View File
@@ -2828,8 +2828,68 @@ with tab_planning:
st.divider()
# ── Section 4: Binding-axis playbook ──────────────────────────
st.markdown("### 4. What to do when each axis binds")
# ── Section 4: Practical rules of thumb by context regime ─────
st.markdown("### 4. Practical rules of thumb (by context regime)")
_rules_rows = [
{"Regime": "Short context (S_kv < L*)",
"Batch strategy": "Pack as many users as HBM allows; aim for "
"B ≈ 2·B*",
"Utilization": "High (near compute floor)",
"Cost / token": "Low — cheap tier",
"Deployment": "Standard elastic pool"},
{"Regime": "Long context (S_kv > L*)",
"Batch strategy": "Fewer users per replica; more HBM per user; "
"lower B",
"Utilization": "Drops as 1 / (1 + S_kv/L*)",
"Cost / token": "Higher — accept lower utilization",
"Deployment": "Length-tier replica with more CP sharding"},
{"Regime": "Extreme long context (S_kv ≫ L*)",
"Batch strategy": "Dedicated pool, heavy CP; often "
"disaggregated prefill",
"Utilization": "Very low without sparse attention / MLA",
"Cost / token": "Much higher — priced accordingly",
"Deployment": "Long-context tier + disaggregated prefill/decode"},
]
st.dataframe(pd.DataFrame(_rules_rows), width='stretch',
hide_index=True)
st.divider()
# ── Section 5: Sample deployment templates ────────────────────
st.markdown("### 5. Sample deployment templates (same model, different sharding)")
st.caption(
"One base model runs on all three configs. What changes is the "
"sharding (CP/TP/PP) that shapes the effective 'small' vs "
"'large' setup. The API gateway routes each request to the "
"right tier based on its context length."
)
_tmpl_rows = [
{"Tier": "Config_small",
"CP": 1, "TP": 8, "PP": 1,
"Total GPUs / replica": 8,
"Max context": "up to 32k",
"Typical B": 64,
"Best for": "Chat, RAG, short prompts — high utilization"},
{"Tier": "Config_medium",
"CP": 4, "TP": 8, "PP": 1,
"Total GPUs / replica": 32,
"Max context": "up to 128k",
"Typical B": 32,
"Best for": "Long-doc analysis, code review — standard tier"},
{"Tier": "Config_large",
"CP": 32, "TP": 8, "PP": 1,
"Total GPUs / replica": 256,
"Max context": "up to 1M",
"Typical B": 4,
"Best for": "Whole-repo / long-video / massive-doc — premium tier"},
]
st.dataframe(pd.DataFrame(_tmpl_rows), width='stretch',
hide_index=True)
st.divider()
# ── Section 6: Binding-axis playbook ──────────────────────────
st.markdown("### 6. What to do when each axis binds")
_playbook_rows = [
{"Binding axis": "A. Capacity (weights)",
"Symptom": "Weights alone are close to per-PE HBM budget",