New section at the bottom of the Physical Layout tab that sums the
per-stage latencies into end-to-end metrics for the current sharding:
- **TPOT** = one decode-step time × N_layers (per output token)
- **TTFT** = one prefill pass time × N_layers (for a prompt of the
input length, computed in prefill mode)
- **E2E** = TTFT + (N_out − 1) × TPOT
- **Throughput/replica** = B / TPOT (tokens/sec)
Two number inputs: prompt length (default = sidebar S_kv) and target
N_out tokens (default 100). Four KPI cards + a breakdown table with
per-layer and all-layer µs/ms for attention/FFN in both modes.
Both TTFT and TPOT are computed by cloning the sidebar TopologyConfig
with mode swapped ('decode' for TPOT, 'prefill' for TTFT with T_q =
prompt length), then reusing all_stages / all_ffn_stages — no new
math. Caveat noted in caption: PP=1 assumption (no microbatch
overlap / bubble modeling).
Sanity: Llama 3 8B on CP=2/TP=8/default SIP → TPOT ≈ 4.2 ms/token,
TTFT ≈ 1.1 s for an 8k prompt. Cross-checkable on the Per-stage
latency tab.
Smoke test in test_stage_shapes asserts the section title appears
exactly once and sits inside tab_layout (before tab_memory).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
mukesh
bc5e704572