Two new plots at the top of the Chip Roofline tab (right after the
knob row), side by side:
1. **Latency per decode step** — one forward pass's total time as B
grows. weight_fetch (flat in B), compute (linear), KV (linear),
total. The SLO / TTFT view — bigger B = longer step.
2. **Cost per token** = step latency ÷ B. weight (shrinks 1/B),
compute (flat), KV (flat), total. The efficiency / cost view —
bigger B (up to ~2·B*) = cheaper per token.
Same underlying decomposition, two divisors. Puts the classic
throughput ↔ latency tradeoff in one glance.
Also added a **B (batch size) slider** next to the existing S_kv
slider. Both plots draw a vertical marker at the current B; the
regime KPI ("memory-bound / compute-bound / kv-bound") now uses the
slider B, so you can sweep B live and watch the label flip when you
cross B*.
step_latency_curve + StepLatencyPoint added to chip_roofline; 5 new
tests cover: weight_s batch-invariant, compute_s/kv_s linear in B,
step_total == per_token_total × B (definitional), and step ==
per_token at B=1.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
mukesh
221097ef08