paper: §2 platform deep-edit + §1 GQA framing + figure regen
§2 (KernBench Platform): - Fix Table 2 HBM aggregate BW (1024 → 2048 GB/s); drop stale hbm_total_bw_gbs from topology.yaml (never read by sim_engine) - Split PE_CPU / PE_SCHED fixed-cost row; disambiguate from the 40-cycle command-dispatch FIXED term - §2.2 two-pass: expand to describe Pass 1 timing and Pass 2 data data-correctness path - §2.3 dispatch model: add motivation sentence for the descriptor-size linear form - §2.4 Accuracy: reorder GEMM → All-reduce → Probe → Simplifications; drop FSIM aside (already covered by §4 Fig 5 caption); soften 'every ns' → 'every modeled latency contribution' - §2.5 HW config: add 64 TFLOP/s vs 2048 GB/s (~31 FLOP/byte) balance-point intuition and forward pointer to §5 GQA decode - Fig 1 caption: separate illustrative topology from experimental configuration §1: tighten GQA-as-primary-bandwidth-bottleneck framing. Figures: regenerate SIP / CUBE architecture (SVG sources + PDF + generator scripts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -25,9 +25,12 @@ serves as the common evaluation platform for all mechanisms and kernels
|
||||
discussed in this study.
|
||||
|
||||
This report focuses on Grouped-Query Attention (GQA), one of the most
|
||||
performance- and bandwidth-critical components of LLM inference. Modern
|
||||
decoder-only models such as Llama~3 and Mistral have largely transitioned
|
||||
from GPT-3-style multi-head attention (MHA) to GQA, in which multiple
|
||||
performance- and bandwidth-critical components of LLM inference. GQA
|
||||
dominates inference-time memory traffic and KV-cache capacity in modern
|
||||
LLM serving, making it the primary bandwidth bottleneck on memory-centric
|
||||
architectures such as AHBM. Modern decoder-only models such as Llama~3
|
||||
and Mistral have largely transitioned from GPT-3-style multi-head
|
||||
attention (MHA) to GQA, in which multiple
|
||||
query heads share a single KV head to reduce KV cache capacity and
|
||||
memory-bandwidth requirements. While GQA improves system efficiency at
|
||||
the model level, mapping it efficiently onto AHBM introduces three
|
||||
|
||||
Reference in New Issue
Block a user