ywkang/kernbench2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

paper: §2 platform deep-edit + §1 GQA framing + figure regen

Browse Source 
§2 (KernBench Platform):
- Fix Table 2 HBM aggregate BW (1024 → 2048 GB/s); drop stale
  hbm_total_bw_gbs from topology.yaml (never read by sim_engine)
- Split PE_CPU / PE_SCHED fixed-cost row; disambiguate from the
  40-cycle command-dispatch FIXED term
- §2.2 two-pass: expand to describe Pass 1 timing and Pass 2 data
  data-correctness path
- §2.3 dispatch model: add motivation sentence for the
  descriptor-size linear form
- §2.4 Accuracy: reorder GEMM → All-reduce → Probe →
  Simplifications; drop FSIM aside (already covered by §4 Fig 5
  caption); soften 'every ns' → 'every modeled latency
  contribution'
- §2.5 HW config: add 64 TFLOP/s vs 2048 GB/s (~31 FLOP/byte)
  balance-point intuition and forward pointer to §5 GQA decode
- Fig 1 caption: separate illustrative topology from experimental
  configuration

§1: tighten GQA-as-primary-bandwidth-bottleneck framing.

Figures: regenerate SIP / CUBE architecture (SVG sources + PDF +
generator scripts).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Yangwook Kang
2026-06-16 16:00:23 -07:00
parent 77eece81c4
commit 6b6e29968a
12 changed files with 869 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/report/1H-codesign-paper/build/main.pdf
BIN
View File
Binary file not shown.
docs/report/1H-codesign-paper/figures/cube_architecture.pdf
BIN
View File
Binary file not shown.
docs/report/1H-codesign-paper/figures/cube_architecture.svg
+312
View File
@@ -0,0 +1,312 @@
<svg xmlns="http://www.w3.org/2000/svg" width="860" height="798" viewBox="55 2 860 798">
<title>cube</title>
<rect width="970" height="900" fill="#ffffff"/>
<text x="485" y="22" text-anchor="middle" font-family="monospace" font-size="18" font-weight="bold" fill="#1f2937">CUBE TOPOLOGY — 17.0×14.0mm | 6×6 Router Mesh | n_to_one mode | 64 pseudo-ch</text>
<text x="485" y="40" text-anchor="middle" font-family="monospace" font-size="15" fill="#ffffff">Per-PE: 8 ch × 32.0 GB/s = 256.0 GB/s | Cube total: 64 × 32.0 = 2048.0 GB/s</text>
<rect x="60" y="60" width="850.0" height="700.0" rx="6" fill="none" stroke="#94a3b8" stroke-width="2" stroke-dasharray="8,4"/>
<rect x="260" y="285" width="450" height="250" rx="6" fill="#ecfdf5" stroke="#047857" stroke-width="2" opacity="0.6"/>
<text x="485" y="395" text-anchor="middle" font-family="monospace" font-size="16" font-weight="bold" fill="#047857">HBM_CTRL | 64 pseudo channels</text>
<text x="485" y="412" text-anchor="middle" font-family="monospace" font-size="14" fill="#059669">Total BW: 2048 GB/s</text>
<rect x="270.0" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="283.4" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="296.9" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="310.3" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="323.8" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="337.2" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="350.6" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="364.1" y="289" width="12.9" height="8" rx="1" fill="#3b82f6" opacity="0.8"/>
<rect x="377.5" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="390.9" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="404.4" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="417.8" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="431.2" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="444.7" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="458.1" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="471.6" y="289" width="12.9" height="8" rx="1" fill="#60a5fa" opacity="0.8"/>
<rect x="485.0" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="498.4" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="511.9" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="525.3" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="538.8" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="552.2" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="565.6" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="579.1" y="289" width="12.9" height="8" rx="1" fill="#475569" opacity="0.8"/>
<rect x="592.5" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<rect x="605.9" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<rect x="619.4" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<rect x="632.8" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<rect x="646.2" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<rect x="659.7" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<rect x="673.1" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<rect x="686.6" y="289" width="12.9" height="8" rx="1" fill="#94a3b8" opacity="0.8"/>
<text x="324" y="286" text-anchor="middle" font-family="monospace" font-size="12" fill="#3b82f6">PE0×8ch</text>
<text x="431" y="286" text-anchor="middle" font-family="monospace" font-size="12" fill="#60a5fa">PE1×8ch</text>
<text x="539" y="286" text-anchor="middle" font-family="monospace" font-size="12" fill="#475569">PE2×8ch</text>
<text x="646" y="286" text-anchor="middle" font-family="monospace" font-size="12" fill="#94a3b8">PE3×8ch</text>
<rect x="270.0" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="283.4" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="296.9" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="310.3" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="323.8" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="337.2" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="350.6" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="364.1" y="523" width="12.9" height="8" rx="1" fill="#f59e0b" opacity="0.8"/>
<rect x="377.5" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="390.9" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="404.4" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="417.8" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="431.2" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="444.7" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="458.1" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="471.6" y="523" width="12.9" height="8" rx="1" fill="#fbbf24" opacity="0.8"/>
<rect x="485.0" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="498.4" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="511.9" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="525.3" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="538.8" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="552.2" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="565.6" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="579.1" y="523" width="12.9" height="8" rx="1" fill="#ef4444" opacity="0.8"/>
<rect x="592.5" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<rect x="605.9" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<rect x="619.4" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<rect x="632.8" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<rect x="646.2" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<rect x="659.7" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<rect x="673.1" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<rect x="686.6" y="523" width="12.9" height="8" rx="1" fill="#f87171" opacity="0.8"/>
<text x="324" y="539" text-anchor="middle" font-family="monospace" font-size="12" fill="#f59e0b">PE4×8ch</text>
<text x="431" y="539" text-anchor="middle" font-family="monospace" font-size="12" fill="#fbbf24">PE5×8ch</text>
<text x="539" y="539" text-anchor="middle" font-family="monospace" font-size="12" fill="#ef4444">PE6×8ch</text>
<text x="646" y="539" text-anchor="middle" font-family="monospace" font-size="12" fill="#f87171">PE7×8ch</text>
<line x1="135" y1="135" x2="285" y2="135" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="135" x2="135" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="135" x2="435" y2="135" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="135" x2="285" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="435" y1="135" x2="585" y2="135" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="435" y1="135" x2="435" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="585" y1="135" x2="685" y2="135" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="585" y1="135" x2="585" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="135" x2="835" y2="135" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="135" x2="685" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="835" y1="135" x2="835" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="260" x2="285" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="260" x2="135" y2="335" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="260" x2="435" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="260" x2="285" y2="335" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="435" y1="260" x2="585" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="435" y1="260" x2="435" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="585" y1="260" x2="685" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="585" y1="260" x2="585" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="260" x2="835" y2="260" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="260" x2="685" y2="335" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="835" y1="260" x2="835" y2="335" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="335" x2="285" y2="335" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="335" x2="135" y2="485" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="335" x2="685" y2="335" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="335" x2="285" y2="485" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="335" x2="835" y2="335" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="335" x2="685" y2="485" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="835" y1="335" x2="835" y2="485" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="485" x2="285" y2="485" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="485" x2="135" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="485" x2="685" y2="485" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="485" x2="285" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="485" x2="835" y2="485" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="485" x2="685" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="835" y1="485" x2="835" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="560" x2="285" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="560" x2="135" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="560" x2="435" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="560" x2="285" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="435" y1="560" x2="585" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="435" y1="560" x2="435" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="585" y1="560" x2="685" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="585" y1="560" x2="585" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="560" x2="835" y2="560" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="560" x2="685" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="835" y1="560" x2="835" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="135" y1="685" x2="285" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="285" y1="685" x2="435" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="435" y1="685" x2="585" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="585" y1="685" x2="685" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<line x1="685" y1="685" x2="835" y2="685" stroke="#94a3b8" stroke-width="1" opacity="0.4"/>
<circle cx="135" cy="135" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="135" y="138" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r0c0</text>
<rect x="119" y="81" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="135" y="92" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE0</text>
<line x1="135" y1="127" x2="149" y2="97" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<circle cx="285" cy="135" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="285" y="138" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r0c1</text>
<rect x="269" y="81" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="285" y="92" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE1</text>
<line x1="285" y1="127" x2="299" y2="97" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<circle cx="435" cy="135" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="435" y="138" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r0c2</text>
<circle cx="585" cy="135" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="585" y="138" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r0c3</text>
<circle cx="685" cy="135" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="685" y="138" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r0c4</text>
<circle cx="835" cy="135" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="835" y="138" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r0c5</text>
<circle cx="135" cy="260" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="135" y="263" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r1c0</text>
<circle cx="285" cy="260" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="285" y="263" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r1c1</text>
<circle cx="435" cy="260" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="435" y="263" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r1c2</text>
<rect x="419" y="206" width="32" height="16" rx="3" fill="#ffffff" stroke="#f59e0b" stroke-width="1"/>
<text x="435" y="217" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#f59e0b">M_CPU</text>
<line x1="435" y1="252" x2="449" y2="222" stroke="#f59e0b" stroke-width="1" opacity="0.6"/>
<circle cx="585" cy="260" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="585" y="263" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r1c3</text>
<circle cx="685" cy="260" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="685" y="263" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r1c4</text>
<rect x="669" y="206" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="685" y="217" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE2</text>
<line x1="685" y1="252" x2="699" y2="222" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<circle cx="835" cy="260" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="835" y="263" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r1c5</text>
<rect x="819" y="206" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="835" y="217" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE3</text>
<line x1="835" y1="252" x2="849" y2="222" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<circle cx="135" cy="335" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="135" y="338" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r2c0</text>
<circle cx="285" cy="335" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="285" y="338" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r2c1</text>
<circle cx="685" cy="335" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="685" y="338" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r2c4</text>
<circle cx="835" cy="335" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="835" y="338" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r2c5</text>
<circle cx="135" cy="485" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="135" y="488" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r3c0</text>
<rect x="119" y="523" width="32" height="16" rx="3" fill="#ffffff" stroke="#d97706" stroke-width="1"/>
<text x="135" y="534" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#d97706">SRAM</text>
<line x1="135" y1="493" x2="149" y2="523" stroke="#d97706" stroke-width="1" opacity="0.6"/>
<circle cx="285" cy="485" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="285" y="488" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r3c1</text>
<circle cx="685" cy="485" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="685" y="488" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r3c4</text>
<circle cx="835" cy="485" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="835" y="488" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r3c5</text>
<circle cx="135" cy="560" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="135" y="563" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r4c0</text>
<rect x="119" y="598" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="135" y="609" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE4</text>
<line x1="135" y1="568" x2="149" y2="598" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<circle cx="285" cy="560" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="285" y="563" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r4c1</text>
<rect x="269" y="598" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="285" y="609" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE5</text>
<line x1="285" y1="568" x2="299" y2="598" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<circle cx="435" cy="560" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="435" y="563" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r4c2</text>
<circle cx="585" cy="560" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="585" y="563" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r4c3</text>
<circle cx="685" cy="560" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="685" y="563" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r4c4</text>
<circle cx="835" cy="560" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="835" y="563" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r4c5</text>
<circle cx="135" cy="685" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="135" y="688" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r5c0</text>
<circle cx="285" cy="685" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="285" y="688" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r5c1</text>
<circle cx="435" cy="685" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="435" y="688" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r5c2</text>
<circle cx="585" cy="685" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="585" y="688" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r5c3</text>
<circle cx="685" cy="685" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="685" y="688" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r5c4</text>
<rect x="669" y="723" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="685" y="734" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE6</text>
<line x1="685" y1="693" x2="699" y2="723" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<circle cx="835" cy="685" r="17" fill="#ffffff" stroke="#94a3b8" stroke-width="1"/>
<text x="835" y="688" text-anchor="middle" font-family="monospace" font-size="12" fill="#1f2937">r5c5</text>
<rect x="819" y="723" width="32" height="16" rx="3" fill="#ffffff" stroke="#a855f7" stroke-width="1"/>
<text x="835" y="734" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#a855f7">PE7</text>
<line x1="835" y1="693" x2="849" y2="723" stroke="#a855f7" stroke-width="1" opacity="0.6"/>
<polyline points="135,143 208,216 251,216 324,289" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="239" y="216" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<polyline points="285,143 358,216 358,216 431,289" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="368" y="216" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<polyline points="685,268 674,278 549,278 539,289" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="622" y="278" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<polyline points="835,268 824,278 657,278 646,289" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="751" y="278" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<polyline points="135,552 146,542 313,542 324,531" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="239" y="542" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<polyline points="285,552 296,542 421,542 431,531" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="368" y="542" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<polyline points="685,677 612,604 612,604 539,531" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="622" y="604" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<polyline points="835,677 762,604 719,604 646,531" fill="none" stroke="#10b981" stroke-width="1.5" opacity="0.6" stroke-dasharray="4,3"/>
<text x="751" y="604" font-family="monospace" font-size="12" fill="#047857">256GB/s</text>
<rect x="65" y="360" width="50" height="100" rx="3" fill="#ffffff" stroke="#475569" stroke-width="1.5" opacity="0.9"/>
<text x="90" y="357" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#475569">UCIe-W</text>
<rect x="67" y="362" width="46" height="23" rx="2" fill="#cbd5e1" opacity="0.7"/>
<text x="90" y="376" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c0</text>
<polyline points="127,135 120,142 120,366 113,374" fill="none" stroke="#cbd5e1" stroke-width="1" opacity="0.5"/>
<rect x="67" y="386" width="46" height="23" rx="2" fill="#94a3b8" opacity="0.7"/>
<text x="90" y="400" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c1</text>
<polyline points="127,260 120,267 120,390 113,398" fill="none" stroke="#94a3b8" stroke-width="1" opacity="0.5"/>
<rect x="67" y="410" width="46" height="23" rx="2" fill="#64748b" opacity="0.7"/>
<text x="90" y="424" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c2</text>
<polyline points="127,560 120,553 120,428 113,422" fill="none" stroke="#64748b" stroke-width="1" opacity="0.5"/>
<rect x="67" y="434" width="46" height="23" rx="2" fill="#374151" opacity="0.7"/>
<text x="90" y="448" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c3</text>
<polyline points="127,685 120,678 120,452 113,446" fill="none" stroke="#374151" stroke-width="1" opacity="0.5"/>
<rect x="435" y="65" width="100" height="50" rx="3" fill="#ffffff" stroke="#475569" stroke-width="1.5" opacity="0.9"/>
<text x="485" y="62" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#475569">UCIe-N</text>
<rect x="437" y="67" width="23" height="46" rx="2" fill="#cbd5e1" opacity="0.7"/>
<text x="448" y="93" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c0</text>
<polyline points="135,127 142,120 442,120 448,113" fill="none" stroke="#cbd5e1" stroke-width="1" opacity="0.5"/>
<rect x="461" y="67" width="23" height="46" rx="2" fill="#94a3b8" opacity="0.7"/>
<text x="472" y="93" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c1</text>
<polyline points="285,127 292,120 466,120 472,113" fill="none" stroke="#94a3b8" stroke-width="1" opacity="0.5"/>
<rect x="485" y="67" width="23" height="46" rx="2" fill="#64748b" opacity="0.7"/>
<text x="496" y="93" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c2</text>
<polyline points="685,127 678,120 504,120 496,113" fill="none" stroke="#64748b" stroke-width="1" opacity="0.5"/>
<rect x="509" y="67" width="23" height="46" rx="2" fill="#374151" opacity="0.7"/>
<text x="520" y="93" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c3</text>
<polyline points="835,127 828,120 528,120 520,113" fill="none" stroke="#374151" stroke-width="1" opacity="0.5"/>
<rect x="855" y="360" width="50" height="100" rx="3" fill="#ffffff" stroke="#475569" stroke-width="1.5" opacity="0.9"/>
<text x="880" y="357" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#475569">UCIe-E</text>
<rect x="857" y="362" width="46" height="23" rx="2" fill="#cbd5e1" opacity="0.7"/>
<text x="880" y="376" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c0</text>
<polyline points="843,135 850,142 850,367 857,374" fill="none" stroke="#cbd5e1" stroke-width="1" opacity="0.5"/>
<rect x="857" y="386" width="46" height="23" rx="2" fill="#94a3b8" opacity="0.7"/>
<text x="880" y="400" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c1</text>
<polyline points="843,260 850,267 850,391 857,398" fill="none" stroke="#94a3b8" stroke-width="1" opacity="0.5"/>
<rect x="857" y="410" width="46" height="23" rx="2" fill="#64748b" opacity="0.7"/>
<text x="880" y="424" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c2</text>
<polyline points="843,560 850,553 850,428 857,422" fill="none" stroke="#64748b" stroke-width="1" opacity="0.5"/>
<rect x="857" y="434" width="46" height="23" rx="2" fill="#374151" opacity="0.7"/>
<text x="880" y="448" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c3</text>
<polyline points="843,685 850,678 850,452 857,446" fill="none" stroke="#374151" stroke-width="1" opacity="0.5"/>
<rect x="435" y="705" width="100" height="50" rx="3" fill="#ffffff" stroke="#475569" stroke-width="1.5" opacity="0.9"/>
<text x="485" y="702" text-anchor="middle" font-family="monospace" font-size="14" font-weight="bold" fill="#475569">UCIe-S</text>
<rect x="437" y="707" width="23" height="46" rx="2" fill="#cbd5e1" opacity="0.7"/>
<text x="448" y="733" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c0</text>
<polyline points="135,693 142,700 442,700 448,707" fill="none" stroke="#cbd5e1" stroke-width="1" opacity="0.5"/>
<rect x="461" y="707" width="23" height="46" rx="2" fill="#94a3b8" opacity="0.7"/>
<text x="472" y="733" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c1</text>
<polyline points="285,693 292,700 466,700 472,707" fill="none" stroke="#94a3b8" stroke-width="1" opacity="0.5"/>
<rect x="485" y="707" width="23" height="46" rx="2" fill="#64748b" opacity="0.7"/>
<text x="496" y="733" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c2</text>
<polyline points="685,693 678,700 504,700 496,707" fill="none" stroke="#64748b" stroke-width="1" opacity="0.5"/>
<rect x="509" y="707" width="23" height="46" rx="2" fill="#374151" opacity="0.7"/>
<text x="520" y="733" text-anchor="middle" font-family="monospace" font-size="10" fill="#1f2937">c3</text>
<polyline points="835,693 828,700 528,700 520,707" fill="none" stroke="#374151" stroke-width="1" opacity="0.5"/>
<rect x="60" y="775" width="10" height="10" rx="2" fill="#3b82f6" stroke="#94a3b8" stroke-width="0.5"/>
<text x="74" y="784" font-family="monospace" font-size="13" fill="#1f2937">PE Router</text>
<rect x="147" y="775" width="10" height="10" rx="2" fill="#f59e0b" stroke="#94a3b8" stroke-width="0.5"/>
<text x="161" y="784" font-family="monospace" font-size="13" fill="#1f2937">M_CPU / SRAM</text>
<rect x="255" y="775" width="10" height="10" rx="2" fill="#475569" stroke="#94a3b8" stroke-width="0.5"/>
<text x="269" y="784" font-family="monospace" font-size="13" fill="#1f2937">UCIe</text>
<rect x="307" y="775" width="10" height="10" rx="2" fill="#1f2937" stroke="#94a3b8" stroke-width="0.5"/>
<text x="321" y="784" font-family="monospace" font-size="13" fill="#1f2937">Relay</text>
<rect x="366" y="775" width="10" height="10" rx="2" fill="#10b981" stroke="#94a3b8" stroke-width="0.5"/>
<text x="380" y="784" font-family="monospace" font-size="13" fill="#1f2937">HBM Link</text>
<rect x="446" y="775" width="10" height="10" rx="2" fill="#1f2937" stroke="#94a3b8" stroke-width="0.5"/>
<text x="460" y="784" font-family="monospace" font-size="13" fill="#1f2937">Mesh Link</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 30 KiB

docs/report/1H-codesign-paper/figures/sip_architecture.pdf
BIN
View File
Binary file not shown.
docs/report/1H-codesign-paper/figures/sip_architecture.svg
+94
View File
@@ -0,0 +1,94 @@
<svg xmlns="http://www.w3.org/2000/svg" width="508" height="495" viewBox="70 40 508 495">
<title>sip</title>
<rect width="648" height="648" fill="#ffffff"/>
<line x1="108.0" y1="144.0" x2="252.0" y2="144.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="180.0" y="140.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="108.0" y1="144.0" x2="108.0" y2="264.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="108.0" y="200.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="252.0" y1="144.0" x2="396.0" y2="144.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="324.0" y="140.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="252.0" y1="144.0" x2="252.0" y2="264.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="252.0" y="200.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="396.0" y1="144.0" x2="540.0" y2="144.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="468.0" y="140.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="396.0" y1="144.0" x2="396.0" y2="264.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="396.0" y="200.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="540.0" y1="144.0" x2="540.0" y2="264.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="540.0" y="200.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="108.0" y1="264.0" x2="252.0" y2="264.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="180.0" y="260.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="108.0" y1="264.0" x2="108.0" y2="384.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="108.0" y="320.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="252.0" y1="264.0" x2="396.0" y2="264.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="324.0" y="260.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="252.0" y1="264.0" x2="252.0" y2="384.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="252.0" y="320.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="396.0" y1="264.0" x2="540.0" y2="264.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="468.0" y="260.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="396.0" y1="264.0" x2="396.0" y2="384.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="396.0" y="320.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="540.0" y1="264.0" x2="540.0" y2="384.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="540.0" y="320.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="108.0" y1="384.0" x2="252.0" y2="384.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="180.0" y="380.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="108.0" y1="384.0" x2="108.0" y2="504.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="108.0" y="440.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="252.0" y1="384.0" x2="396.0" y2="384.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="324.0" y="380.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="252.0" y1="384.0" x2="252.0" y2="504.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="252.0" y="440.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="396.0" y1="384.0" x2="540.0" y2="384.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="468.0" y="380.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="396.0" y1="384.0" x2="396.0" y2="504.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="396.0" y="440.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="540.0" y1="384.0" x2="540.0" y2="504.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="540.0" y="440.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="108.0" y1="504.0" x2="252.0" y2="504.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="180.0" y="500.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="252.0" y1="504.0" x2="396.0" y2="504.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="324.0" y="500.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<line x1="396.0" y1="504.0" x2="540.0" y2="504.0" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="468.0" y="500.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">1.0mm 512GB/s</text>
<polyline points="324.0,56.0 108.0,56.0 108.0,144.0" fill="none" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="216.0" y="96.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">2.5mm 512GB/s</text>
<polyline points="324.0,56.0 252.0,56.0 252.0,144.0" fill="none" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="288.0" y="96.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">2.5mm 512GB/s</text>
<polyline points="324.0,56.0 396.0,56.0 396.0,144.0" fill="none" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="360.0" y="96.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">2.5mm 512GB/s</text>
<polyline points="324.0,56.0 540.0,56.0 540.0,144.0" fill="none" stroke="#000000" stroke-width="1" opacity="0.8"/>
<text x="432.0" y="96.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#000000">2.5mm 512GB/s</text>
<rect x="84.0" y="128.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="108.0" y="148.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (0,0)</text>
<rect x="228.0" y="128.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="252.0" y="148.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (1,0)</text>
<rect x="372.0" y="128.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="396.0" y="148.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (2,0)</text>
<rect x="516.0" y="128.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="540.0" y="148.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (3,0)</text>
<rect x="84.0" y="248.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="108.0" y="268.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (0,1)</text>
<rect x="228.0" y="248.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="252.0" y="268.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (1,1)</text>
<rect x="372.0" y="248.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="396.0" y="268.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (2,1)</text>
<rect x="516.0" y="248.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="540.0" y="268.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (3,1)</text>
<rect x="84.0" y="368.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="108.0" y="388.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (0,2)</text>
<rect x="228.0" y="368.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="252.0" y="388.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (1,2)</text>
<rect x="372.0" y="368.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="396.0" y="388.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (2,2)</text>
<rect x="516.0" y="368.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="540.0" y="388.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (3,2)</text>
<rect x="84.0" y="488.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="108.0" y="508.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (0,3)</text>
<rect x="228.0" y="488.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="252.0" y="508.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (1,3)</text>
<rect x="372.0" y="488.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="396.0" y="508.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (2,3)</text>
<rect x="516.0" y="488.0" width="48.0" height="32.0" rx="4" fill="#cbd5e1" stroke="#000000" stroke-width="1"/>
<text x="540.0" y="508.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#1e293b">CUBE (3,3)</text>
<rect x="308.0" y="50.0" width="32.0" height="12.0" rx="4" fill="#0ea5e9" stroke="#000000" stroke-width="1"/>
<text x="324.0" y="60.0" text-anchor="middle" font-family="monospace" font-size="10" fill="#ffffff">IO io0</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

docs/report/1H-codesign-paper/main.pdf
BIN
View File
Binary file not shown.
docs/report/1H-codesign-paper/sections/01-introduction.tex
+6 -3
View File
@@ -25,9 +25,12 @@ serves as the common evaluation platform for all mechanisms and kernels
discussed in this study.
This report focuses on Grouped-Query Attention (GQA), one of the most
performance- and bandwidth-critical components of LLM inference. Modern
decoder-only models such as Llama~3 and Mistral have largely transitioned
from GPT-3-style multi-head attention (MHA) to GQA, in which multiple
performance- and bandwidth-critical components of LLM inference. GQA
dominates inference-time memory traffic and KV-cache capacity in modern
LLM serving, making it the primary bandwidth bottleneck on memory-centric
architectures such as AHBM. Modern decoder-only models such as Llama~3
and Mistral have largely transitioned from GPT-3-style multi-head
attention (MHA) to GQA, in which multiple
query heads share a single KV head to reduce KV cache capacity and
memory-bandwidth requirements. While GQA improves system efficiency at
the model level, mapping it efficiently onto AHBM introduces three
docs/report/1H-codesign-paper/sections/01-introduction_ko.tex
+73
View File
@@ -0,0 +1,73 @@
\section{서론}
\label{sec:intro}
AHBM은 연산 유닛을 HBM 스택 내부에 직접 통합한 메모리 중심 가속기
아키텍처이다. 각 처리 요소(PE)는 전용 HBM 슬라이스와 짝지어지며,
TCM과 SRAM을 거쳐 메모리와 MAC 어레이 사이의 데이터를 단계적으로
이동시킨다. 이러한 메모리 중심 구조에서 커널 성능은 연산 처리량만이
아니라, 메모리 계층과 PE 사이에 데이터를 얼마나 효율적으로 배치하고,
이동시키며, 공유하는가에 의해 결정된다. 따라서 AHBM에서의 AI 커널
최적화는 커널 알고리즘과 아키텍처 메커니즘을 함께 발전시키는
하드웨어--소프트웨어 코디자인을 필요로 한다.
상세한 성능 분석과 신속한 설계 탐색을 위해, 우리는
\textbf{KernBench}---AHBM을 위한 소스 수준 이산 사건 시뮬레이션
플랫폼---을 개발하였다. KernBench는 메모리 시스템 지연, PE 실행
모델, PE 간 통신, 호스트 측 오케스트레이션을 포함한 AHBM 실행 모델을
구현하면서, 커널과 호스트 소프트웨어를 소스 코드로부터 직접 실행한다.
이를 통해 실행 거동을 세밀하게 관찰할 수 있을 뿐만 아니라, 컴파일러나
런타임과 같은 상위 소프트웨어 스택과 독립적으로 하드웨어--소프트웨어
코디자인 선택을 체계적으로 평가할 수 있다. 본 보고서에 제시된 모든
결과는 KernBench를 사용하여 얻은 것이며, KernBench는 본 연구에서
다루는 모든 메커니즘과 커널의 공통 평가 플랫폼 역할을 한다.
본 보고서는 LLM 추론에서 가장 성능 및 대역폭 임계 구성요소 중 하나인
Grouped-Query Attention(GQA)에 초점을 맞춘다. Llama~3, Mistral과
같은 최신 디코더 전용 모델은 GPT-3 형태의 다중-헤드 어텐션(MHA)으로
부터 GQA---여러 질의 헤드가 하나의 KV 헤드를 공유함으로써 KV 캐시
용량과 메모리 대역폭 요구량을 줄이는 방식---로 대거 전환되었다. GQA는
모델 수준에서 시스템 효율을 개선하지만, 이를 AHBM에 효율적으로
매핑하기 위해서는 세 가지 아키텍처 요구사항이 발생한다: PE 간 통신을
최소화하기 위한 KV 캐시와 가중치의 최적 배치, 불가피한 PE 간 트래픽에
대한 저오버헤드 지원, 그리고 각 PE 내에서 메모리 접근과 연산의
효율적인 파이프라이닝이다.
이러한 요구사항을 충족하기 위해, 본 보고서는 세 가지 하드웨어--
소프트웨어 코디자인 메커니즘을 제안한다. 첫째, GQA-aware 데이터
배치는 TCM/SRAM/HBM 계층에 KV 캐시와 가중치를 분산 배치하여 통신
오버헤드를 줄이고 데이터 지역성을 향상시킨다. 둘째, PE\_IPCQ는 리덕션
등 통신 집약적 연산을 위한 효율적인 온디바이스 집합 통신 프리미티브를
제공한다. 셋째, composite-command GEMM 파이프라인은 PE\_SCHEDULER의
제어 하에 각 PE 내부에서 메모리 이동과 연산을 긴밀하게 파이프라이닝
하여, 명령 오버헤드를 줄이면서도 MAC 어레이의 가동률을 효율적으로
유지한다.
연산 측 인에이블러(composite-command GEMM 파이프라인,
\S\ref{sec:gemm})와 통신 측 인에이블러(PE\_IPCQ,
\S\ref{sec:allreduce})는 우선 독립적으로 개발 및 평가된다. 이후 융합
GQA 커널(\S\ref{sec:gqa})은 이들을 GQA-aware 데이터 배치와 결합하여
AHBM 상에서 종단간 어텐션 구현을 시연한다. 대응 관계는 직접적이다:
어텐션의 $QK^{\top}$와 $PV$ 곱은 정확히 composite-command 파이프라인이
이득을 주는 GEMM이며, KV 리덕션은 정확히 PE\_IPCQ가 이득을 주는 집합
연산이다. 이러한 메커니즘들이 결합되어, 융합 GQA 커널은 AHBM의 HBM
대역폭을 효율적으로 활용할 수 있게 된다.
본 연구에서 GQA가 주된 동기 부여 워크로드 역할을 하지만, 도출된
메커니즘들은 훨씬 광범위한 AI 커널 분류군에서 재사용 가능한 구성
요소로 의도되었다. PE\_IPCQ는 분산 및 통신 집약적 워크로드 전반에
걸쳐 집합 통신을 지원할 수 있으며, composite-command 실행은 GEMM
기반 커널, 피드포워드 네트워크(FFN), 정규화, 그 외 융합 연산자
파이프라인에 적용 가능하다. 계층적 데이터 배치 프레임워크와 함께 이
메커니즘들은 AHBM에서의 향후 AI 커널과 통신 라이브러리의 기반을
형성한다. 2026년 하반기에는 이 기반을 FFN 및 MoE 주도 워크로드와
완전한 LLM 실행의 종단간 최적화로 확장할 예정이다.
본 보고서의 나머지 부분은 다음과 같이 구성된다.
\S\ref{sec:platform}에서는 KernBench 플랫폼과 본 연구 전반에 걸쳐
사용된 AHBM 구성을 기술한다. \S\ref{sec:gemm},
\S\ref{sec:allreduce}, \S\ref{sec:gqa}에서는 각각 composite-command
GEMM 파이프라인, PE\_IPCQ 집합 통신, 융합 GQA 커널을 다룬다.
\S\ref{sec:discussion}에서는 이러한 결과의 광범위한 아키텍처적 함의를
논의한다. 마지막으로 \S\ref{sec:conclusion}과 \S\ref{sec:future}에서는
주요 결과를 요약하고 FFN, MoE, 전체 모델 최적화에 관한 향후 과제를
제시한다.
docs/report/1H-codesign-paper/sections/02-platform.tex
+138 -91
View File
@@ -1,16 +1,15 @@
\begin{figure*}[t]
\centering
\begin{subfigure}[b]{0.42\textwidth}
\begin{subfigure}[b]{0.495\textwidth}
\centering
\includegraphics[width=\linewidth,height=0.6\linewidth,keepaspectratio]{sip_architecture.pdf}
\caption{SIP level: $4{\times}4$ CUBE mesh + IO chiplet.}
\includegraphics[width=\linewidth,height=0.7\linewidth,keepaspectratio]{sip_architecture.pdf}
\caption{SIP Architecture}
\label{fig:sip-arch}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.42\textwidth}
\centering
\includegraphics[width=\linewidth,height=0.6\linewidth,keepaspectratio]{cube_architecture.pdf}
\caption{CUBE level (zoom-in of one CUBE in (a)): 8 PEs, HBM
channels, M\_CPU, SRAM, NoC router mesh, UCIe links.}
\includegraphics[width=\linewidth,height=0.84\linewidth]{cube_architecture.pdf}
\caption{CUBE Architecture}
\label{fig:cube-arch}
\end{subfigure}
@@ -26,9 +25,12 @@
on-device collective control plane.}
\label{fig:pe-arch}
\end{subfigure}
\caption{Modeled hardware graph at the SIP, CUBE, and PE levels
(one example configuration; specific parameters in \S\ref{sec:hw} /
Table~\ref{tab:hw}). KernBench is not tied to this particular
\caption{Modeled hardware graph at the SIP, CUBE, and PE levels.
This figure is an \emph{illustrative topology} chosen for
readability; the \emph{experimental configuration} used throughout
this report (port counts, slice counts, and link parameters) is
specified in \S\ref{sec:hw} / Table~\ref{tab:hw}, and numbers in the
two may differ. KernBench is not tied to this particular
arrangement: each box is a modeled component node, each line a
directed link with bandwidth and propagation attributes, and any
topology that respects those attributes is supported.}
@@ -144,8 +146,20 @@ KernBench is layered along the flow of a request:
controllers, and the inter-chiplet links.
\end{itemize}
Data and timing are handled in two passes, so that a kernel's numeric
results and its latency are computed consistently but independently.
Data and timing are handled in two passes, so that a kernel's
numeric results and its latency are computed consistently but
independently. \textbf{Pass~1 (timing)} runs the kernel under
the discrete-event engine: memory ops (\textsf{tl.load},
\textsf{tl.store}) execute against a host-side \textsf{MemoryStore}
and return real tensor data, while compute ops (GEMM, vector-math)
only emit records into an op-log carrying their operands, shapes,
dtypes, and scheduled time. \textbf{Pass~2 (data)} replays that
op-log offline in numpy, producing the actual numeric outputs and
comparing them against a reference computation under per-dtype
tolerances (e.g.\ \texttt{rtol}/\texttt{atol} $=10^{-3}$ for f16).
Pass~2 is optional---runs that need only latency skip it---but when
enabled it guarantees that every reported timing number corresponds
to a kernel whose numeric output has been verified end-to-end.
\subsection{Latency model: graph traversal and contention}
\label{sec:latency}
@@ -185,7 +199,7 @@ fires them one at a time, with ties broken under a deterministic
policy so that the same kernel on the same topology always yields the
same trace. Per-request correlation IDs are stamped at injection and
carried through every hop, so the path from injection to completion
is fully traceable. Every nanosecond in a reported latency
is fully traceable. Every modeled latency contribution
corresponds to exactly one of these events on exactly one node or
edge---there is no slack in the budget.
@@ -202,6 +216,65 @@ collective engines hold the request for their service time before
releasing it downstream. Each of these is attached to a specific node
or edge in the graph; together they make up the entire latency budget.
Beyond data movement and execution latency, KernBench
also models the control-plane cost required to issue work
to accelerator engines. Since every DMA, GEMM, vector-
math, and IPCQ operation is initiated through the
\textsf{PE\_CPU} $\rightarrow$ \textsf{PE\_SCHED} path,
command dispatch latency contributes to the end-to-end
execution time of all kernels.
\paragraph{Command dispatch overhead model.} The cost incurred by
the \textsf{PE\_CPU} when it dispatches a command to one of the
accelerator engines is modelled structurally rather than with a
per-operation calibration table. The \textsf{PE\_CPU} charges, per
command,
\[
d_{\text{cmd}} = \textsf{FIXED} + b_{\text{logical}} \cdot R,
\]
This linear-in-size form reflects the serialization cost of writing a
command descriptor into the scheduler queue: a fixed per-command
bookkeeping cost plus a byte-wise descriptor-transfer cost.
Concretely, $b_{\text{logical}}$ is the command's hardware-logical byte
size, \textsf{FIXED} captures the fixed per-command cost (queue-tail
update, completion registration) and $R$ captures the per-byte cost of
serializing the command descriptor into the scheduler queue. This
\textsf{FIXED} is distinct from Table~\ref{tab:hw}'s
\SI{2}{\nano\second} / \SI{1}{\nano\second} \textsf{PE\_CPU} /
\textsf{PE\_SCHED} fixed costs: the latter are per-component
traversal overheads each command pays when it transits those nodes,
whereas the 40-cycle \textsf{FIXED} term is the command-descriptor
issue cost. The
default anchoring (\textsf{FIXED} $= 40$ cycles, $R = 0.0625$
cycles/byte, i.e.\ \SI{16}{\byte\per\cycle}, at \SI{1}{\giga\hertz})
places a typical composite at roughly \SI{43}{\nano\second}, and a
hard cap on a composite's descriptor size prevents the model from
rewarding arbitrarily large fused commands beyond what real descriptor
queues accept. In the configurations measured here, command issue is
not the bottleneck---data movement is---so this term stays small
relative to DMA and collective time.
\begin{table*}[t]
\centering
\caption{PE\,$\to$\,HBM DMA latency probe at varying hop distances
(\SI{32}{\kibi\byte} transfer; output captured from \texttt{kernbench
probe}). \emph{Util\%} is the achieved bandwidth as a fraction of the
path's bottleneck-edge bandwidth.}
\label{tab:probe-pe-dma}
\small
\begin{tabular}{@{}lrrr@{}}
\toprule
Case & Latency~(\si{\nano\second}) & Util\% (\,32\,KiB) & Util\% (\,1\,MiB) \\
\midrule
PE\,$\to$\,local HBM & 141.0 & 90.8 & 100.0 \\
PE\,$\to$\,same-half HBM & 147.9 & 86.6 & ~99.9 \\
PE\,$\to$\,cross-half HBM & 161.2 & 79.4 & ~99.8 \\
PE\,$\to$\,cross-CUBE (best) & 330.5 & 77.5 & ~99.6 \\
PE\,$\to$\,cross-CUBE (worst) & 677.1 & 37.8 & ~97.8 \\
\bottomrule
\end{tabular}
\end{table*}
\subsubsection{Congestion and contention modeling}
\label{sec:congestion}
@@ -225,26 +298,6 @@ it reveals where the real bottlenecks form and which hardware levers
actually relieve them---which is exactly the question the codesign
work in this report turns on.
\paragraph{Command dispatch overhead model.} The cost incurred by
the \textsf{PE\_CPU} when it dispatches a command to one of the
accelerator engines is modelled structurally rather than with a
per-operation calibration table. The \textsf{PE\_CPU} charges, per
command,
\[
d_{\text{cmd}} = \textsf{FIXED} + b_{\text{logical}} \cdot R,
\]
where $b_{\text{logical}}$ is the command's hardware-logical byte size,
\textsf{FIXED} captures the fixed per-command cost (queue-tail update,
completion registration) and $R$ captures the per-byte cost of
serializing the command descriptor into the scheduler queue. The
default anchoring (\textsf{FIXED} $= 40$ cycles, $R = 0.0625$
cycles/byte, i.e.\ \SI{16}{\byte\per\cycle}, at \SI{1}{\giga\hertz})
places a typical composite at roughly \SI{43}{\nano\second}, and a
hard cap on a composite's descriptor size prevents the model from
rewarding arbitrarily large fused commands beyond what real descriptor
queues accept. In the configurations measured here, command issue is
not the bottleneck---data movement is---so this term stays small
relative to DMA and collective time.
\subsection{Accuracy}
\label{sec:accuracy}
@@ -254,41 +307,29 @@ dominate kernel latency on this class of hardware: per-edge bandwidth
occupancy and flit-level serialization, HBM pseudo-channel parallelism,
and per-component switching overhead. Two independent cross-checks
drawn from the experiments in this report confirm that this precision
translates into physically reasonable kernel latencies. First, in the
GEMM study (\S\ref{sec:gemm}), simulator-measured MAC efficiency
tracks an analytic ideal-pipeline model within roughly
translates into physically reasonable kernel latencies.
First, in the GEMM study (\S\ref{sec:gemm}), simulator-measured MAC
efficiency tracks an analytic ideal-pipeline model within roughly
\SIrange{10}{20}{\percent} across a wide range of tile counts; the
residual gap is attributable to pipeline-fill and DMA effects the
analytic model omits. Second, in the all-reduce study
(\S\ref{sec:allreduce}, Fig.~\ref{fig:allreduce-cmp}), simulator
latency for a 2D-torus over six devices follows the expected
startup-plus-per-packet shape across the entire payload sweep---tight
at small payloads where startup dominates, and within a single-digit
multiplicative factor at the largest payloads, where the residual gap
is explained by per-router switching the analytic shape elides. A
single-device point from an external full-system simulator (FSIM) at
the largest payload sits an order of magnitude above the KernBench
multi-device torus, illustrating the well-known gap between an
achievable-kernel number and a full end-to-end-stack number rather
than a model error. The known simplifications---FIFO router
arbitration (instead of round-robin), HBM scheduler without
write-buffer reordering, no bank conflict, no refresh or thermal
effects, and no upstream backpressure---are the price of a
deterministic, inspectable model. These simplifications bound the
absolute accuracy, but the agreement with both analytic models and the
external full-system simulator data above indicates that KernBench is
sufficiently accurate for evaluating the \emph{relative}
hardware--software design trade-offs (tiling A vs.\ B, topology X
vs.\ Y, with vs.\ without composite command, mesh vs.\ torus) that
are the primary objective of this work.
analytic model omits.
A third source of confidence comes from directly probing the
simulator's per-traversal behaviour. Running \texttt{kernbench probe}
on the modelled topology issues a sequence of PE-to-HBM DMA reads at
progressively greater hop distances and reports the per-component
overhead, per-edge serialization, and per-PC drain that the model
charges (Table~\ref{tab:probe-pe-dma}). Three properties stand out.
First, the reported latency increases \emph{monotonically} with hop
Second, in the all-reduce study (\S\ref{sec:allreduce},
Fig.~\ref{fig:allreduce-cmp}), simulator latency for a 2D-torus over
six devices follows the expected startup-plus-per-packet shape across
the entire payload sweep---tight at small payloads where startup
dominates, and within a single-digit multiplicative factor at the
largest payloads, where the residual gap is explained by per-router
switching the analytic shape elides.
Third, the simulator's per-traversal behaviour can be probed
directly. Running \texttt{kernbench probe} on the modelled topology
issues a sequence of PE-to-HBM DMA reads at progressively greater
hop distances and reports the per-component overhead, per-edge
serialization, and per-PC drain that the model charges
(Table~\ref{tab:probe-pe-dma}). Three properties stand out. First,
the reported latency increases \emph{monotonically} with hop
count---from \SI{141}{\nano\second} at the local HBM slice to
\SI{677}{\nano\second} at the worst-case remote-CUBE slice---with the
increment per added hop matching the per-router overhead and the
@@ -308,26 +349,19 @@ self-consistent---a model error in any of them would surface as a
non-monotonic or under-utilising curve here long before it polluted a
kernel-level measurement.
\begin{table*}[t]
\centering
\caption{PE\,$\to$\,HBM DMA latency probe at varying hop distances
(\SI{32}{\kibi\byte} transfer; output captured from \texttt{kernbench
probe}). \emph{Util\%} is the achieved bandwidth as a fraction of the
path's bottleneck-edge bandwidth.}
\label{tab:probe-pe-dma}
\small
\begin{tabular}{@{}lrrr@{}}
\toprule
Case & Latency~(\si{\nano\second}) & Util\% (\,32\,KiB) & Util\% (\,1\,MiB) \\
\midrule
PE\,$\to$\,local HBM & 141.0 & 90.8 & 100.0 \\
PE\,$\to$\,same-half HBM & 147.9 & 86.6 & ~99.9 \\
PE\,$\to$\,cross-half HBM & 161.2 & 79.4 & ~99.8 \\
PE\,$\to$\,cross-CUBE (best) & 330.5 & 77.5 & ~99.6 \\
PE\,$\to$\,cross-CUBE (worst) & 677.1 & 37.8 & ~97.8 \\
\bottomrule
\end{tabular}
\end{table*}
The known simplifications---FIFO router arbitration (instead of
round-robin), HBM scheduler without write-buffer reordering, no bank
conflict, no refresh or thermal effects, and no upstream
backpressure---are the price of a deterministic, inspectable model.
These simplifications bound the absolute accuracy, but the agreement
with both analytic models and the probe's internal-consistency
checks above indicates that KernBench is sufficiently accurate for
evaluating the \emph{relative} hardware--software design trade-offs
(tiling A vs.\ B, topology X vs.\ Y, with vs.\ without composite
command, mesh vs.\ torus) that are the primary objective of this
work.
\subsection{Modeled hardware configuration}
\label{sec:hw}
@@ -344,10 +378,22 @@ such that inference workloads can effectively saturate
the available HBM bandwidth. The aggregate compute
throughput is therefore balanced against memory-system
bandwidth rather than being intentionally over- or
under-provisioned. HBM bandwidth is distributed evenly
across the PEs within a CUBE, with each PE responsible
for servicing approximately one-eighth of the CUBE's
memory bandwidth through its dedicated HBM channels.
under-provisioned. Concretely, 8 PEs $\times$
\SI{8}{\tera\flop\per\second} give roughly
\SI{64}{\tera\flop\per\second} of f16 compute per CUBE
against \SI{2048}{\giga\byte\per\second} of HBM bandwidth,
which places the balanced point at about
$\sim$31~FLOP/byte; inference decode kernels typically
operate well below that, so HBM bandwidth---not compute---is
the natural ceiling on this configuration. For reference, the
GQA decode kernels evaluated in \S\ref{sec:gqa} operate at
arithmetic intensity well below this balance point, so their
ceiling is set by HBM and inter-PE traffic rather than by GEMM
throughput. HBM bandwidth
is distributed evenly across the PEs within a CUBE, with
each PE responsible for servicing approximately one-eighth
of the CUBE's memory bandwidth through its dedicated HBM
channels.
The on-chip interconnect is configured using bandwidth
and latency parameters representative of commercially
@@ -371,7 +417,7 @@ payload sizes are introduced in their respective sections.
\begin{table}[t]
\centering
\caption{Modeled hardware configuration (shared by all experiments).}
\caption{Modeled hardware configuration}
\label{tab:hw}
\small
\begin{tabular}{@{}ll@{}}
@@ -389,11 +435,12 @@ GEMM engine peak & \SI{8}{\tera\flop\per\second} (f16) \\
TCM (on-PE) & \SI{16}{\mega\byte}, \SI{512}{\giga\byte\per\second} R/W \\
\quad kernel scratch & \SI{1}{\mega\byte} \\
DMA engines & 1 read + 1 write \\
CPU / scheduler overhead & \SI{2}{\nano\second} / \SI{1}{\nano\second} \\
\textsf{PE\_CPU} fixed cost & \SI{2}{\nano\second} \\
\textsf{PE\_SCHED} fixed cost & \SI{1}{\nano\second} \\
\midrule
\multicolumn{2}{@{}l}{\emph{Memory (per CUBE)}} \\
HBM capacity & \SI{48}{\giga\byte} (8 slices) \\
HBM aggregate BW & \SI{1024}{\giga\byte\per\second} \\
HBM aggregate BW & \SI{2048}{\giga\byte\per\second} \\
HBM pseudo-channels & 64 (8 per PE), \SI{32}{\giga\byte\per\second} each \\
SRAM (shared) & \SI{32}{\mega\byte}, \SI{128}{\giga\byte\per\second} link \\
HBM burst & \SI{256}{\byte} \\