09721040caa906eb580921f096807c06d65c1eec
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ywkang
|
b6315c3c90 |
paper(1H): reflect D8 single-op cost model in §2/§3.4 + figure/diagram regen
Two strands bundled as the 1H-codesign-paper refresh unit: (A) This session — single-op cost-model reflection (depends on |
||
|
ywkang
|
e8b7f4f064 |
paper(gemm): per-PE HBM BW + TFLOPS evaluation plots; unify peak to 8 TFLOP/s
Two new evaluation plots for §3 GEMM, both using the composite window
as the denominator (the up-front tl.load of A in load_ref is therefore
excluded — only HBM traffic and compute inside the composite count):
- gemm_hbm_bw_util.png: per-PE HBM bandwidth utilization against the
256 GB/s per-PE ceiling. Answers "is this workload memory-bound on
this configuration?" — small/single-tile shapes sit at 22–46%
(pipeline-fill-limited), large or output-write-dominated shapes
saturate (≥85%).
- gemm_per_pe_tflops.png: per-PE GEMM throughput against the 8 TFLOP/s
engine peak. The deep-K K=3072 load_ref case lands at
7.18 TFLOP/s (~90% of peak) — the configuration's clean win.
ref_ref at the same shape drops to 3.83 TFLOP/s because the second
operand doubles HBM pressure and the kernel hits the BW ceiling.
The variant gap on this chart is the operational cost of NOT
pre-staging the activation.
Both charts compare two operand-staging variants on every shape:
load_ref ("activation pre-staged", weight-only HBM streaming) and
ref_ref (both A and W streamed from HBM). load_load is omitted —
with both operands pre-staged the composite carries no HBM traffic
and the BW metric collapses to 0.
Analytic / measured peak unified to the single hardware spec
(8.0 TFLOP/s = 8000 flops/ns). T_STAGE is now derived as
tile_flops / peak (= 16.384 ns for the 32×64×32 tile) rather than
the previous hardcoded 16 ns, so MAC efficiency and per-PE TFLOPS%
share the same denominator and never disagree. Side effect: max
analytic-vs-measured gap tightens from 2.2 ppt to 1.4 ppt across
the sweep.
§3 Results restructured: lead with the two new evaluation plots
(BW saturation, achieved throughput), then the MAC-utilization
analytic-vs-measured chart serves as the validation step, then the
per-stage engine wall-clock as the supporting diagnostic. §3
Analysis updated to use the load_ref / ref_ref contrast as the
hardware-software boundary line that motivates the GQA kernel of §5.
§2.4 Accuracy claim updated to match: GEMM analytic-vs-measured
agreement is now "within 1.4 ppt across every swept shape", from
~7.7% at single-tile up to ~90% at K=3072. Subtitle/title vertical
stacking glitch in the matplotlib charts fixed in passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
ywkang
|
23992548f7 |
gemm(perf): fix back-to-back DMA pipelining + tighten analytic model
Three coupled fixes that recover small-tile GEMM pipeline efficiency from 53% to 88% (32x3072x32 load_ref, composite_window basis). 1. PE_DMA channel-hold (ADR-0014 D4 clarified): both the _handle_with_hooks (PeInternalTxn) and _pipeline_process (TileToken) paths used to hold the cap=1 DMA channel through the full HBM round-trip, which double-serialized with the HBM_CTRL's own per-PC `available_at` model and prevented back-to-back tile DMAs from amortizing their per-request head latency. Channel is now released after the request is enqueued onto the next hop; HBM serialization is HBM_CTRL's responsibility alone. Tests: new test_pe_dma_back_to_back_pipelining as the oracle (asserts wall < 75% of strict-serialized N x single_op). Existing test_pe_dma_record_start_after_channel_acquire rewritten to assert t_start clustering (channel released fast) instead of the old round-trip-hold invariant. test_pe_dma_same_channel_serializes still passes — HBM_CTRL preserves ordering. Probe regression: PE→local-HBM 32 KiB stays at 141 ns (single-request, unaffected). 2. milestone_1h_gemm bench: matmul_composite was reading MATMUL_M/K/N env vars at module load, so every sweep row replayed the cached 256³ result; values now read inside run(). Drops the stale sys.modules deletion hack. 3. Analytic ideal-pipeline model: dropped the (n_mn-1)·dma_w_per_pair penalty (over-pessimistic for under-tile shapes — it pushed measured > theoretical) and replaced the D_STAGES-derived head with empirical T_PIPELINE_FILL=60 ns / T_PIPELINE_TAIL=30 ns. Max analytic-vs-measured gap across all 7 swept shapes now 2.2 ppt (was 9-44 ppt under the old constants). Paper updates: - §3 (GEMM): 78%→88% measured at 48 tiles, 23%→15% at 1 tile, stage breakdown numbers refreshed (DMA in / Fetch / GEMM all ~785 ns at K=3072), analytic-vs-measured agreement tightened to "within 2.2 ppt". - §2.4 (Accuracy): GEMM tracking claim refreshed accordingly. - §5 (GQA): restore long-ctx 4-cases figures into figures/ (they were dropped from bench output dir as derived artifacts in |
||
|
ywkang
|
6b6e29968a |
paper: §2 platform deep-edit + §1 GQA framing + figure regen
§2 (KernBench Platform): - Fix Table 2 HBM aggregate BW (1024 → 2048 GB/s); drop stale hbm_total_bw_gbs from topology.yaml (never read by sim_engine) - Split PE_CPU / PE_SCHED fixed-cost row; disambiguate from the 40-cycle command-dispatch FIXED term - §2.2 two-pass: expand to describe Pass 1 timing and Pass 2 data data-correctness path - §2.3 dispatch model: add motivation sentence for the descriptor-size linear form - §2.4 Accuracy: reorder GEMM → All-reduce → Probe → Simplifications; drop FSIM aside (already covered by §4 Fig 5 caption); soften 'every ns' → 'every modeled latency contribution' - §2.5 HW config: add 64 TFLOP/s vs 2048 GB/s (~31 FLOP/byte) balance-point intuition and forward pointer to §5 GQA decode - Fig 1 caption: separate illustrative topology from experimental configuration §1: tighten GQA-as-primary-bandwidth-bottleneck framing. Figures: regenerate SIP / CUBE architecture (SVG sources + PDF + generator scripts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
ywkang
|
359a0eaa44 |
paper(platform): Table 1 -> table*, shrink Fig 1 row 1, move Fig 1/2 source up
- Probe table (Table 1, tab:probe-pe-dma) promoted to table* so it
spans both columns and no longer gets clipped at the column edge.
- Fig 1 (hw-arch) row 1 (SIP, CUBE subfigures) trimmed: widths
0.495 -> 0.42 textwidth and a height=0.6\linewidth cap added so
the figure occupies less vertical space and floats can land
earlier in the document.
- Fig 1 and Fig 2 source blocks relocated to the top of §2 so they
enter LaTeX's float queue before any §2 body text. As a result
Fig 1 now lands on page 3 and Fig 2 on page 4 (was 4 and 5).
- Table 2 (tab:hw, modeled hardware config) reverted to plain
\begin{table}; the previous \begin{table*} reshape was not
requested.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
ywkang
|
1cf8dd7868 |
paper(platform): widen SIP/CUBE subfigures, drop height cap
- subfig widths 0.46 -> 0.495 textwidth so the \hfill gap between (a) SIP and (b) CUBE shrinks to ~2% textwidth instead of ~8%. - Removed the height=0.85\linewidth cap on both subfigures; with the cap, the (roughly 1:1) SIP and CUBE diagrams were being scaled down to 85% even when the column had room. Let them render at their natural aspect ratio. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
ywkang
|
b715002b5a |
paper: §2 figure layout + Accuracy subsection + Table 1 → 2-col; §6 trim historical 4-panel content
§2 platform:
- Accuracy promoted from \paragraph to \subsection (sec:accuracy);
it sits at the same heading level as Why / Device / Latency / HW
config and so reads as a first-class component of the platform
description rather than a tail-end footnote.
- Three architecture diagrams (SIP, CUBE, PE) collapsed into one
figure* using subcaption: Fig.~\ref{fig:hw-arch}(a) SIP and (b)
CUBE side-by-side on row 1, (c) PE wide on row 2, total height
capped at ~40% of the page. Subfigure cross-references rewritten
to Fig.~ref{fig:hw-arch}\subref{...} in the body text. main.tex
now pulls in the subcaption package.
- Table 1 (modeled hardware configuration) promoted to table*
(two-column / full page width) and the row layout rewritten as a
4-column tabular so Hierarchy + PE + Command-issue sit on the
left and Memory + Interconnect sit on the right. The previous
single-column rendering was getting cut at the right edge of the
printed column.
§6 GQA:
- Removed the historical four-panel headline table (tab:gqa), the
two figures (fig:gqa-lat, fig:gqa-break), and the prose paragraph
that cited their per-panel numbers. Reason: the underlying
milestone-gqa-headline bench has been simplified in collaborator
commit
|
||
|
ywkang
|
e3f08972da |
paper: §2 polish + §5 fresh all-reduce data + kernbench probe table
§2 platform:
- Add a "memory-centric AHBM" sentence to the device-and-execution
intro: each PE is paired with dedicated HBM bandwidth and on-PE
TCM, so the performance question is about feeding compute from
locally-attached memory + moving the unavoidable inter-PE traffic
efficiently. Makes the AHBM character of the platform visible
before §2.3 starts unpacking the latency model.
- Promote "Congestion and contention modeling" from \paragraph to
\subsubsection: this is the platform's core differentiator over a
peak-BW roofline, so it deserves its own heading.
- Rename "Control-plane (issue) cost model" to "Command dispatch
overhead model" -- describes what it actually charges (the PE_CPU
paying a fixed + per-byte cost to push a command to one of the
accelerator engines) without the more abstract "control-plane"
framing.
- Add a third Accuracy cross-check from kernbench probe: a
PE→HBM DMA-distance sweep that confirms monotonic hop progression,
bandwidth saturation matching the per-edge model, and the built-in
invariants (D2H >= H2D, cross-CUBE best < worst). New
Table~\ref{tab:probe-pe-dma} reports per-traversal latency and
utilisation at 32KiB / 1MiB across five hop classes.
- Captured probe output as figures/probe_pe_dma_summary.txt for
reproducibility.
§5 PE_IPCQ / all-reduce:
- Re-ran milestone-1h-ccl on current sim_engine (post-ADR-0064 Rev2
and the IPCQ slot-wrap Phase-2 race fix). Updated the topology-
comparison table and the buffer-kind caption to the fresh raw
latencies. Headline ratios are preserved:
torus vs mesh saving at 96 KB/PE: 24.8% (was 24.8%) -> ~25%
torus vs ring saving at 96 KB/PE: 19.3% (was 19.3%) -> ~19%
TCM vs HBM saving at 64 KB/PE: 13.4% (was 13.9%) -> ~13%
TCM vs SRAM saving at 64 KB/PE: 37.2% (was 38.3%) -> ~37%
The Executive Summary's "up to ~14%" / "up to ~38%" framing stays
consistent with these post values.
Artifacts refreshed in src/kernbench/benches/1H_milestone_output/:
- ccl/summary.csv + per-topology PNGs + buffer-kind CSV/PNG
- ccl/comparison_mesh_vs_ring_vs_2DTorus_vs_theoretical_vs_fsim.png
- gemm/* (re-run yields identical pe_window structure; PNGs
refreshed)
Paper figures synced to the fresh artifacts:
- figures/allreduce_comparison.png
- figures/allreduce_buffer_kind.png
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
ywkang
|
7f437a20bd |
paper(platform): edit-pass through §2 KernBench Platform done
- Reordered §2 subsections to follow the SIP → CUBE → PE → graph
flow: Why KernBench → Device and execution model → Latency model
→ Modeled hardware configuration. Readers now meet the device
hierarchy before the graph abstraction that re-uses it.
- §2.2 Device and execution model: starts with the SIP/CUBE/PE
hierarchy paragraphs (each anchoring fig:sip-arch, fig:cube-arch,
fig:pe-arch); then the runtime-API/sim-engine/components bullet
list; then the atomic-vs-composite command distinction (corrects
the prior over-narrow framing that read every PE command as
composite -- atomic single-engine commands exist too, and PE_CPU
itself runs control-plane work directly).
- §2.3 Latency model: opens with the four-contribution decomposition
(per-node overhead, per-edge transmission, drain, queuing delay)
and the latency_model schematic; retains existing The hardware as
a graph / From graph to DES / Latency contributions / Congestion
/ Control-plane cost model / Accuracy paragraphs. Accuracy
paragraph now closes on KernBench's sufficiency for *relative*
HW/SW design trade-offs given analytic + external-simulator
agreement.
- New figures and assets:
- figures/sip_architecture.pdf (SIP-level graph view)
- figures/cube_architecture.pdf (CUBE-level zoom-in)
- figures/latency_model.png (conceptual latency-model
schematic with per-node /
per-edge / drain / queuing-delay
colour coding)
- figures/pe_architecture.png (carried over)
- Source-of-truth generator for the latency schematic:
scripts/paper/paper_latency_model_diagram.py (a report-only
harness under scripts/paper/ per the /paper isolation rule).
- main.tex preamble: \usepackage{tikz} added (kept from prior
sequence-diagram draft -- harmless now that the latency model is
a PNG; left in to keep paragraph numbering stable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
ywkang
|
9270d3435a |
paper(platform): WIP — §2 KernBench Platform mid-edit
Mid-pass through §2. Captured progress so far:
- §2 intro previews the three new threads (graph view, DES engine,
congestion) the rewritten latency section weaves together.
- §2.3 retitled to "Latency model: a graph traversed by events" and
restructured into bold-led paragraphs:
- The hardware as a graph (nodes = components, edges = links).
- From graph to discrete-event simulation (node/edge events,
deterministic ordering, correlation-ID trace).
- Latency contributions (per-node fixed, per-edge size-aware,
per-service occupancy).
- Congestion: per-edge FIFO BW occupancy, HBM per-PC parallelism,
component serial workers — the mechanisms that surface real
bottlenecks instead of peak-BW roofline.
- Control-plane cost model (FIXED + b·R) — unchanged.
- Accuracy: extended with a second cross-check from the all-reduce
study (torus vs. analytic startup-plus-per-packet model + FSIM
external single-device reference). Existing GEMM analytic-vs-
measured 10-20% check retained.
- §2.4 Hardware-configuration table unchanged.
Still TODO: cross-check accuracy numbers against a fresh
milestone-1h-gemm/ccl re-run; the current artifacts predate the
ADR-0064 Rev2 cost-model and IPCQ Phase-2 race fix and may need a
refresh before §2.3 final lock-in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
ywkang
|
dd525bfcb7 |
paper: add /paper skill + 1H HW-SW codesign report (GEMM, All-Reduce, fused GQA)
New `/paper` slash-command skill that synthesizes ADR/SPEC content and live KernBench benchmark results into a sectioned LaTeX technical paper compiled to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds every number in committed artifacts or fresh bench runs, and keeps report-only benches isolated. This commit also includes the first generated report: - docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures, toc.md contract, and the built 8-page main.pdf. Covers the platform (source-level kernels, latency model + accuracy, HW config from topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and fused GQA combining both, plus discussion/conclusion/2H future work. - scripts/paper/ — isolated report harnesses (not registered benches): paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine occupancy (the milestone only emitted op-counts); paper_plot_gqa.py renders the GQA figures. GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this config, and the proposed two-composite softmax_merge decode is marked designed-not-measured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |