Both 1-kv (2 users x 64 PE) and 8-kv (16 users x 8 PE) fill all 128 PEs of
the SIP; the throughput gap is per-PE HBM efficiency. Splitting a head over
8 PEs leaves each a single tile, so pipeline-fill and the 8-PE softmax
reduce dominate -> 46% HBM util; one head per PE streams contiguously with
no reduce -> 76%. Throughput ratio (1.6x) tracks the utilization ratio
(1.7x): decode is bandwidth-bound, so throughput follows total HBM
efficiency, not PE count. 1-kv spends hardware on latency (8 PE/head for a
4.8x speedup = 60% strong-scaling efficiency).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beyond a mapping's SIP capacity (16/C) the throughput plateaus: extra
users run in waves at the saturated rate, they are not un-runnable. Draw
a dashed horizontal extension at the ceiling so 1-kv-per-cube (cap 2)
reads as saturating, not stopping, at B=2. Caption updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Concurrency fix (completes the cube_base change): the decode kernel's
output store o_base still used the global cube_id while every load used
the user-local cube_local. For a single user (cube_base=0) they coincide,
so it was masked; a second batched user (cube_base>0) overshot its o shard
into an unmapped VA, mis-decoded to a phantom sip0.cube0.pe8 and raised a
RoutingError. Switch o_base to cube_local (one line); single-user results
are byte-identical (decode smoke/correctness pass).
Batch harness (un-skipped): create all users' tensors first, then submit
all launches deferred and drain together, so deploys don't drive an
already-submitted launch and serialize the batch. Concurrent users on
disjoint CUBE groups now overlap to within 3-5% of the single-user
latency. Swept B in {1,2,capacity} at 8K (high-B dense runs are the
expensive ones; throughput is near-linear in B).
Measured result: aggregate throughput at a full SIP rises from 0.86
(1-kv-per-cube, 2 users) to 1.41 requests/us (8-kv-per-cube, 16 users) —
the dense mapping wins throughput, the spread mapping wins latency.
S5.2 batch paragraph updated projected -> measured; add batch_scaling
figure and plot_batch_scaling.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Metric fixes (test harnesses, deploy artifact + wrong constant):
- wall = max(t_end) - min(t_start): exclude the one-time KV-cache deploy
from the measured decode/prefill step (it was 92-99% of wall, mapping-
invariant, and masked the real per-mapping separation).
- PEAK_PE_HBM_BPS 128 -> 256 B/ns (8 channels/PE x 32 GB/s); the old value
was half the modeled per-PE HBM BW, so hbm_bw_util read >1.0 once wall
was corrected. All six short-context sweep CSVs regenerated/repatched.
Result: the four KV mappings now separate along a {64,32,16,8}-active-PE
ladder (decode 8-kv/1-kv = 4.8x at 8K, 7.4x at 64K), not "modest/tied" as
before; decode is bandwidth-bound at 46-76% of the 256 GB/s per-PE ceiling.
Report (S5.2 rewrite):
- Replace the tied-wall / 8x-per-PE-util claims (both deploy artifacts)
with the corrected separation and a density trade-off (per-CUBE KV
footprint, Fig 16 top panel switched to a wall-invariant metric).
- Add a projected latency-vs-batched-throughput analysis (marked
projected, not measured): dense mappings win throughput, 1-kv wins
latency; converges at long context.
- Regenerate Fig 15/16/17/18; fix plot script hardcoded ROOT path.
Batch experiment (Part 2, cube_base):
- Add backward-compatible cube_base=0 scalar to the decode kernel so a
batched user placed at DPPolicy.cube_start addresses 0-based shards.
Default preserves single-user behavior (32 decode tests pass unchanged).
- New batch harness (skipped): concurrent B>=2 launches hit a sim routing
issue (sip0.cube0.pe8); single-user path verified. Concurrency fix next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Soften short-context claim: 1-kv-/8-kv-per-cube are tied at 8K and
separate as context grows (1-kv faster at the large end).
- Add a bridge noting the six-case placement taxonomy (§5.1) is the
long-context lens; short context turns on head-to-CUBE distribution.
- Remove caption-duplicating numbers from the long-context conclusion.
- Reduce the closing subsection to a GQA-scoped Summary; defer the
cross-cutting hardware-investment thesis to the Discussion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- §6 GQA: rewrite long-context decode from 4-case to 6-case. Data
Placement Policy now presents the six placement options and their
intrinsic per-PE memory / per-token comm costs (no winner predicted);
the long-context subsection selects the best placement for that regime
(Case 6 ★, both-axes S_kv shard) from the measured 6-case sweep. Add
KV-sharding diagram + analytical budget/summary figures.
- Regenerate the 6-row decode sweep (milestone-1h-gqa) and the
sweep-dependent decode panels (latency/traffic/parallelism/memory) so
figures, prose, and sweep_decode.json are mutually consistent.
- §5 All-Reduce: add IPCQ design alternatives (architecture + decision
matrix) as design-rationale schematics (illustrative step-counts, not
measured) and the bench-generated topology diagram.
- §4 GEMM: rename "user-orchestrated/user-level" -> "kernel-orchestrated/
kernel-level" (orchestration runs in the kernel program vs the
scheduler-orchestrated composite); minor accuracy fixes (7.18 TFLOP/s
~10% below peak; ~781 ns DMA).
- Recompile build/main.pdf.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two strands bundled as the 1H-codesign-paper refresh unit:
(A) This session — single-op cost-model reflection (depends on 2d8271c):
- §2 Table 2 (tab:hw): split "FIXED per command" into "FIXED per
single-op command" (8 cycles) and "FIXED per composite command"
(40 cycles); §2 dispatch-overhead prose updated to the two-class split.
- §3.4 (sec:gemm-vs-async): rename paragraph headers + prose to
async-full / async-tiled; "atomic" -> "single-op" throughout; reframe
mechanism #3 from the old DMA-only fast-path to the single-op
fast-path. Headline narrative now: even with EVERY single-op cmd
(96 DMA + 48 dot + 47 add) charged the light 8-cycle FIXED, composite
still wins ~2.8x at K=3072 purely on command-count structure (1 vs
192 commands) -- down from the pre-D8 ~6.3x, and explicitly NOT a
modelling artifact. Numbers refreshed from the regenerated sweep:
async-full 3.83->3.91, async-tiled 1.14->~2.53, under-tile corner
1.06->1.21, depth-2 vs depth-inf spread <1%. New figure wired in.
- build/main.pdf rebuilt (tectonic); pdftotext-verified (no broken
refs; Table 2 split, single-op terms, 2.8x/2.53/192-host-commands
all present).
(B) Prior-session paper work riding along uncommitted: §4 all-reduce
deep-edit, §5 GQA, §6 discussion trims; milestone_1h_ccl.py plot
label "FSIM" -> "H2 2025 SW queue baseline"; regenerated diagrams
under docs/diagrams/** and gemm output PNGs under
1H_milestone_output/gemm/. (Composite-window gemm plots are
unaffected by D8 — D8 only changes single-op dispatch FIXED, which
the composite window excludes.)
All TODO items for the D8 single-op extension are now complete and
pushed across 3 commits (2d8271c cost-model+ADR+tests, 821bbf2 bench
harness, this paper refresh). Full regression green (826 passed, 1
skipped). No remaining work.
NOTE for review (carried from 2d8271c): ADR-0065's "2x CPU-offload win"
headline for GQA decode opt2 may want a refresh to the post-D8 ~1.87x.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The five-stage chain was set as a centered display equation, which
overflowed the column and clipped the last 'DMA_WRITE' label. The
inline prose right below it also repeated the same chain in a
shorter form. Consolidate into one inline parenthetical listing the
five stage names, and reuse 'flows through the chain' for the
self-routing description — no clipping, no repetition.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The matplotlib title and gray subtitle were duplicating what the
figure caption already says in the LaTeX source, and crowding the
top of every chart. Drop both — chart now leaves the title/subtitle
slots blank and the caption is the single source of truth for
chart titling.
Re-render all 5 GEMM figures (mac util measured, mac util theoretical
vs measured, stage breakdown, HBM BW util, per-PE TFLOPS) and resync
into docs/report/1H-codesign-paper/figures/. No data changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new evaluation plots for §3 GEMM, both using the composite window
as the denominator (the up-front tl.load of A in load_ref is therefore
excluded — only HBM traffic and compute inside the composite count):
- gemm_hbm_bw_util.png: per-PE HBM bandwidth utilization against the
256 GB/s per-PE ceiling. Answers "is this workload memory-bound on
this configuration?" — small/single-tile shapes sit at 22–46%
(pipeline-fill-limited), large or output-write-dominated shapes
saturate (≥85%).
- gemm_per_pe_tflops.png: per-PE GEMM throughput against the 8 TFLOP/s
engine peak. The deep-K K=3072 load_ref case lands at
7.18 TFLOP/s (~90% of peak) — the configuration's clean win.
ref_ref at the same shape drops to 3.83 TFLOP/s because the second
operand doubles HBM pressure and the kernel hits the BW ceiling.
The variant gap on this chart is the operational cost of NOT
pre-staging the activation.
Both charts compare two operand-staging variants on every shape:
load_ref ("activation pre-staged", weight-only HBM streaming) and
ref_ref (both A and W streamed from HBM). load_load is omitted —
with both operands pre-staged the composite carries no HBM traffic
and the BW metric collapses to 0.
Analytic / measured peak unified to the single hardware spec
(8.0 TFLOP/s = 8000 flops/ns). T_STAGE is now derived as
tile_flops / peak (= 16.384 ns for the 32×64×32 tile) rather than
the previous hardcoded 16 ns, so MAC efficiency and per-PE TFLOPS%
share the same denominator and never disagree. Side effect: max
analytic-vs-measured gap tightens from 2.2 ppt to 1.4 ppt across
the sweep.
§3 Results restructured: lead with the two new evaluation plots
(BW saturation, achieved throughput), then the MAC-utilization
analytic-vs-measured chart serves as the validation step, then the
per-stage engine wall-clock as the supporting diagnostic. §3
Analysis updated to use the load_ref / ref_ref contrast as the
hardware-software boundary line that motivates the GQA kernel of §5.
§2.4 Accuracy claim updated to match: GEMM analytic-vs-measured
agreement is now "within 1.4 ppt across every swept shape", from
~7.7% at single-tile up to ~90% at K=3072. Subtitle/title vertical
stacking glitch in the matplotlib charts fixed in passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coupled fixes that recover small-tile GEMM pipeline efficiency
from 53% to 88% (32x3072x32 load_ref, composite_window basis).
1. PE_DMA channel-hold (ADR-0014 D4 clarified): both the
_handle_with_hooks (PeInternalTxn) and _pipeline_process
(TileToken) paths used to hold the cap=1 DMA channel through the
full HBM round-trip, which double-serialized with the HBM_CTRL's
own per-PC `available_at` model and prevented back-to-back tile
DMAs from amortizing their per-request head latency. Channel is
now released after the request is enqueued onto the next hop;
HBM serialization is HBM_CTRL's responsibility alone.
Tests: new test_pe_dma_back_to_back_pipelining as the oracle
(asserts wall < 75% of strict-serialized N x single_op). Existing
test_pe_dma_record_start_after_channel_acquire rewritten to assert
t_start clustering (channel released fast) instead of the old
round-trip-hold invariant. test_pe_dma_same_channel_serializes
still passes — HBM_CTRL preserves ordering.
Probe regression: PE→local-HBM 32 KiB stays at 141 ns
(single-request, unaffected).
2. milestone_1h_gemm bench: matmul_composite was reading MATMUL_M/K/N
env vars at module load, so every sweep row replayed the cached
256³ result; values now read inside run(). Drops the stale
sys.modules deletion hack.
3. Analytic ideal-pipeline model: dropped the (n_mn-1)·dma_w_per_pair
penalty (over-pessimistic for under-tile shapes — it pushed
measured > theoretical) and replaced the D_STAGES-derived head
with empirical T_PIPELINE_FILL=60 ns / T_PIPELINE_TAIL=30 ns.
Max analytic-vs-measured gap across all 7 swept shapes now 2.2 ppt
(was 9-44 ppt under the old constants).
Paper updates:
- §3 (GEMM): 78%→88% measured at 48 tiles, 23%→15% at 1 tile,
stage breakdown numbers refreshed (DMA in / Fetch / GEMM all
~785 ns at K=3072), analytic-vs-measured agreement tightened
to "within 2.2 ppt".
- §2.4 (Accuracy): GEMM tracking claim refreshed accordingly.
- §5 (GQA): restore long-ctx 4-cases figures into figures/
(they were dropped from bench output dir as derived artifacts in
92b9221 / e45626c but §5 still cites them by name).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Probe table (Table 1, tab:probe-pe-dma) promoted to table* so it
spans both columns and no longer gets clipped at the column edge.
- Fig 1 (hw-arch) row 1 (SIP, CUBE subfigures) trimmed: widths
0.495 -> 0.42 textwidth and a height=0.6\linewidth cap added so
the figure occupies less vertical space and floats can land
earlier in the document.
- Fig 1 and Fig 2 source blocks relocated to the top of §2 so they
enter LaTeX's float queue before any §2 body text. As a result
Fig 1 now lands on page 3 and Fig 2 on page 4 (was 4 and 5).
- Table 2 (tab:hw, modeled hardware config) reverted to plain
\begin{table}; the previous \begin{table*} reshape was not
requested.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- subfig widths 0.46 -> 0.495 textwidth so the \hfill gap between
(a) SIP and (b) CUBE shrinks to ~2% textwidth instead of ~8%.
- Removed the height=0.85\linewidth cap on both subfigures; with the
cap, the (roughly 1:1) SIP and CUBE diagrams were being scaled
down to 85% even when the column had room. Let them render at
their natural aspect ratio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
§2 platform:
- Accuracy promoted from \paragraph to \subsection (sec:accuracy);
it sits at the same heading level as Why / Device / Latency / HW
config and so reads as a first-class component of the platform
description rather than a tail-end footnote.
- Three architecture diagrams (SIP, CUBE, PE) collapsed into one
figure* using subcaption: Fig.~\ref{fig:hw-arch}(a) SIP and (b)
CUBE side-by-side on row 1, (c) PE wide on row 2, total height
capped at ~40% of the page. Subfigure cross-references rewritten
to Fig.~ref{fig:hw-arch}\subref{...} in the body text. main.tex
now pulls in the subcaption package.
- Table 1 (modeled hardware configuration) promoted to table*
(two-column / full page width) and the row layout rewritten as a
4-column tabular so Hierarchy + PE + Command-issue sit on the
left and Memory + Interconnect sit on the right. The previous
single-column rendering was getting cut at the right edge of the
printed column.
§6 GQA:
- Removed the historical four-panel headline table (tab:gqa), the
two figures (fig:gqa-lat, fig:gqa-break), and the prose paragraph
that cited their per-panel numbers. Reason: the underlying
milestone-gqa-headline bench has been simplified in collaborator
commit 65c365f ("drop misleading single_user_/multi_user_
panels") and no longer reproduces that data, so the section was
left referencing a dataset the current bench cannot regenerate.
- The previously-added 4-cases long-context decode comparison is
now §6's only Results subsection, retitled to
"Results: long-context decode and parallelism strategies".
The headline GQA result that survives into the paper is therefore
the parallelism trade-off study (Case 4 / Cube-SP × PE-SP at 34 us
buying an 8x KV-memory reduction over the latency leaders).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The collaborator commit 7c346de added the comparative figures
(gqa_decode_long_ctx_4cases_{latency,memory,traffic}.png) but the
fused-GQA section still only referenced the four headline panels.
This commit closes that loop:
- New §6 subsection "Long-context decode: parallelism strategies"
added between Results and Analysis. It lays out the four
parallelism strategies (Cube-SP/Repl x PE-TP/SP) and pulls in the
three new figures.
- The discussion highlights the central trade: the fastest strategy
(Case 3, Cube-Repl x PE-SP at 20.2 us) requires replicating the
full KV cache to every CUBE, while Case 4 (Cube-SP x PE-SP, the
chosen design marked *) gives back ~14 us in exchange for an 8x
KV-memory reduction. Case 4's ~190 IPCQ copies + ~190 DMA reads
are precisely the on-device collective traffic PE_IPCQ and the
torus links of §5 are provisioned to absorb -- a direct payoff
of the communication-side codesign work.
- Connects back to §5 (PE_IPCQ / all-reduce) so the reader sees the
capstone arc: the GEMM enabler exposes the data-movement bound,
the communication enabler attacks it, and the long-context
parallelism study shows how the choice between the two extremes is
framed by KV memory vs. on-device collective traffic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
§2 platform:
- Add a "memory-centric AHBM" sentence to the device-and-execution
intro: each PE is paired with dedicated HBM bandwidth and on-PE
TCM, so the performance question is about feeding compute from
locally-attached memory + moving the unavoidable inter-PE traffic
efficiently. Makes the AHBM character of the platform visible
before §2.3 starts unpacking the latency model.
- Promote "Congestion and contention modeling" from \paragraph to
\subsubsection: this is the platform's core differentiator over a
peak-BW roofline, so it deserves its own heading.
- Rename "Control-plane (issue) cost model" to "Command dispatch
overhead model" -- describes what it actually charges (the PE_CPU
paying a fixed + per-byte cost to push a command to one of the
accelerator engines) without the more abstract "control-plane"
framing.
- Add a third Accuracy cross-check from kernbench probe: a
PE→HBM DMA-distance sweep that confirms monotonic hop progression,
bandwidth saturation matching the per-edge model, and the built-in
invariants (D2H >= H2D, cross-CUBE best < worst). New
Table~\ref{tab:probe-pe-dma} reports per-traversal latency and
utilisation at 32KiB / 1MiB across five hop classes.
- Captured probe output as figures/probe_pe_dma_summary.txt for
reproducibility.
§5 PE_IPCQ / all-reduce:
- Re-ran milestone-1h-ccl on current sim_engine (post-ADR-0064 Rev2
and the IPCQ slot-wrap Phase-2 race fix). Updated the topology-
comparison table and the buffer-kind caption to the fresh raw
latencies. Headline ratios are preserved:
torus vs mesh saving at 96 KB/PE: 24.8% (was 24.8%) -> ~25%
torus vs ring saving at 96 KB/PE: 19.3% (was 19.3%) -> ~19%
TCM vs HBM saving at 64 KB/PE: 13.4% (was 13.9%) -> ~13%
TCM vs SRAM saving at 64 KB/PE: 37.2% (was 38.3%) -> ~37%
The Executive Summary's "up to ~14%" / "up to ~38%" framing stays
consistent with these post values.
Artifacts refreshed in src/kernbench/benches/1H_milestone_output/:
- ccl/summary.csv + per-topology PNGs + buffer-kind CSV/PNG
- ccl/comparison_mesh_vs_ring_vs_2DTorus_vs_theoretical_vs_fsim.png
- gemm/* (re-run yields identical pe_window structure; PNGs
refreshed)
Paper figures synced to the fresh artifacts:
- figures/allreduce_comparison.png
- figures/allreduce_buffer_kind.png
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Reordered §2 subsections to follow the SIP → CUBE → PE → graph
flow: Why KernBench → Device and execution model → Latency model
→ Modeled hardware configuration. Readers now meet the device
hierarchy before the graph abstraction that re-uses it.
- §2.2 Device and execution model: starts with the SIP/CUBE/PE
hierarchy paragraphs (each anchoring fig:sip-arch, fig:cube-arch,
fig:pe-arch); then the runtime-API/sim-engine/components bullet
list; then the atomic-vs-composite command distinction (corrects
the prior over-narrow framing that read every PE command as
composite -- atomic single-engine commands exist too, and PE_CPU
itself runs control-plane work directly).
- §2.3 Latency model: opens with the four-contribution decomposition
(per-node overhead, per-edge transmission, drain, queuing delay)
and the latency_model schematic; retains existing The hardware as
a graph / From graph to DES / Latency contributions / Congestion
/ Control-plane cost model / Accuracy paragraphs. Accuracy
paragraph now closes on KernBench's sufficiency for *relative*
HW/SW design trade-offs given analytic + external-simulator
agreement.
- New figures and assets:
- figures/sip_architecture.pdf (SIP-level graph view)
- figures/cube_architecture.pdf (CUBE-level zoom-in)
- figures/latency_model.png (conceptual latency-model
schematic with per-node /
per-edge / drain / queuing-delay
colour coding)
- figures/pe_architecture.png (carried over)
- Source-of-truth generator for the latency schematic:
scripts/paper/paper_latency_model_diagram.py (a report-only
harness under scripts/paper/ per the /paper isolation rule).
- main.tex preamble: \usepackage{tikz} added (kept from prior
sequence-diagram draft -- harmless now that the latency model is
a PNG; left in to keep paragraph numbering stable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mid-pass through §2. Captured progress so far:
- §2 intro previews the three new threads (graph view, DES engine,
congestion) the rewritten latency section weaves together.
- §2.3 retitled to "Latency model: a graph traversed by events" and
restructured into bold-led paragraphs:
- The hardware as a graph (nodes = components, edges = links).
- From graph to discrete-event simulation (node/edge events,
deterministic ordering, correlation-ID trace).
- Latency contributions (per-node fixed, per-edge size-aware,
per-service occupancy).
- Congestion: per-edge FIFO BW occupancy, HBM per-PC parallelism,
component serial workers — the mechanisms that surface real
bottlenecks instead of peak-BW roofline.
- Control-plane cost model (FIXED + b·R) — unchanged.
- Accuracy: extended with a second cross-check from the all-reduce
study (torus vs. analytic startup-plus-per-packet model + FSIM
external single-device reference). Existing GEMM analytic-vs-
measured 10-20% check retained.
- §2.4 Hardware-configuration table unchanged.
Still TODO: cross-check accuracy numbers against a fresh
milestone-1h-gemm/ccl re-run; the current artifacts predate the
ADR-0064 Rev2 cost-model and IPCQ Phase-2 race fix and may need a
refresh before §2.3 final lock-in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- §1 Introduction rewritten with AHBM-first opening, in-paper KernBench
platform overview, GQA motivation and three architectural requirements,
three matching HW-SW co-design mechanisms (GQA-aware placement,
PE_IPCQ, composite-command GEMM under PE_SCHEDULER), broader-
applicability outlook (PE_IPCQ as collective substrate, composite
command for fused operator pipelines), and outline. Consistent with
the Executive Summary structure.
- Executive Summary: extended broader-applicability closing with a
forward-looking line previewing FFN/MoE integration.
- §5 section heading renamed from
"All-Reduce Acceleration via PE_IPCQ" to
"PE_IPCQ and Collective Communication".
- Title page date line refined to "2026 H1 Report".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New `/paper` slash-command skill that synthesizes ADR/SPEC content and live
KernBench benchmark results into a sectioned LaTeX technical paper compiled
to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds
every number in committed artifacts or fresh bench runs, and keeps
report-only benches isolated.
This commit also includes the first generated report:
- docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures,
toc.md contract, and the built 8-page main.pdf. Covers the platform
(source-level kernels, latency model + accuracy, HW config from
topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and
fused GQA combining both, plus discussion/conclusion/2H future work.
- scripts/paper/ — isolated report harnesses (not registered benches):
paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine
occupancy (the milestone only emitted op-counts); paper_plot_gqa.py
renders the GQA figures.
GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are
generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this
config, and the proposed two-composite softmax_merge decode is marked
designed-not-measured.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mukesh
ywkang