paper: add /paper skill + 1H HW-SW codesign report (GEMM, All-Reduce, fused GQA)
New `/paper` slash-command skill that synthesizes ADR/SPEC content and live KernBench benchmark results into a sectioned LaTeX technical paper compiled to PDF with Tectonic (auto-installed). The skill negotiates a TOC, grounds every number in committed artifacts or fresh bench runs, and keeps report-only benches isolated. This commit also includes the first generated report: - docs/report/1H-codesign-paper/ — main.tex + per-section .tex, figures, toc.md contract, and the built 8-page main.pdf. Covers the platform (source-level kernels, latency model + accuracy, HW config from topology.yaml), GEMM via composite command, All-Reduce via PE_IPCQ, and fused GQA combining both, plus discussion/conclusion/2H future work. - scripts/paper/ — isolated report harnesses (not registered benches): paper_gqa_latency.py harvests per-panel GQA end-to-end latency + engine occupancy (the milestone only emitted op-counts); paper_plot_gqa.py renders the GQA figures. GEMM/All-Reduce reuse committed milestone figures/CSVs; GQA results are generated fresh. Honest flags retained: PE_CPU dispatch cost is 0 in this config, and the proposed two-composite softmax_merge decode is marked designed-not-measured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,40 @@
|
||||
\section{Future Work --- 2H}
|
||||
\label{sec:future}
|
||||
|
||||
The 1H work covered attention end to end. The natural next step is to
|
||||
complete the decoder and then to ask how compute and data should be
|
||||
distributed for realistic, agentic workloads.
|
||||
|
||||
\paragraph{From attention to the full decoder: FFN and MoE.} A decoder
|
||||
block is attention followed by a feed-forward network (FFN), and in modern
|
||||
models that FFN is increasingly a mixture-of-experts (MoE) layer. The FFN
|
||||
is the compute-rich counterpart to attention---it is where the composite
|
||||
command's MAC-efficiency gains (\S\ref{sec:gemm}) should matter most---so
|
||||
adding it gives a balanced view of a full block instead of its
|
||||
memory-bound half alone. MoE adds a new dimension: a routing step selects a
|
||||
few experts per token, turning the dense FFN GEMM into a sparse, data-
|
||||
dependent dispatch. The open questions are how to issue expert GEMMs as
|
||||
composites under data-dependent token counts, and how to move tokens to
|
||||
experts efficiently---an all-to-all-shaped communication pattern distinct
|
||||
from the all-reduce studied here, and a natural extension of the PE\_IPCQ
|
||||
work.
|
||||
|
||||
\paragraph{Compute and data distribution for full LLM decoding.} With both
|
||||
attention and FFN/MoE in hand, the question becomes where each layer's
|
||||
compute and state should live. Attention is KV-bound and favors keeping the
|
||||
KV cache close to the PEs that consume it; FFN/MoE is compute-bound and
|
||||
favors spreading GEMM work across PEs; MoE routing makes the optimal
|
||||
placement token- and time-dependent. A 2H goal is to use KernBench to
|
||||
explore these placement and parallelization trade-offs---tensor vs.\ expert
|
||||
vs.\ sequence parallelism---under a single, software-stack-independent
|
||||
model, so the interconnect and memory implications of each choice are
|
||||
measured rather than assumed.
|
||||
|
||||
\paragraph{Agentic workloads.} Agentic inference interleaves many
|
||||
short, bursty decode requests with tool use and long shared contexts,
|
||||
which stresses the system differently from a single long generation:
|
||||
context reuse across requests, dynamic batching, and uneven expert load all
|
||||
change how compute and data should be dispersed. Characterizing how total
|
||||
compute and data movement distribute across the SIP/CUBE/PE hierarchy under
|
||||
such workloads---and which of the 1H hardware levers still dominate when the
|
||||
workload is this irregular---is the broader 2H agenda.
|
||||
Reference in New Issue
Block a user