paper(intro): edit-pass through §1 Introduction done

- §1 Introduction rewritten with AHBM-first opening, in-paper KernBench
  platform overview, GQA motivation and three architectural requirements,
  three matching HW-SW co-design mechanisms (GQA-aware placement,
  PE_IPCQ, composite-command GEMM under PE_SCHEDULER), broader-
  applicability outlook (PE_IPCQ as collective substrate, composite
  command for fused operator pipelines), and outline. Consistent with
  the Executive Summary structure.
- Executive Summary: extended broader-applicability closing with a
  forward-looking line previewing FFN/MoE integration.
- §5 section heading renamed from
  "All-Reduce Acceleration via PE_IPCQ" to
  "PE_IPCQ and Collective Communication".
- Title page date line refined to "2026 H1 Report".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/report/1H-codesign-paper/main.tex

@@ -25,7 +25,7 @@
 \date{
 \small
 AGI Computing Lab, System Technology Group\\
-H1 2026
+2026 H1 Report
 }
 
 \begin{document}

docs/report/1H-codesign-paper/sections/00-exec-summary.tex

@@ -30,4 +30,7 @@ composite-command execution enables efficient fusion of memory
 movement, GEMM kernels, normalization, and other element-wise
 operations. Together with the hierarchical data-placement framework,
 these capabilities form a reusable foundation for future AI kernels
-and communication libraries on AHBM.
+and communication libraries on AHBM. Looking ahead, these same mechanisms provide
+the basis for optimizing FFN-intensive workloads such as Mixture-of-Experts (MoE)
+layers and for developing integrated optimization strategies spanning both attention
+and MoE components within next-generation AI models.

docs/report/1H-codesign-paper/sections/01-introduction.tex

@@ -1,49 +1,83 @@
 \section{Introduction}
 \label{sec:intro}
 
-The performance of a large language model on an accelerator is decided less
+AHBM integrates compute units directly into the HBM stack. Each
-by peak FLOPS than by how well the kernel exploits the memory system and
+processing element (PE) is paired with a dedicated slice of HBM and uses
-the interconnect. This is especially true of attention, which in the
+local TCM and SRAM to stage data between memory and the MAC array. On
-decode phase reads a large KV cache to do a small amount of arithmetic.
+this memory-centric architecture, kernel performance depends not only on
-Optimizing such kernels is fundamentally a \emph{codesign} problem: the
+compute throughput but also on how effectively data is placed, moved,
-algorithm (how to tile, fuse, and reduce) and the hardware (what issue
+and shared across the memory hierarchy and between PEs. Optimizing AI
-mechanism, what collective engine, what memory hierarchy) have to be
+kernels on AHBM therefore requires hardware--software co-design, in
-designed against each other. Optimizing only the software leaves the
+which kernel algorithms and architectural mechanisms are developed
-hardware idle; adding hardware that the software cannot reach is wasted
+together.
-area.
 
-\paragraph{The 1H focus: attention.} This half's work targets attention.
+To enable detailed performance analysis and rapid design exploration, we
-Multi-head attention (MHA) was studied previously and is taken here as the
+developed \textbf{KernBench}, a source-level discrete-event simulation
-established baseline. The new work is FlashAttention-style tiling and
+platform for AHBM. KernBench implements the AHBM execution model,
-Grouped-Query Attention (GQA)---the form used by modern long-context
+including memory-system latencies, the PE execution model, inter-PE
-decoders, in which several query heads share a single KV head to shrink the
+communication, and host-side orchestration, while executing both kernel
-KV cache. Realizing a fast \emph{fused} GQA kernel, however, is not a
+and host software directly from source code. This provides fine-grained
-single optimization. It decomposes into two enabling optimizations that we
+visibility into execution behavior. It enables systematic evaluation of hardware--software co-design choices
-studied separately and then brought together inside the fused kernel:
+independently of higher-level software stacks such as compilers and
+runtimes. All
+results presented in this report are obtained using KernBench, which
+serves as the common evaluation platform for all mechanisms and kernels
+discussed in this study.
 
-\begin{enumerate}
+This report focuses on Grouped-Query Attention (GQA), one of the most
-  \item \textbf{GEMM optimization} via a composite command
+performance- and bandwidth-critical components of LLM inference. Modern
-  (\S\ref{sec:gemm}): a way to issue a tiled matrix multiply as one
+decoder-only models such as Llama~3 and Mistral have largely transitioned
-  self-routing pipeline so the MAC array stays fed without per-tile command
+from GPT-3-style multi-head attention (MHA) to GQA, in which multiple
-  overhead. Attention's $Q\!\cdot\!K^{\top}$ and $P\!\cdot\!V$ products are
+query heads share a single KV head to reduce KV cache capacity and
-  exactly such GEMMs.
+memory-bandwidth requirements. While GQA improves system efficiency at
-  \item \textbf{Communication optimization} via PE\_IPCQ
+the model level, mapping it efficiently onto AHBM introduces three
-  (\S\ref{sec:allreduce}): a per-PE on-device collective engine that
+architectural requirements: optimized placement of KV caches and weights
-  performs all-reduce overlapped with compute. Attention's multi-user and
+to minimize inter-PE communication, low-overhead support for unavoidable
-  sequence-parallel KV reductions are exactly such collectives.
+cross-PE traffic, and efficient pipelining of memory accesses and
-\end{enumerate}
+computation within each PE.
 
-The report therefore reads as \emph{two enablers building to one capstone}:
+To address these requirements, this report introduces three
-GEMM (\S\ref{sec:gemm}) and all-reduce (\S\ref{sec:allreduce}) are
+hardware--software co-design mechanisms. First, GQA-aware data placement
-developed and measured on their own, and then fused GQA
+distributes KV caches and weights across the TCM/SRAM/HBM hierarchy to
-(\S\ref{sec:gqa}) shows them operating together. Before the results,
+reduce communication overhead and improve data locality. Second,
-\S\ref{sec:platform} describes the KernBench platform---why a
+PE\_IPCQ provides an efficient on-device collective communication
-software-stack-independent, source-level simulator is the right tool for
+primitive for reductions and other communication-intensive operations.
-this question, how it executes a kernel, how it computes latency and how
+Third, a composite-command GEMM pipeline tightly pipelines memory
-accurate that is, and the exact hardware configuration used throughout.
+movement and computation within each PE under PE\_SCHEDULER control,
-We close with a cross-cutting discussion of which hardware changes are
+reducing command overhead while keeping the MAC array efficiently
-worth their cost (\S\ref{sec:discussion}), a conclusion
+utilized.
-(\S\ref{sec:conclusion}), and the 2H agenda
+
-(\S\ref{sec:future}): extending from attention to the feed-forward and
+The compute enabler (the composite-command GEMM pipeline,
-mixture-of-experts layers, and to the compute/data distribution questions
+\S\ref{sec:gemm}) and the communication enabler (PE\_IPCQ,
-that full LLM decoding and agentic workloads raise.
+\S\ref{sec:allreduce}) are first developed and evaluated independently.
+The fused GQA kernel (\S\ref{sec:gqa}) then combines them with
+GQA-aware data placement to demonstrate an end-to-end attention
+implementation on AHBM. The correspondence is direct: attention's
+$QK^{\top}$ and $PV$ products are precisely the GEMMs that benefit from
+the composite-command pipeline, while its KV reductions are precisely
+the collective operations that benefit from PE\_IPCQ. Together, these
+mechanisms enable the fused GQA kernel to efficiently exploit AHBM's HBM
+bandwidth.
+
+Although GQA serves as the motivating workload for this study, the
+resulting mechanisms are intended as reusable building blocks for a much
+broader class of AI kernels. PE\_IPCQ can support collective
+communication across distributed and communication-intensive workloads,
+while composite-command execution can be applied to GEMM-based kernels,
+feed-forward networks (FFNs), normalization, and other fused operator
+pipelines. Together with the hierarchical data-placement framework,
+these mechanisms form a foundation for future AI kernels and
+communication libraries on AHBM. In the second half of 2026, this
+foundation will be extended to FFN- and MoE-dominated workloads and to
+end-to-end optimization of complete LLM execution.
+
+The remainder of this report is organized as follows.
+Section~\ref{sec:platform} describes the KernBench platform and the AHBM
+configuration used throughout this study. Sections~\ref{sec:gemm},
+\ref{sec:allreduce}, and \ref{sec:gqa} present the composite-command
+GEMM pipeline, PE\_IPCQ collective communication, and the fused GQA
+kernel, respectively. Section~\ref{sec:discussion} discusses the broader
+architectural implications of these results. Finally,
+Sections~\ref{sec:conclusion} and \ref{sec:future} summarize the key
+findings and outline future work on FFN, MoE, and full-model
+optimization.

docs/report/1H-codesign-paper/sections/04-allreduce.tex

@@ -1,4 +1,4 @@
-\section{All-Reduce Acceleration via PE\_IPCQ}
+\section{PE\_IPCQ and Collective Communication}
 \label{sec:allreduce}
 
 \subsection{Why it is needed}