\section{Future Work --- 2H}
\label{sec:future}

The 1H work covered attention end to end. The natural next step is to
complete the decoder and then to ask how compute and data should be
distributed for realistic, agentic workloads.

\paragraph{From attention to the full decoder: FFN and MoE.} A decoder
block is attention followed by a feed-forward network (FFN), and in modern
models that FFN is increasingly a mixture-of-experts (MoE) layer. The FFN
is the compute-rich counterpart to attention---it is where the composite
command's MAC-efficiency gains (\S\ref{sec:gemm}) should matter most---so
adding it gives a balanced view of a full block instead of its
memory-bound half alone. MoE adds a new dimension: a routing step selects a
few experts per token, turning the dense FFN GEMM into a sparse, data-
dependent dispatch. The open questions are how to issue expert GEMMs as
composites under data-dependent token counts, and how to move tokens to
experts efficiently---an all-to-all-shaped communication pattern distinct
from the all-reduce studied here, and a natural extension of the PE\_IPCQ
work.

\paragraph{Compute and data distribution for full LLM decoding.} With both
attention and FFN/MoE in hand, the question becomes where each layer's
compute and state should live. Attention is KV-bound and favors keeping the
KV cache close to the PEs that consume it; FFN/MoE is compute-bound and
favors spreading GEMM work across PEs; MoE routing makes the optimal
placement token- and time-dependent. A 2H goal is to use KernBench to
explore these placement and parallelization trade-offs---tensor vs.\ expert
vs.\ sequence parallelism---under a single, software-stack-independent
model, so the interconnect and memory implications of each choice are
measured rather than assumed.

\paragraph{Agentic workloads.} Agentic inference interleaves many
short, bursty decode requests with tool use and long shared contexts,
which stresses the system differently from a single long generation:
context reuse across requests, dynamic batching, and uneven expert load all
change how compute and data should be dispersed. Characterizing how total
compute and data movement distribute across the SIP/CUBE/PE hierarchy under
such workloads---and which of the 1H hardware levers still dominate when the
workload is this irregular---is the broader 2H agenda.