Implement ADR-0021: PE pipeline refactor with token self-routing

Step 1-2: Backup existing code
- builtin/ → builtin_legacy/ (unchanged backup)
- custom/pe_accel/ → custom/pe_accel_legacy/ (unchanged backup)

Step 3-4: New pipeline types and tiling
- pe_types.py: StageType, Stage, TilePlan, PipelinePlan, PipelineContext, TileToken
- tiling.py: generate_gemm_plan, generate_math_plan (ported from pe_accel)

Step 5: Component implementations (ADR-0021 D4-D6)
- PE_SCHEDULER: _feed_loop (singleton FIFO feeder) + plan generation
- PE_FETCH_STORE: new component — TCM ↔ Register File
- PE_GEMM: TileToken pipeline + legacy PeInternalTxn dual-mode
- PE_MATH: TileToken pipeline + legacy dual-mode
- PE_DMA: TileToken pipeline + legacy + fabric Transaction triple-mode
- PE_TCM: TcmRequest handler with dual-channel BW serialization

Step 6: Infrastructure
- topology.yaml: pe_fetch_store component + chaining edges
- components.yaml: pe_fetch_store_v1 registration
- builder.py: PE_COMP_OFFSETS, _add_pe_internal_edges, PE view positions
- Tests: node/edge counts, PE component sets updated

All components handle both TileToken (pipeline) and PeInternalTxn (legacy).
Token self-routing: components read next stage from token.plan, chain via out_port.
366 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-08 23:35:31 -07:00
parent 161132cdcb
commit b6eb97c49a
40 changed files with 4055 additions and 214 deletions
+63 -6
View File
@@ -1,7 +1,18 @@
"""PE_TCM: tightly-coupled memory with BW-based access serialization (ADR-0021).
Models scratchpad memory inside the PE. Handles both legacy Transaction forwarding
and TcmRequest from PE_FETCH_STORE for BW-serialized read/write access.
Two channels (read/write) with independent serialization.
Ported from pe_accel TcmBlock timing model.
"""
from __future__ import annotations
from collections.abc import Generator
from typing import TYPE_CHECKING
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
import simpy
from kernbench.components.base import ComponentBase
@@ -10,16 +21,62 @@ if TYPE_CHECKING:
from kernbench.topology.types import Node
class PeTcmComponent(ComponentBase):
"""PE_TCM: tightly-coupled memory / local SRAM staging buffer.
@dataclass
class TcmRequest:
"""Request to read from or write to TCM (used by PE_FETCH_STORE)."""
Terminal storage component for PE-internal dataflow (ADR-0014 D5).
Phase 0: applies overhead_ns and drain_ns at terminal.
direction: str # "read" or "write"
nbytes: int
done: simpy.Event
tag: str = ""
class PeTcmComponent(ComponentBase):
"""PE_TCM: BW-serialized scratchpad memory (ADR-0021 D1).
Dual-channel: read and write can proceed in parallel,
but concurrent reads serialize, concurrent writes serialize.
BW from topology attrs or pe_template links.
"""
def __init__(self, node: Node, ctx: ComponentContext | None = None) -> None:
super().__init__(node, ctx)
self._read_bw: float = float(node.attrs.get("read_bw_gbs", 512.0))
self._write_bw: float = float(node.attrs.get("write_bw_gbs", 512.0))
self._read_res: simpy.Resource | None = None
self._write_res: simpy.Resource | None = None
def run(self, env, nbytes: int) -> Generator:
def start(self, env: simpy.Environment) -> None:
self._read_res = simpy.Resource(env, capacity=1)
self._write_res = simpy.Resource(env, capacity=1)
super().start(env)
def run(self, env: simpy.Environment, nbytes: int) -> Generator:
overhead_ns = float(self.node.attrs.get("overhead_ns", 0.0))
yield env.timeout(overhead_ns)
def _worker(self, env: simpy.Environment) -> Generator:
"""Dispatch TcmRequest (from fetch_store) and Transaction (fabric)."""
while True:
msg: Any = yield self._inbox.get()
if isinstance(msg, TcmRequest):
env.process(self._handle_tcm_request(env, msg))
else:
env.process(self._forward_txn(env, msg))
def _handle_tcm_request(self, env: simpy.Environment, req: TcmRequest) -> Generator:
"""BW-serialized access: acquire channel, apply delay, signal done."""
if req.direction == "write":
res = self._write_res
bw = self._write_bw
else:
res = self._read_res
bw = self._read_bw
assert res is not None
with res.request() as lock:
yield lock
if bw > 0 and req.nbytes > 0:
delay_ns = req.nbytes / bw
yield env.timeout(delay_ns)
req.done.succeed()