Implement ADR-0020: 2-pass data execution with greenlet kernel runner

Step 1 — Foundation:
- OpRecord/OpLogger: op log infrastructure with t_start stable ordering
- MemoryStore: numpy ndarray tensor-granular storage (reference semantics)
- data_op=True flag on DmaReadCmd, DmaWriteCmd, GemmCmd, MathCmd, CompositeCmd
- numpy/greenlet dependencies added to pyproject.toml

Step 2 — ComponentBase hooks:
- _on_process_start/end hooks in _forward_txn (fabric messages)
- _handle_with_hooks in PeEngineBase (PE-internal commands)
- op_logger optional — zero overhead when disabled

Step 3 — KernelRunner + greenlet:
- KernelRunner: greenlet ↔ SimPy bridge in triton_emu/kernel_runner.py
- TLContext: _emit() method routes to greenlet switch or command list
- tl.load() returns real numpy data in greenlet mode
- Dynamic control flow supported (memory-read based branching)

Step 4 — PE_CPU integration:
- Greenlet mode when ctx.memory_store is set, legacy fallback otherwise
- Refactored into _execute_greenlet/_execute_legacy/_send_response
- ComponentContext gains memory_store and op_logger fields

Step 5 — DataExecutor:
- Phase 2 numpy execution for GEMM/Math ops from op_log
- _compute_math: all unary/binary/reduction ops
- verify(): compare MemoryStore against expected with dtype tolerance

28 new tests, 366 total passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-08 00:22:44 -07:00
parent 140b85436a
commit 51004c311c
14 changed files with 1181 additions and 59 deletions
+20 -1
View File
@@ -33,6 +33,7 @@ class ComponentBase(ABC):
self.ctx = ctx
self.in_ports: dict[str, simpy.Store] = {}
self.out_ports: dict[str, simpy.Store] = {}
self._op_logger: Any | None = None # OpLogger, set by GraphEngine if enabled
def start(self, env: simpy.Environment) -> None:
"""Called once after all ports are wired.
@@ -64,9 +65,21 @@ class ComponentBase(ABC):
txn: Any = yield self._inbox.get()
env.process(self._forward_txn(env, txn))
def _on_process_start(self, env: simpy.Environment, msg: Any) -> None:
"""Op log hook: record service start for data_op messages (ADR-0020 D2)."""
if self._op_logger and getattr(msg, "data_op", False):
self._op_logger.record_start(env.now, self.node.id, msg)
def _on_process_end(self, env: simpy.Environment, msg: Any) -> None:
"""Op log hook: record service end for data_op messages (ADR-0020 D2)."""
if self._op_logger and getattr(msg, "data_op", False):
self._op_logger.record_end(env.now, self.node.id, msg)
def _forward_txn(self, env: simpy.Environment, txn: Any) -> Generator:
"""Apply run() latency, then forward to next hop or drain at terminal."""
self._on_process_start(env, txn)
yield from self.run(env, txn.nbytes)
self._on_process_end(env, txn)
next_hop = txn.next_hop # duck-typed: Transaction.next_hop
if next_hop:
yield self.out_ports[next_hop].put(txn.advance())
@@ -120,10 +133,16 @@ class PeEngineBase(ComponentBase):
while True:
msg: Any = yield self._inbox.get()
if isinstance(msg, PeInternalTxn):
env.process(self.handle_command(env, msg))
env.process(self._handle_with_hooks(env, msg))
else:
env.process(self._forward_txn(env, msg))
def _handle_with_hooks(self, env: simpy.Environment, pe_txn: Any) -> Generator:
"""Wrap handle_command with op log hooks on the inner command."""
self._on_process_start(env, pe_txn.command)
yield from self.handle_command(env, pe_txn)
self._on_process_end(env, pe_txn.command)
@abstractmethod
def handle_command(self, env: simpy.Environment, pe_txn: Any) -> Generator:
"""Process a PE-internal command (PeInternalTxn).