Kernel-launch sync (ADR-0009 D5) and IPCQ drain at inbound (ADR-0023)
- KernelLaunchMsg gains target_start_ns: IO_CPU stamps a global barrier (max path latency across every target PE), M_CPU passes it through, PE_CPU yields until it before recording pe_exec_start. Every PE in a launch begins kernel execution at the same env.now regardless of its dispatch path length — eliminates per-PE dispatch-offset artifact in cross-PE and cross-cube latency measurements. - PE_DMA._handle_ipcq_inbound now pays Transaction.drain_ns at the top, matching the terminal-drain behavior of ComponentBase._forward_txn for every non-IPCQ Transaction. SRC-side tl.send stays fire-and-forget (sender doesn't yield on sub_done); tl.recv now blocks until bytes have actually drained into its inbox. - ComponentContext: new compute_path_latency_ns helper + node_overhead_ns field populated by GraphEngine. - tests/test_kernel_launch_sync.py: asserts all PEs in one launch produce identical pe_exec_ns for a no-op kernel (zero spread). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -186,13 +186,37 @@ class PeDmaComponent(PeEngineBase):
|
||||
# ── IPCQ inbound (fabric → PE_DMA → MemoryStore + PE_IPCQ) ──────
|
||||
|
||||
def _handle_ipcq_inbound(self, env: simpy.Environment, txn: Any) -> Generator:
|
||||
"""At destination PE_DMA: atomically write data and forward metadata.
|
||||
"""At destination PE_DMA: pay terminal drain, then atomically write
|
||||
data and forward metadata.
|
||||
|
||||
ADR-0023 D9 (drain at inbound terminal): the Transaction carries
|
||||
``drain_ns = nbytes / bottleneck_bw_on_path`` stamped by the sender
|
||||
PE_DMA. Like every other Transaction terminal in the simulator (see
|
||||
``ComponentBase._forward_txn``), this drain must be paid when the
|
||||
Transaction reaches its destination. SRC-side ``tl.send`` is
|
||||
fire-and-forget — it never yields on ``sub_done`` — so paying the
|
||||
drain here does NOT delay the sender. What it DOES delay is the
|
||||
IpcqMetaArrival forwarded below: that delay is the only signal
|
||||
``tl.recv`` on DST blocks on, which is exactly the desired
|
||||
semantics — "send dispatches and returns; recv waits until the
|
||||
bytes have actually landed in its inbox".
|
||||
|
||||
The drain MUST be paid before the atomic block — inserting a yield
|
||||
inside would break invariant I6.
|
||||
|
||||
I6 (MUST): no SimPy yield between MemoryStore.write and the
|
||||
IpcqMetaArrival put into PE_IPCQ.
|
||||
"""
|
||||
from kernbench.common.ipcq_types import IpcqMetaArrival
|
||||
|
||||
# Pay terminal BW drain before the atomic write/metadata forward.
|
||||
# Without this, IPCQ effectively got fabric bandwidth for free at
|
||||
# the terminal (only intermediate-hop overhead_ns was charged),
|
||||
# making IPCQ lower than raw DMA at large sizes in benchmarks.
|
||||
drain = getattr(txn, "drain_ns", 0.0)
|
||||
if drain > 0:
|
||||
yield env.timeout(drain)
|
||||
|
||||
token = txn.request
|
||||
|
||||
# ── ATOMIC: do not introduce yield between these two operations ──
|
||||
|
||||
Reference in New Issue
Block a user