# ADR-0009: Kernel Execution Messaging and Completion Semantics

## Status

Accepted

## Context

Kernel execution is initiated by the host and proceeds through
device control components:

Host → IO_CPU → M_CPU → PE_CPU → schedulers → engines

Completion propagates in reverse order.

To keep benchmarks simple and topology-agnostic,
kernel execution must be endpoint-driven with deterministic aggregation.

---

## Decision

### D1. Kernel launch is an endpoint request

A kernel launch is initiated by submitting a single KernelLaunch request
to the IO_CPU endpoint.

The runtime API MUST:

- construct the kernel launch request,
- submit it to IO_CPU,
- await a single completion result.

The runtime API MUST NOT orchestrate internal fan-out.

---

### D2. Tensor arguments are passed by metadata

KernelLaunch requests MUST reference tensor arguments via:

- host-owned tensor handles, or
- resolved device address maps derived from those handles.

Bulk tensor data MUST NOT be embedded in kernel launch messages.

---

### D3. Fan-out and aggregation are component responsibilities

- IO_CPU fans out work to M_CPUs.
- M_CPU fans out work to PE_CPUs.
- PE_CPU manages kernel execution and engine dispatch.

Completion semantics:

- M_CPU completes when all targeted PEs complete or a failure policy triggers.
- IO_CPU completes when all targeted CUBEs complete or a failure policy triggers.

---

### D4. Completion and failure propagation

- All messages MUST carry correlation identifiers.
- Completion and failure MUST propagate deterministically to the host.
- The simulation engine provides futures/handles to observe completion.

---

### D5. Launch timing is endpoint-synchronized

All PEs targeted by a single kernel launch MUST begin executing the kernel
body at the same simulated time, regardless of their dispatch path length
from the launch entry point.

Rationale. The dispatch tree Host → IO_CPU → M_CPU → PE_CPU has variable
latency at every level. PEs near their M_CPU receive the launch earlier
than PEs farther away; cubes near an IO_CPU receive it earlier than cubes
farther away. Without synchronization, each PE's kernel begins at a
different `env.now`, making per-PE metrics such as `pe_exec_ns` a function
of dispatch-path geometry rather than of the kernel's behavior —
producing measurement artifacts in benchmarks that time kernel-internal
waits (for example `tl.recv` on cross-cube or cross-SIP hops).

Mechanism.

- `KernelLaunchMsg` carries an optional `target_start_ns: float | None`.
- **IO_CPU** is the canonical stamper. On fan-out to M_CPUs, it
  computes `target_start_ns = env.now + max_latency` where `max_latency`
  is the maximum `ComponentContext.compute_path_latency_ns(path)` across
  every target (sip, cube, pe) tuple — `path = find_node_path(io_cpu,
  pe_cpu_id)`. The stamped value is placed on the request carried by
  every fanned-out sub-Transaction.
- **M_CPU** passes an already-stamped `target_start_ns` through
  unchanged. Only when the value is absent (e.g. a direct
  launch-to-M_CPU unit test) does M_CPU compute a per-cube barrier
  `env.now + max(local command-path latency)`.
- **PE_CPU** yields `env.timeout(target_start_ns - env.now)` at the top
  of `_execute_kernel`, before recording `pe_exec_start` and invoking
  the kernel body.
- When `target_start_ns is None`, PE_CPU falls through to the legacy
  unsynchronized behavior — preserving backward compatibility.

IO_CPU-level stamping guarantees every PE across every targeted cube
uses the same barrier sim-time, eliminating both the within-cube
dispatch-offset artifact *and* the cross-cube offset artifact in
multi-cube launches. Models a real-hardware timed-broadcast launch
(latency-equalized dispatch tree).

The synchronization is internal to the engine / IO_CPU / M_CPU / PE_CPU
control plane — runtime API and application kernels are unchanged.

---

## Links

- SPEC R1, R2, R7, R8
- ADR-0007 (Runtime API boundaries)
- ADR-0008 (Tensor deployment)