kernbench2/docs/adr/ADR-0016-dev-iochiplet-noc-and-memory-path.md

ADR-0016: IOChiplet NOC and Memory Data Path

Status

Accepted

Context

ADR-0003 D2 defines IO chiplets as SIP-level components providing PCIe-EP and IO_CPU interfaces, but does not specify internal routing within the IO chiplet. ADR-0015 D4 was updated to document the M_CPU bypass for Memory R/W, but the IO chiplet's internal NOC architecture that enables this routing was not formally documented.

The IO chiplet needs an internal routing fabric (io_noc) to:

• connect pcie_ep, io_cpu, and per-cube UCIe PHY ports
• route memory operations (MemoryWrite/Read) directly to cube fabric without passing through io_cpu
• route kernel launch commands through io_cpu for command interpretation

Decision

D1. IOChiplet internal NOC (io_noc)

Each IO chiplet instance contains an internal NOC node (io_noc) that connects:

• pcie_ep — host-facing PCIe endpoint
• io_cpu — command processor for kernel launch interpretation
• io_ucie-{PHY}.conn{N} — per-PHY connection nodes to cube UCIe ports

The io_noc is a forwarding-only fabric (forwarding_v1 implementation) with zero overhead. All routing decisions are made by the simulation engine based on message type, not by io_noc itself.

D2. IOChiplet UCIe decomposition

Each IO chiplet PHY port is decomposed into:

• io_ucie-{PHY} — the UCIe protocol endpoint (overhead = 8ns)
• io_ucie-{PHY}.conn{N} — N connection nodes between io_noc and io_ucie

This mirrors the cube-side UCIe decomposition (ADR-0015 D1) and allows multiple independent NOC-to-UCIe connections per PHY.

D3. Memory R/W path (M_CPU bypass)

Memory operations (MemoryWrite, MemoryRead) are routed directly from pcie_ep through io_noc to the target cube, bypassing io_cpu entirely:

pcie_ep → io_noc → conn → io_ucie → [cube UCIe] → router mesh → hbm_ctrl

This avoids the 10ns io_cpu overhead for pure data transfers. The simulation engine's _process_memory_direct() method uses find_memory_path() which resolves the shortest path from pcie_ep to the target HBM node.

D4. Kernel Launch path (via io_cpu)

Kernel launch commands require io_cpu for command interpretation and PE fan-out setup:

pcie_ep → io_noc → io_cpu → io_noc → conn → io_ucie → [cube UCIe]
  → noc → m_cpu → PE

The engine's _entry_points() method routes KernelLaunchMsg through both pcie_ep (entry) and io_cpu (command processing).

D5. IOChiplet-to-cube port mapping

Each IO chiplet instance declares which cube ports it connects to:

cube_ports:
  - { cube: {xy: [0,0]}, cube_side: N, phy: P0, distance_mm: 2.0 }
  - { cube: {xy: [1,0]}, cube_side: N, phy: P1, distance_mm: 2.0 }

The topology builder creates edges from io_ucie PHY nodes to the corresponding cube UCIe port nodes, with the specified distance and the IO chiplet's per_connection_bw_gbs as link bandwidth.

Consequences

• IO chiplet has a well-defined internal routing fabric
• Memory operations avoid unnecessary io_cpu overhead
• Kernel launch commands still get proper command interpretation
• The io_noc pattern is consistent with cube-level NOC design
• ADR-0003 D2 is extended (not contradicted) by this ADR

Links

• ADR-0003 D2 (IO chiplet definition)
• ADR-0015 D4 (fabric paths for Memory R/W and Kernel Launch)
• ADR-0012 D1 (host-to-IO_CPU message schema)