kernbench2/docs/adr/ADR-0011-memory-addressing-simplification.md

ADR-0011: Memory Addressing — PA-first with VA/MMU Extension

Status

Accepted (Phase 1 VA/MMU implemented)

Context

A realistic system uses host-side virtual addressing and an MMU/IOMMU-style translation path for DMA: host allocates physical memory at PE level, maps it into a virtual address space, installs mappings, and DMA requests use virtual addresses that are translated to physical addresses.

The PA-only model (Phase 0) was insufficient for running standard Triton kernels that use base_addr + offset patterns on sharded tensors — each PE's shard has a different PA, but the kernel needs a single contiguous address space.

---

Decision

D1. Phase 0 model is PA-only (original, retained as fallback)

• All device memory accesses (MemoryRead/MemoryWrite) operate on device physical addresses (PA) plus size.
• PA-only mode remains functional via PageFault fallback in PE_DMA.

D2. Allocation produces PA mappings

Device allocation selects PE-local memory regions and returns PA mappings sufficient to execute kernels and issue DMA requests.

D3. Phase 1: VA/MMU layer (implemented)

D3.1 Virtual Address Model

• Each tensor gets a single contiguous VA range (TensorHandle.va_base).
• TensorShard does NOT carry a va field — shard VA is derived as va_base + offset_bytes.
• Kernels receive va_base as their pointer argument (via TensorArg.va_base).
• DmaReadCmd.src_addr and DmaWriteCmd.dst_addr carry VA (not PA).

D3.2 PE_MMU Component

• Hybrid design: SimPy component (inbox for MmuMapMsg) + utility (synchronous translate() called by PE_DMA).
• Page-aligned dict lookup for O(1) VA→PA translation.
• tlb_overhead_ns configurable per-access latency.
• PageFault fallback: if VA has no mapping, PE_DMA treats it as PA directly (backward compatibility with PA-only tests).

D3.3 Mapping Installation

• MmuMapMsg traverses the fabric: Host → PCIE_EP → IO_CPU (cube fan-out) → M_CPU (PE fan-out) → NOC → PE_MMU. Latency is measured end-to-end.
• MmuMapMsg.target_sips controls SIP-level routing to prevent cross-SIP mapping contamination for replicated tensors.
• Mapping strategy based on DPPolicy.cube:
  • Replicate (cube="replicate"): per-(sip, cube) local mapping only. Each cube's PEs see only their local PA. No cross-cube mapping installed.
  • Sharded (cube="shard_m", etc.): broadcast all shard mappings to all target cubes. Enables cross-PE and cross-cube DMA.

D3.4 Tensor Lifecycle

• del tensor triggers automatic cleanup via Tensor.__del__ + weakref to RuntimeContext. Sends MmuUnmapMsg through fabric, returns VA and PA space.
• with RuntimeContext(...) as ctx: provides scope-based bulk cleanup.
• RuntimeContext._tensors uses weakref.ref to avoid preventing GC.
• PEMemAllocator uses free-list with coalescing (not bump allocator).
• VirtualAllocator uses free-list with coalescing for VA space.

D3.5 Allocators

• VirtualAllocator: device-wide VA space, page-aligned alloc/free with coalescing.
• PEMemAllocator: per-PE HBM/TCM, free-list based alloc/free with coalescing.
• Page size configurable via topology.yaml pe_mmu attrs (default 4096).

---

Consequences

• Triton kernels use base_addr + offset patterns naturally on sharded tensors.
• All latency remains explicit via graph traversal, including MMU mapping installation and per-access TLB overhead.
• PA-only mode retained as fallback (PageFault → treat as PA).
• Benchmark parameter renamed ctx → torch for PyTorch code compatibility.
• IPCQ and other fixed-address resources bypass MMU (use PA directly).

---

Links

• ADR-0007 (runtime_api vs sim_engine boundaries)
• ADR-0008 (tensor deployment)
• ADR-0009 (kernel execution)
• ADR-0014 (PE-internal execution model)
• ADR-0015 (component port/wire model)
• SPEC R2 (latency by traversal)