kernbench2/docs/adr/ADR-0002-lat-routing-distance.md

# ADR-0002: Routing Distance, Ordering & Bypass Rules

## Status
Accepted

## Date
2026-02-27

## Context
The KernBench Graph Latency Simulator must compare kernel execution time
across different architectures and topologies by computing end-to-end
latency from graph traversal.

To support meaningful comparison:
- routing must be deterministic
- latency must reflect actual interconnect structure
- local vs remote traffic must be distinguishable
- “bypass” optimizations must not undermine debuggability or correctness

The simulator also aims to avoid software-managed metadata and hidden
shortcuts that obscure control paths.

## Decision

### D1. Distance is accumulated latency, not hop count
- Routing “distance” is defined as the **sum of per-node and per-link latency**.
- Hop count alone must not be used for ordering or path selection.
- Size-aware serialization latency (bytes / BW) contributes to distance.

### D2. Routing order is derived from graph traversal
- The chosen route is the path with minimum accumulated latency
  given the constructed graph and routing policy.
- Deterministic ordering must be guaranteed for identical inputs
  (topology + policy + request).

### D3. Bypass is explicit and graph-represented
- All paths must be explicitly represented in the graph and subject to latency accumulation.
- Example: PE_DMA connects to the NOC router mesh (ADR-0017 D7). All destinations
  (HBM, shared SRAM, inter-cube UCIe) are reached via explicit mesh hops.
  Local HBM access has minimal hops (switching overhead only); remote access
  traverses additional routers.
- Implicit or “magic” bypass paths are disallowed.

### D4. No zero-latency end-to-end paths

- Every routed request must incur **end-to-end** latency > 0.
- Individual fabric segments (e.g., NOC hops) MAY have distance_mm = 0
  when the fabric is distributed and distance is not meaningful at that granularity.
  This is allowed because other components on the same path (e.g., PE_DMA, SRAM,
  UCIe endpoints) contribute non-zero latency, ensuring the end-to-end invariant holds.
- Fully zero-latency end-to-end paths are disallowed, except for explicit
  test-only stubs clearly marked as such.

### D5. Policy vs topology responsibility split
- Topology builder:
  - defines nodes and links and their latency/BW parameters
- Routing policy:
  - selects among available graph paths based on decoded domains
- Routing policy must not assume missing links; missing connectivity
  is a topology construction error.

### D6. No software-managed routing metadata
- Routing decisions must not rely on per-request software-managed metadata
  that tracks distance, hop count, or ordering outside the graph model.
- All distance/order computation is derived from traversal itself.

## Alternatives Considered

1) **Hop-count based routing**
- Rejected: ignores heterogeneous latency/BW and misrepresents
  architectural differences.

2) **Implicit local shortcuts**
- Rejected: breaks debuggability and violates traversal-based latency.

3) **Software-managed distance metadata**
- Rejected: increases control overhead and obscures routing semantics.

## Consequences

### Positive
- Clear, debuggable hop-by-hop traces (SPEC R2, R4).
- Architecture comparisons reflect real interconnect structure.
- Routing behavior is reproducible and deterministic.

### Tradeoffs / Costs
- Graph construction must be correct and complete.
- Bypass modeling requires explicit graph representation,
  which slightly increases topology description complexity.

## Implementation Notes (Non-normative)
- Recommended responsibilities:
  - Graph builder: ensure all required paths exist.
  - Router: select next hop based on decoded domains and policy.
- Tests should assert:
  - non-zero end-to-end latency
  - deterministic routing for identical inputs
  - bypass paths appear explicitly in emitted traces

## Links
- SPEC.md: R1 (routing), R2 (latency), R3 (topology), R5 (multi-domain comm)
- ADR-0001: PhysAddr layout & decoding contract