kernbench2/docs/adr/ADR-0033-lat-latency-model-assumptions.md

# ADR-0033 — Latency Model: Assumptions and Known Simplifications

## Status

Accepted

## Context

The simulator is an analytical, event-driven performance model — not a
cycle-accurate or RTL-level simulator. Many real-HW effects are approximated
or omitted by design. To keep the model auditable and reviewable as a whole,
this ADR consolidates the assumptions in one place. Individual component ADRs
(ADR-0015, ADR-0017, ADR-0004) define the *mechanisms*; this document defines
the *limits of fidelity*.

## Decisions

### D1. Modeled precisely

- **Per-directed-edge BW occupancy** (FIFO serialization via `available_at`) —
  ADR-0015 D2.
- **Per-component switching/overhead latency** (`overhead_ns` attr).
- **HBM per-pseudo-channel parallelism** via stateless `pc_avail[N]` array
  with address-based PC selection (ADR-0034 D3). Burst granularity tunable
  (`burst_bytes`, default 256B). Read and write share each PC's
  `available_at` (real HW command bus is per-PC shared).
- **HBM direction switching penalty mechanism**: per-PC last-direction
  tracking + configurable `switch_penalty_ns`. Default 0 — see D2.
- **Wire chunk-streaming (Phase 2c)**: each wire decomposes Transactions
  with payload into `Flit` objects of `flit_bytes` (default = HBM
  `burst_bytes` = 256B). The wire emits each flit individually after
  `prop_ns + flit_nbytes/bw_gbs` so the link's bandwidth throttles
  flit arrival rate per real-HW wormhole semantics.
- **Separate Stores per directed edge** (Phase 2c key fix): the wire
  is the *only* conduit between `src.out_ports[dst]` and
  `dst.in_ports[src]`. Earlier the two were aliased to the same
  `simpy.Store`; when the wire put a chunkified flit back, the
  destination's `fan_in` could pull it before the wire applied
  bandwidth delay, leaving half the flits bypassing the bottleneck.
- **Flit-aware pass-through** (`TransitComponent`, `HbmCtrlComponent`):
  forward each flit serially with per-transaction overhead applied
  ONCE on the first-flit arrival (header decode model). Subsequent
  flits pipeline through with no extra delay. Wormhole emerges
  naturally across multi-hop paths.
- **HBM CTRL per-flit PC commit**: each flit arriving at HBM CTRL
  schedules a PC commit at `max(env.now, pc_avail[pc]) + chunk_time`,
  with the `is_last` flit waiting for the last PC commit before
  signaling `txn.done`.
- **Non-flit-aware components (default) reassemble flits at
  ``_fan_in``** before the legacy `_forward_txn` path runs. This
  preserves backward compatibility for components that have not yet
  been migrated to flit-aware processing (e.g., `MCpuComponent`,
  `IoCpuComponent` sub-txn generators). Such components reassemble
  *once per leg boundary*, NOT per hop — multi-hop wormhole timing
  through a chain of flit-aware routers is preserved.

### D2. Approximated (with known directional error)

| Effect | Real HW | Our model | Error direction |
|--------|---------|-----------|----------------|
| Router output port arbitration | Round-robin / weighted | Wire edge FIFO + serial worker | Fair when one txn per cycle; multi-stream sharing not modeled at flit level |
| HBM scheduler / write buffer | FR-FCFS + watermark drain | FIFO, no reordering | Pessimistic for mixed R/W when alternations are dense — default `switch_penalty_ns = 0` assumes ideal scheduler amortizes |
| Flit ↔ burst granularity | 32B flit < 256B burst | `flit_bytes = burst_bytes = 256B` | Sub-flit fine-grained timing noise; affects very small wire arbitration windows only |
| Wire-level RR fairness | Per-cycle multi-flow arbitration on shared link | Single serial wire process per edge | Fair only when one transaction is in flight on a given edge at a time. Multi-stream concurrent traffic on the same edge serializes by FIFO order |

### D3. Ignored (out of scope)

- Bank-level row buffer conflict penalty (assume no conflicts — best case;
  the model has no per-bank state within a PC, so same-bank reuse cannot be
  detected).
- HBM tRP / tRCD / tFAW / tRC timing constraints (absorbed into the steady-state
  `burst_time = burst_bytes / pc_bw_gbs`).
- Refresh, ECC, thermal throttling, power gating.
- Clock domain crossings, PLL lock time.
- Upstream backpressure due to downstream buffer occupancy (input ports use
  unbounded `simpy.Store`).
- Sub-flit cycle-level arbitration at routers (flit granularity is our
  smallest unit).

### D4. Workload sensitivity

Workloads where the above simplifications meaningfully affect results:

- **Random scatter/gather**: bank conflict ignored → model optimistic.
- **Heavy mixed R/W intensive** (e.g., GEMM bias accumulation): HBM scheduler
  absent. With default `switch_penalty_ns = 0` we assume ideal amortization;
  setting it non-zero models pessimistic per-alternation cost.
- **High concurrency (>10 active flows on one link)**: HoL blocking and VC
  limits not modeled → model optimistic.
- **Very small (sub-flit) transactions**: flit quantization noise.
- **Concurrent multi-flow on a single wire**: wire is serial FIFO at the
  flit level, so per-flow fairness within a single edge is not modeled.
  Pre-edge merging (multiple sources arriving at a router and being
  forwarded to the same downstream wire) is correctly modeled via the
  flit-aware router's serial worker.

### D5. Verification policy

For workloads in D4, cross-check against real HW or a cycle-accurate
simulator before drawing absolute-magnitude conclusions. The model remains
accurate for **relative comparisons** within the modeled regime.

### D6. Future work

Note: multi-stream merging at routers IS modeled correctly — each
in_port has its own fan_in process, all push to a shared inbox, and
the router worker forwards in inbox FIFO order. Flits from different
upstream streams naturally interleave at flit granularity. The items
below are different concerns, ordered by expected workload impact.

**Higher impact (workload accuracy gap)**:

- [ ] **Bank-level conflict modeling** within a PC (opt-in via
  `track_banks: true`). Currently we assume no same-bank reuse;
  random scatter/gather workloads are optimistic here.
- [ ] **HBM scheduler** with write buffer + watermark drain (Tier 2
  from the design discussion). Default `switch_penalty_ns=0` is the
  ideal-amortization stand-in; bursty mixed R/W workloads benefit
  from explicit modeling.
- [ ] **Backpressure** modeling for finite component buffers. Matters
  at high concurrency / sustained saturation where buffer occupancy
  causes upstream stalls.
- [ ] **Op_log integration with chunk-streaming**: currently op_log
  fires on PE-internal command messages (DmaReadCmd, DmaWriteCmd,
  GemmCmd, MathCmd) which are not chunkified. Integration would
  require flit-aware components to also emit op_log start/end hooks
  per transaction (start on first flit, end on is_last).

**Lower impact (academic / specific use cases)**:

- [ ] **Cycle-accurate router arbitration policies** (RR with
  priorities, age, iSLIP). The FIFO inbox is already approximately
  fair when flit arrival times differ slightly between streams (the
  common case for similar-rate workloads). True impact appears only
  for: (a) priority/QoS modeling, (b) per-stream tail latency
  analysis under sustained saturation. Not critical for makespan or
  average-latency studies.
- [ ] **Sub-flit (32B) granularity** for finer wire arbitration
  cycles. Our `flit_bytes` equals burst (256B); real HW arbitrates
  per 32B flit. Effect is small for most workloads (sub-flit timing
  noise on small messages).

## Consequences

- Single review point for all model fidelity questions. Each future PR
  touching latency must update the relevant section here.
- Workload-specific magnitude error envelopes are explicit.
- Builder-side derivation of `pc_bw_gbs = hbm_to_router_bw_gbs / num_pcs`
  enforces the ADR-0017 D8 invariant in code rather than relying on yaml
  manual consistency.
- Wire transfer time is charged once per bottleneck-link transit (Phase 2c
  per-flit timing) rather than via terminal `drain_ns` injection. Single
  transactions land at `drain + commit_time + small_overheads`; multi-hop
  preserves wormhole pipelining; multi-stream merge correctly serializes
  at the shared wire's FIFO.

## Cross-references

- ADR-0015 — component / port / wire model.
- ADR-0017 — Cube NOC architecture and HBM connectivity.
- ADR-0004 — memory semantics, local HBM.
- ADR-0034 — HBM controller internal design.