kernbench2/docs/adr/ADR-0033-latency-model-assumptions.md

ADR-0033 — Latency Model: Assumptions and Known Simplifications

Status

Accepted

Context

The simulator is an analytical, event-driven performance model — not a cycle-accurate or RTL-level simulator. Many real-HW effects are approximated or omitted by design. To keep the model auditable and reviewable as a whole, this ADR consolidates the assumptions in one place. Individual component ADRs (ADR-0015, ADR-0019, ADR-0004) define the mechanisms; this document defines the limits of fidelity.

Decisions

D1. Modeled precisely

• Per-directed-edge BW occupancy (FIFO serialization via available_at) — ADR-0015 D2.
• Per-component switching/overhead latency (overhead_ns attr).
• HBM per-pseudo-channel parallelism via stateless pc_avail[N] array with global round-robin chunking. Burst granularity tunable (burst_bytes, default 256B). Read and write share each PC's available_at (real HW command bus is per-PC shared).
• HBM direction switching penalty mechanism: per-PC last-direction tracking + configurable switch_penalty_ns. Default 0 — see D2.
• Wire chunk-streaming (Phase 2c): each wire decomposes Transactions with payload into Flit objects of flit_bytes (default = HBM burst_bytes = 256B). The wire emits each flit individually after prop_ns + flit_nbytes/bw_gbs so the link's bandwidth throttles flit arrival rate per real-HW wormhole semantics.
• Separate Stores per directed edge (Phase 2c key fix): the wire is the only conduit between src.out_ports[dst] and dst.in_ports[src]. Earlier the two were aliased to the same simpy.Store; when the wire put a chunkified flit back, the destination's fan_in could pull it before the wire applied bandwidth delay, leaving half the flits bypassing the bottleneck.
• Flit-aware pass-through (TransitComponent, HbmCtrlComponent): forward each flit serially with per-transaction overhead applied ONCE on the first-flit arrival (header decode model). Subsequent flits pipeline through with no extra delay. Wormhole emerges naturally across multi-hop paths.
• HBM CTRL per-flit PC commit: each flit arriving at HBM CTRL schedules a PC commit at max(env.now, pc_avail[pc]) + chunk_time, with the is_last flit waiting for the last PC commit before signaling txn.done.
• Non-flit-aware components (default) reassemble flits at _fan_in before the legacy _forward_txn path runs. This preserves backward compatibility for components that have not yet been migrated to flit-aware processing (e.g., MCpuComponent, IoCpuComponent sub-txn generators). Such components reassemble once per leg boundary, NOT per hop — multi-hop wormhole timing through a chain of flit-aware routers is preserved.

D2. Approximated (with known directional error)

Effect	Real HW	Our model	Error direction
Router output port arbitration	Round-robin / weighted	Wire edge FIFO + serial worker	Fair when one txn per cycle; multi-stream sharing not modeled at flit level
HBM scheduler / write buffer	FR-FCFS + watermark drain	FIFO, no reordering	Pessimistic for mixed R/W when alternations are dense — default switch_penalty_ns = 0 assumes ideal scheduler amortizes
Flit ↔ burst granularity	32B flit < 256B burst	flit_bytes = burst_bytes = 256B	Sub-flit fine-grained timing noise; affects very small wire arbitration windows only
Wire-level RR fairness	Per-cycle multi-flow arbitration on shared link	Single serial wire process per edge	Fair only when one transaction is in flight on a given edge at a time. Multi-stream concurrent traffic on the same edge serializes by FIFO order

D3. Ignored (out of scope)

• Bank-level row buffer conflict penalty (assume no conflicts — best case; round-robin chunk assignment is address-blind so we cannot detect same-bank reuse).
• HBM tRP / tRCD / tFAW / tRC timing constraints (absorbed into the steady-state burst_time = burst_bytes / pc_bw_gbs).
• Refresh, ECC, thermal throttling, power gating.
• Clock domain crossings, PLL lock time.
• Upstream backpressure due to downstream buffer occupancy (input ports use unbounded simpy.Store).
• Sub-flit cycle-level arbitration at routers (flit granularity is our smallest unit).

D4. Workload sensitivity

Workloads where the above simplifications meaningfully affect results:

• Random scatter/gather: bank conflict ignored → model optimistic.
• Heavy mixed R/W intensive (e.g., GEMM bias accumulation): HBM scheduler absent. With default switch_penalty_ns = 0 we assume ideal amortization; setting it non-zero models pessimistic per-alternation cost.
• High concurrency (>10 active flows on one link): HoL blocking and VC limits not modeled → model optimistic.
• Very small (sub-flit) transactions: flit quantization noise.
• Concurrent multi-flow on a single wire: wire is serial FIFO at the flit level, so per-flow fairness within a single edge is not modeled. Pre-edge merging (multiple sources arriving at a router and being forwarded to the same downstream wire) is correctly modeled via the flit-aware router's serial worker.

D5. Verification policy

For workloads in D4, cross-check against real HW or a cycle-accurate simulator before drawing absolute-magnitude conclusions. The model remains accurate for relative comparisons within the modeled regime.

D6. Future work

Note: multi-stream merging at routers IS modeled correctly — each in_port has its own fan_in process, all push to a shared inbox, and the router worker forwards in inbox FIFO order. Flits from different upstream streams naturally interleave at flit granularity. The items below are different concerns.

• Cycle-accurate router arbitration policies (RR with priorities, age, iSLIP). Currently the inbox FIFO order is used as a proxy for fair RR — works when flit arrival times differ slightly between streams, but doesn't reflect intentional priority/QoS.
• Sub-flit (32B) granularity for finer wire arbitration cycles. Our flit_bytes equals burst (256B); real HW arbitrates per 32B flit. Effect is small for most workloads (sub-flit timing noise).
• Address-based PC selection at HBM CTRL (replace the address-blind global round-robin). When two transactions of size num_pcs × burst_bytes (e.g., 2KB at 8 PCs × 256B) arrive concurrently, both claim PCs 0..7 via global RR, producing full per-PC contention. Real HW uses address bits to select PCs, so different-address transactions hit different PC patterns. Address modeling would let the simulator reflect cache-line/page-aware layouts.
• Bank-level conflict modeling within a PC (opt-in via track_banks: true). Currently we assume no same-bank reuse.
• HBM scheduler with write buffer + watermark drain (Tier 2 from the design discussion). Default switch_penalty_ns=0 is the ideal-amortization stand-in.
• Backpressure modeling for finite component buffers.
• Op_log integration with chunk-streaming: currently op_log fires on PE-internal command messages (DmaReadCmd, DmaWriteCmd, GemmCmd, MathCmd) which are not chunkified. Integration would require flit-aware components to also emit op_log start/end hooks per transaction (start on first flit, end on is_last).

Consequences

• Single review point for all model fidelity questions. Each future PR touching latency must update the relevant section here.
• Workload-specific magnitude error envelopes are explicit.
• Builder-side derivation of pc_bw_gbs = hbm_to_router_bw_gbs / num_pcs enforces the ADR-0019 D9 invariant in code rather than relying on yaml manual consistency.
• Wire transfer time is charged once per bottleneck-link transit (Phase 2c per-flit timing) rather than via terminal drain_ns injection. Single transactions land at drain + commit_time + small_overheads; multi-hop preserves wormhole pipelining; multi-stream merge correctly serializes at the shared wire's FIFO.

Cross-references

• ADR-0015 — component / port / wire model.
• ADR-0019 — NoC and local HBM topology.
• ADR-0004 — memory semantics, local HBM.