kernbench2/docs/adr/ADR-0034-dev-hbm-controller-internal-design.md

ADR-0034: HBM Controller Internal Design

Status

Accepted

Context

HbmCtrlComponent is the per-PE HBM partition endpoint at the leaf of the cube NOC. One instance is created per PE under the topology node sip{S}.cube{C}.hbm_ctrl.pe{idx} and attaches to that PE's router (ADR-0017 D4). The component models per-pseudo-channel (PC) scheduling, burst-granular commit timing, address-based PC selection, and response routing back to the requester.

This ADR documents the component as currently implemented. ADR-0017 D4/D8 defines where HBM CTRL attaches and what aggregate BW it must deliver. ADR-0033 D1/D2 defines what fidelity of HBM modelling is in scope. This ADR fills the gap between those two — the per-instance internal scheduling model.

Decision

D1. Role

HbmCtrlComponent is a per-PE HBM partition endpoint. One instance per PE (default 8 per cube, set by cube.memory_map.hbm_slices_per_cube) attaches to that PE's router via the peX.hbm attachment list in cube_mesh.yaml (ADR-0017 D4). In the default n:1 channel mapping (ADR-0017 D8) the instance aggregates channels_per_pe pseudo-channels into one endpoint.

The component models:

• Per-PC scheduling (D2) with R/W command-bus sharing.
• Address-based PC selection (D3).
• Burst-granular commit timing (D4).
• Flit-aware per-flit PC commit and async finalize (D5, D6).
• Command-only Transaction handling for read-data drain (D7).
• Response routing back to the requester (D8).

It does not model:

• Bank-level row-buffer conflicts, refresh, ECC, thermal throttling (ADR-0033 D3).
• Cross-PE HBM contention beyond its own router edge (handled by the router mesh — ADR-0017 D3).
• 1:1 channel mode (ADR-0017 D8 future work).

D2. Per-PC scheduling model

Per-instance state initialised in start():

• _pc_avail: list[float] — earliest sim-time each PC is free; length num_pcs, initial 0.0.
• _pc_last_dir: list["R"|"W"|None] — direction of the last commit on each PC, used for switch-penalty detection (D4); initial None.

num_pcs and burst_bytes must each be a positive power of two so that address-based PC selection (D3) reduces to a shift-and-mask.

Read and write requests share the same _pc_avail slot per PC — the real HW per-PC command bus is shared between read and write traffic, so issuing a write to PC k blocks a subsequent read to PC k by exactly the burst time.

Direction dir for a request is inferred from the request type:

• MemoryWriteMsg → "W".
• PeDmaMsg with is_write=True → "W".
• All others (MemoryReadMsg, PeDmaMsg read) → "R".

D3. Address-based PC selection

PC index for an access is derived from the access address by shift and mask:

pc_shift = log2(burst_bytes)         # default 8  (burst=256B)
pc_mask  = num_pcs - 1               # default 7  (8 PCs)
pc       = (address >> pc_shift) & pc_mask

Computed once in start() from topology config so alternative (burst_bytes, num_pcs) pairs stay consistent. For the canonical default (256, 8) this places the PC select field at bits [10:8] of the HBM byte offset: bits [7:0] are within-burst (same PC), bits [10:8] are the 3-bit PC index, bits [36:11] are row/bank/column within the PC slice (see phyaddr.py comment).

Address-based striping — as opposed to address-blind global round-robin — preserves PC parallelism for offset-disjoint concurrent transfers: each transfer's bursts land deterministically on the PC set implied by its byte addresses, so multi-PE workloads accessing disjoint regions do not collide on a single PC.

D4. Burst granularity and PC commit timing

A single PC commit takes:

chunk_time = burst_bytes / pc_bw_gbs    # ns

• burst_bytes (default 256) is the burst granularity matching the flit size (ADR-0033 D1).
• pc_bw_gbs is builder-derived from hbm_to_router_bw_gbs / num_pcs (topology/builder.py), enforcing the ADR-0017 D8 invariant that aggregate per-PE BW equals the router-to-HBM link BW.

Per-PC commit scheduling for an arriving access on PC pc with direction dir:

switch_cost = switch_penalty_ns
              if pc_last_dir[pc] not in (None, dir) else 0
start  = max(env.now, pc_avail[pc]) + switch_cost
finish = start + chunk_time
pc_avail[pc]    = finish
pc_last_dir[pc] = dir

Default switch_penalty_ns = 0 — Tier 0 assumption that an ideal HBM scheduler amortises R/W switching cost (ADR-0033 D2). Non-zero values model pessimistic per-alternation cost.

D5. Flit-aware per-flit PC commit (primary path)

_handle_flit is the primary worker path. For each arriving Flit:

1. On the first flit of a transaction (tid = id(txn) not in _txn_state):
   • Apply overhead_ns once via run(env, nbytes) — header decode model, first-flit overhead pattern (ADR-0033 D1).
   • Initialise _txn_state[tid] = {"last_finish": env.now}.
2. Compute pc = _pc_for_address(flit.address) (D3).
3. Apply the per-PC schedule (D4) using the request direction (D2).
4. Update state["last_finish"] = max(state["last_finish"], finish).
5. If flit.is_last: pop _txn_state[tid] and spawn _finalize_txn (D6).

Per-flit address-aware commit is the mechanism that lets concurrent multi-PE traffic to disjoint HBM offsets pipeline through distinct PCs in parallel.

D6. Async finalize per transaction

When a transaction's last flit has been scheduled, finalisation runs in a separately-spawned process:

def _finalize_txn(env, txn, last_finish):
    wait = last_finish - env.now
    if wait > 0:
        yield env.timeout(wait)
    yield from _send_response(env, txn)

_handle_flit spawns this via env.process(...) and returns immediately, so the worker can pick up the next inbox message while the last PC commit drains.

Without this split — i.e. if the worker itself did yield env.timeout(wait) — concurrent single-flit transactions whose addresses hit distinct PCs would still serialise at chunk_time each inside the worker, hiding the PC parallelism that D3 and D5 are designed to expose.

D7. Non-flit fallback for command-only transactions

_handle_txn runs when the inbox delivers a Transaction rather than a Flit. This is the path for command-only requests that the wire does not chunk into flits — most notably MemoryReadMsg whose command txn carries nbytes=0 (data drain is modelled at HBM CTRL post-processing, not as inbound flits).

Procedure:

1. work_bytes = txn.nbytes if txn.nbytes > 0 else int(request.nbytes or 0) — for read commands, work is sized by the request.
2. n_chunks = ceil(work_bytes / burst_bytes) if work_bytes > 0 else 0.
3. chunk_interval = drain_ns / n_chunks (when both > 0) — chunks are scheduled over time at drain/n_chunks ns intervals to model the bottleneck-link's data arrival rate (ADR-0033 D1 chunk-loop drain).
4. Apply run(env, txn.nbytes) once for overhead_ns.
5. For each chunk i, advance chunk_interval ns then apply the D4 schedule with pc = _pc_for_address(base_address + i * burst_bytes).
6. After scheduling all chunks, wait last_finish - env.now then call _send_response.

_handle_txn shares the same _pc_avail / _pc_last_dir state with _handle_flit — there is exactly one source of PC scheduling truth across both paths.

D8. Response routing

_send_response dispatches on request type and path geometry:

Case	Trigger	Response
PE_DMA	isinstance(txn.request, PeDmaMsg)	New reverse-path Transaction (is_response=True, nbytes=0), same done
Bypass — Memory Read	"m_cpu" not in any(txn.path) AND MemoryReadMsg	Reverse-path Transaction with nbytes=request.nbytes (data return)
Bypass — Memory Write	"m_cpu" not in any(txn.path) AND not Memory Read	txn.done.succeed() (write completes locally)
Default	otherwise	New ResponseMsg(correlation_id, request_id, src_cube, src_pe, success=True) on reverse path

The "bypass" classification matches the Memory R/W fabric path defined in ADR-0015 D4 (PCIE_EP → io_noc → ucie → cube router → hbm_ctrl, without M_CPU). The PE_DMA case is its own dedicated reverse-path to keep the inner-loop DMA fast (PE_DMA reads/writes do not synthesise a ResponseMsg envelope).

In all reverse-path cases, the response Transaction is put onto out_ports[reverse_path[1]] — the first hop back along the recorded forward path. If reverse_path has fewer than 2 entries (degenerate path), the original txn.done is signalled directly.

D9. Configurable attributes

Attribute	Default	Source	Notes
num_pcs	8	topology cube hbm_ctrl.attrs	Must be power of 2
pc_bw_gbs	32.0	builder-derived: hbm_to_router_bw_gbs / num_pcs	Enforces ADR-0017 D8 invariant
burst_bytes	256	topology attrs	Must be power of 2; equals flit_bytes (ADR-0033 D1)
switch_penalty_ns	0.0	topology attrs	Tier 0 default; non-zero models pessimistic R/W switching
efficiency	1.0	topology attrs	Applied at builder time to hbm_to_router_bw_gbs (router-edge BW scaling only)
overhead_ns	0.0	topology attrs	First-flit decode overhead (D5)

pc_bw_gbs is derived by topology/builder.py rather than configured directly so the aggregate per-PE BW matches the router-to-HBM link BW without yaml-side duplication.

Consequences

Positive

• Address-based PC selection preserves multi-stream HBM parallelism that an address-blind round-robin would collapse — important for multi-PE workloads with disjoint HBM regions.
• Flit-aware path (D5) + async finalize (D6) preserves wormhole pipelining and exposes PC parallelism for back-to-back single-flit transactions.
• Single source of PC scheduling truth (D4 mechanism, used by both D5 flit path and D7 chunk-loop path).
• Builder-derived pc_bw_gbs enforces ADR-0017 D8 in code, not yaml discipline.

Negative

• No bank-level conflict modelling within a PC; address-blind to bank/row-buffer reuse (ADR-0033 D3).
• No HBM scheduler (FR-FCFS / write-buffer / watermark drain); fixed FIFO per PC. Bursty mixed R/W is approximated by switch_penalty_ns (ADR-0033 D2).
• _txn_state is a regular dict keyed by id(txn); in-flight state accumulates per concurrent transaction and is removed only on is_last. Adequate for current workloads.

Links

• ADR-0001 (Physical address layout — PC bit field comment)
• ADR-0015 D4 (Memory R/W fabric path — bypass response case)
• ADR-0017 D4 (Per-PE HBM partitioning — attachment to PE routers)
• ADR-0017 D8 (HBM channel mapping mode — n:1 aggregate this ADR implements)
• ADR-0017 D9 (AddressResolver — hbm_ctrl.pe{pe_id} endpoint resolution)
• ADR-0033 D1 (Modelled precisely — per-PC parallelism, switch penalty, flit-aware PC commit, first-flit overhead, chunk-loop drain)
• ADR-0033 D2 (Switch-penalty default 0 — ideal scheduler amortisation)