kernbench2/docs/adr/ADR-0034-dev-hbm-controller-internal-design.md

# ADR-0034: HBM Controller Internal Design

## Status

Accepted

## Context

`HbmCtrlComponent` is the per-PE HBM partition endpoint at the leaf of
the cube NOC. One instance is created per PE under the topology node
`sip{S}.cube{C}.hbm_ctrl.pe{idx}` and attaches to that PE's router
(ADR-0017 D4). The component models per-pseudo-channel (PC) scheduling,
burst-granular commit timing, address-based PC selection, and response
routing back to the requester.

This ADR documents the component as currently implemented. ADR-0017 D4/D8
defines *where* HBM CTRL attaches and *what* aggregate BW it must
deliver. ADR-0033 D1/D2 defines *what fidelity* of HBM modelling is in
scope. This ADR fills the gap between those two — the per-instance
internal scheduling model.

## Decision

### D1. Role

`HbmCtrlComponent` is a per-PE HBM partition endpoint. One instance per
PE (default 8 per cube, set by `cube.memory_map.hbm_slices_per_cube`)
attaches to that PE's router via the `peX.hbm` attachment list in
`cube_mesh.yaml` (ADR-0017 D4). In the default n:1 channel mapping
(ADR-0017 D8) the instance aggregates `channels_per_pe` pseudo-channels
into one endpoint.

The component models:

- Per-PC scheduling (D2) with R/W command-bus sharing.
- Address-based PC selection (D3).
- Burst-granular commit timing (D4).
- Flit-aware per-flit PC commit and async finalize (D5, D6).
- Command-only Transaction handling for read-data drain (D7).
- Response routing back to the requester (D8).

It does not model:

- Bank-level row-buffer conflicts, refresh, ECC, thermal throttling
  (ADR-0033 D3).
- Cross-PE HBM contention beyond its own router edge (handled by the
  router mesh — ADR-0017 D3).
- 1:1 channel mode (ADR-0017 D8 future work).

### D2. Per-PC scheduling model

Per-instance state initialised in `start()`:

- `_pc_avail: list[float]` — earliest sim-time each PC is free; length
  `num_pcs`, initial 0.0.
- `_pc_last_dir: list["R"|"W"|None]` — direction of the last commit on
  each PC, used for switch-penalty detection (D4); initial `None`.

`num_pcs` and `burst_bytes` must each be a positive power of two so
that address-based PC selection (D3) reduces to a shift-and-mask.

Read and write requests share the same `_pc_avail` slot per PC — the
real HW per-PC command bus is shared between read and write traffic, so
issuing a write to PC k blocks a subsequent read to PC k by exactly the
burst time.

Direction `dir` for a request is inferred from the request type:

- `MemoryWriteMsg` → `"W"`.
- `PeDmaMsg` with `is_write=True` → `"W"`.
- All others (`MemoryReadMsg`, `PeDmaMsg` read) → `"R"`.

### D3. Address-based PC selection

PC index for an access is derived from the access address by shift and
mask:

```text
pc_shift = log2(burst_bytes)         # default 8  (burst=256B)
pc_mask  = num_pcs - 1               # default 7  (8 PCs)
pc       = (address >> pc_shift) & pc_mask
```

Computed once in `start()` from topology config so alternative
`(burst_bytes, num_pcs)` pairs stay consistent. For the canonical
default `(256, 8)` this places the PC select field at bits `[10:8]` of
the HBM byte offset: bits `[7:0]` are within-burst (same PC), bits
`[10:8]` are the 3-bit PC index, bits `[36:11]` are row/bank/column
within the PC slice (see `phyaddr.py` comment).

Address-based striping — as opposed to address-blind global
round-robin — preserves PC parallelism for offset-disjoint concurrent
transfers: each transfer's bursts land deterministically on the PC set
implied by its byte addresses, so multi-PE workloads accessing disjoint
regions do not collide on a single PC.

### D4. Burst granularity and PC commit timing

A single PC commit takes:

```text
chunk_time = burst_bytes / pc_bw_gbs    # ns
```

- `burst_bytes` (default 256) is the burst granularity matching the
  flit size (ADR-0033 D1).
- `pc_bw_gbs` is **builder-derived** from
  `hbm_to_router_bw_gbs / num_pcs` (`topology/builder.py`), enforcing
  the ADR-0017 D8 invariant that aggregate per-PE BW equals the
  router-to-HBM link BW.

Per-PC commit scheduling for an arriving access on PC `pc` with
direction `dir`:

```text
switch_cost = switch_penalty_ns
              if pc_last_dir[pc] not in (None, dir) else 0
start  = max(env.now, pc_avail[pc]) + switch_cost
finish = start + chunk_time
pc_avail[pc]    = finish
pc_last_dir[pc] = dir
```

Default `switch_penalty_ns = 0` — Tier 0 assumption that an ideal HBM
scheduler amortises R/W switching cost (ADR-0033 D2). Non-zero values
model pessimistic per-alternation cost.

### D5. Flit-aware per-flit PC commit (primary path)

`_handle_flit` is the primary worker path. For each arriving `Flit`:

1. On the **first** flit of a transaction (`tid = id(txn)` not in
   `_txn_state`):
   - Apply `overhead_ns` once via `run(env, nbytes)` — header decode
     model, first-flit overhead pattern (ADR-0033 D1).
   - Initialise `_txn_state[tid] = {"last_finish": env.now}`.
2. Compute `pc = _pc_for_address(flit.address)` (D3).
3. Apply the per-PC schedule (D4) using the request direction (D2).
4. Update `state["last_finish"] = max(state["last_finish"], finish)`.
5. If `flit.is_last`: pop `_txn_state[tid]` and spawn `_finalize_txn`
   (D6).

Per-flit address-aware commit is the mechanism that lets concurrent
multi-PE traffic to disjoint HBM offsets pipeline through distinct PCs
in parallel.

### D6. Async finalize per transaction

When a transaction's last flit has been scheduled, finalisation runs in
a separately-spawned process:

```python
def _finalize_txn(env, txn, last_finish):
    wait = last_finish - env.now
    if wait > 0:
        yield env.timeout(wait)
    yield from _send_response(env, txn)
```

`_handle_flit` spawns this via `env.process(...)` and returns
immediately, so the worker can pick up the next inbox message while the
last PC commit drains.

Without this split — i.e. if the worker itself did
`yield env.timeout(wait)` — concurrent single-flit transactions whose
addresses hit distinct PCs would still serialise at `chunk_time` each
inside the worker, hiding the PC parallelism that D3 and D5 are
designed to expose.

### D7. Non-flit fallback for command-only transactions

`_handle_txn` runs when the inbox delivers a `Transaction` rather than a
`Flit`. This is the path for command-only requests that the wire does
not chunk into flits — most notably `MemoryReadMsg` whose command txn
carries `nbytes=0` (data drain is modelled at HBM CTRL post-processing,
not as inbound flits).

Procedure:

1. `work_bytes = txn.nbytes if txn.nbytes > 0 else int(request.nbytes or 0)`
   — for read commands, work is sized by the request.
2. `n_chunks = ceil(work_bytes / burst_bytes)` if `work_bytes > 0` else
   0.
3. `chunk_interval = drain_ns / n_chunks` (when both > 0) — chunks are
   scheduled over time at `drain/n_chunks` ns intervals to model the
   bottleneck-link's data arrival rate (ADR-0033 D1 chunk-loop drain).
4. Apply `run(env, txn.nbytes)` once for `overhead_ns`.
5. For each chunk `i`, advance `chunk_interval` ns then apply the D4
   schedule with `pc = _pc_for_address(base_address + i * burst_bytes)`.
6. After scheduling all chunks, wait `last_finish - env.now` then call
   `_send_response`.

`_handle_txn` shares the same `_pc_avail` / `_pc_last_dir` state with
`_handle_flit` — there is exactly one source of PC scheduling truth
across both paths.

### D8. Response routing

`_send_response` dispatches on request type and path geometry:

| Case | Trigger | Response |
| --- | --- | --- |
| PE_DMA | `isinstance(txn.request, PeDmaMsg)` | New reverse-path Transaction (`is_response=True`, `nbytes=0`), same `done` |
| Bypass — Memory Read | `"m_cpu" not in any(txn.path)` AND `MemoryReadMsg` | Reverse-path Transaction with `nbytes=request.nbytes` (data return) |
| Bypass — Memory Write | `"m_cpu" not in any(txn.path)` AND not Memory Read | `txn.done.succeed()` (write completes locally) |
| Default | otherwise | New `ResponseMsg(correlation_id, request_id, src_cube, src_pe, success=True)` on reverse path |

The "bypass" classification matches the Memory R/W fabric path defined
in ADR-0015 D4 (PCIE_EP → io_noc → ucie → cube router → hbm_ctrl,
without M_CPU). The PE_DMA case is its own dedicated reverse-path to
keep the inner-loop DMA fast (PE_DMA reads/writes do not synthesise a
ResponseMsg envelope).

In all reverse-path cases, the response Transaction is put onto
`out_ports[reverse_path[1]]` — the first hop back along the recorded
forward path. If `reverse_path` has fewer than 2 entries (degenerate
path), the original `txn.done` is signalled directly.

### D9. Configurable attributes

| Attribute | Default | Source | Notes |
| --- | --- | --- | --- |
| `num_pcs` | 8 | topology cube `hbm_ctrl.attrs` | Must be power of 2 |
| `pc_bw_gbs` | 32.0 | builder-derived: `hbm_to_router_bw_gbs / num_pcs` | Enforces ADR-0017 D8 invariant |
| `burst_bytes` | 256 | topology attrs | Must be power of 2; equals `flit_bytes` (ADR-0033 D1) |
| `switch_penalty_ns` | 0.0 | topology attrs | Tier 0 default; non-zero models pessimistic R/W switching |
| `efficiency` | 1.0 | topology attrs | Applied at builder time to `hbm_to_router_bw_gbs` (router-edge BW scaling only) |
| `overhead_ns` | 0.0 | topology attrs | First-flit decode overhead (D5) |

`pc_bw_gbs` is derived by `topology/builder.py` rather than configured
directly so the aggregate per-PE BW matches the router-to-HBM link BW
without yaml-side duplication.

## Consequences

### Positive

- Address-based PC selection preserves multi-stream HBM parallelism
  that an address-blind round-robin would collapse — important for
  multi-PE workloads with disjoint HBM regions.
- Flit-aware path (D5) + async finalize (D6) preserves wormhole
  pipelining and exposes PC parallelism for back-to-back single-flit
  transactions.
- Single source of PC scheduling truth (D4 mechanism, used by both D5
  flit path and D7 chunk-loop path).
- Builder-derived `pc_bw_gbs` enforces ADR-0017 D8 in code, not yaml
  discipline.

### Negative

- No bank-level conflict modelling within a PC; address-blind to
  bank/row-buffer reuse (ADR-0033 D3).
- No HBM scheduler (FR-FCFS / write-buffer / watermark drain); fixed
  FIFO per PC. Bursty mixed R/W is approximated by `switch_penalty_ns`
  (ADR-0033 D2).
- `_txn_state` is a regular dict keyed by `id(txn)`; in-flight state
  accumulates per concurrent transaction and is removed only on
  `is_last`. Adequate for current workloads.

## Links

- ADR-0001 (Physical address layout — PC bit field comment)
- ADR-0015 D4 (Memory R/W fabric path — bypass response case)
- ADR-0017 D4 (Per-PE HBM partitioning — attachment to PE routers)
- ADR-0017 D8 (HBM channel mapping mode — n:1 aggregate this ADR
  implements)
- ADR-0017 D9 (AddressResolver — `hbm_ctrl.pe{pe_id}` endpoint
  resolution)
- ADR-0033 D1 (Modelled precisely — per-PC parallelism, switch penalty,
  flit-aware PC commit, first-flit overhead, chunk-loop drain)
- ADR-0033 D2 (Switch-penalty default 0 — ideal scheduler amortisation)