Files

T

ywkang fc6abbc8ee Add CHANGES.md, README, update SPEC/ADRs for release 2

- CHANGES.md: detailed changelog for release 1 and 2
- README.md: full project docs with install, probe, run, test usage
- SPEC.md: add ADR-0014~0017 references, update R7 for pcie_ep endpoint
- ADR-0003: update NOC description to reference ADR-0017
- ADR-0004: add HBM efficiency factor (0.8) to BW guarantee contract
- ADR-0014: status Proposed -> Accepted
- ADR-0015: update D4 to M_CPU bypass for Memory R/W, add ADR-0016/0017 links
- ADR-0016 (new): IOChiplet NOC and memory data path
- ADR-0017 (new): Cube NOC 2D mesh architecture
- Fix MD lint warnings (unfenced code blocks) across all docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-19 01:43:15 -07:00

2.9 KiB

Raw Blame History

ADR-0004: Memory Semantics & Local-HBM Bandwidth Guarantee

Status

Accepted

Context

Accurately modeling PE↔HBM behavior is essential for kernel latency estimation. Each PE has a notion of “local HBM” that must guarantee full HBM bandwidth, independent of intervening on-die fabric bandwidth.

Decision

D1. Local HBM definition

Each PE is assigned a logically defined “local HBM” region.
Local HBM corresponds to the pseudo-channel subset directly attached to that PE’s DMA path via the XBAR (top or bottom, depending on PE corner placement).
The path is: PE_DMA → XBAR.top/bottom → HBM_CTRL.
The mapping (HBM pseudo-channels → PE local regions) is derived from topology configuration.

D2. Local HBM bandwidth guarantee contract

Accesses from a PE to its local HBM MUST guarantee full effective HBM read/write bandwidth independent of intervening fabric bandwidth limits.
Effective HBM bandwidth = spec bandwidth x efficiency factor. The efficiency factor (configured via hbm_ctrl.attrs.efficiency, default 0.8) models real-world DRAM inefficiencies (refresh cycles, bank conflicts, page misses). For example: 256 GB/s spec x 0.8 = 204.8 GB/s effective.
The topology builder applies the efficiency factor to xbar-to-hbm edge bandwidth at graph construction time, so all downstream routing and latency computation uses the effective value.
This guarantee is modeled by:
- a dedicated logical path and/or service model that enforces HBM BW at the PE-local-HBM interaction point,
- while still incurring non-zero latency along explicitly modeled components.

D3. Cross-half HBM semantics

A PE connected to XBAR.bottom that accesses HBM pseudo-channels on the XBAR.top half (or vice versa) traverses a bridge:
- PE_DMA → XBAR.bottom → bridge → XBAR.top → HBM_CTRL
Bridge bandwidth may limit cross-half HBM access relative to local-half access.

D4. Non-local HBM semantics (inter-cube / inter-SIP)

Accesses from a PE to HBM in a different cube or SIP MAY be limited by:
- NOC bandwidth within the cube,
- inter-cube UCIe links,
- inter-SIP fabric (PCIe/UAL).
These paths MUST be explicit and traceable.

D5. Shared SRAM semantics

Each CUBE contains a shared SRAM accessible by all PEs in that CUBE.
Access path: PE_DMA → NOC → shared SRAM.
Shared SRAM bandwidth is limited by the NOC↔SRAM link bandwidth.
Shared SRAM is not part of the HBM address space; it is a separate memory domain.

Verification Notes

Tests should cover:

local-HBM case: BW matches HBM BW regardless of fabric BW parameter
cross-half HBM case: latency includes bridge traversal
non-local cases (inter-cube/inter-SIP): BW/latency respond to fabric/link parameters
shared SRAM case: access via NOC with correct BW

2.9 KiB Raw Blame History Unescape Escape