365 lines
8.0 KiB
Markdown
365 lines
8.0 KiB
Markdown
# ADR-0014: PE Internal Execution Model (PE_CPU, PE_SCHEDULER, and Composite Commands)
|
||
|
||
## Status
|
||
|
||
Proposed
|
||
|
||
## Context
|
||
|
||
ADR-0003 (system hierarchy) and ADR-0009 (kernel execution semantics) reference PE internals but do not define:
|
||
|
||
- the dispatch model inside a PE,
|
||
- the responsibilities of PE_SCHEDULER,
|
||
- the PE_TCM-centric dataflow contract used by accelerator engines.
|
||
|
||
We need a deterministic and debuggable PE-internal execution contract that supports:
|
||
|
||
- simple single-engine commands
|
||
- composite commands that build a tiled pipeline across DMA and accelerator engines
|
||
|
||
The simulator must produce deterministic traces and allow modeling of PE-internal pipelining without introducing nondeterministic engine scheduling.
|
||
|
||
## Decision
|
||
|
||
### D1. PE internal component roles
|
||
|
||
Each PE contains the following logical components.
|
||
|
||
**PE_CPU**
|
||
|
||
- Executes kernel instruction stream or kernel control logic.
|
||
- Generates PE commands.
|
||
- Submits commands to PE_SCHEDULER.
|
||
- PE_CPU does NOT enqueue work directly into engine queues.
|
||
|
||
**PE_SCHEDULER**
|
||
|
||
- The sole dispatcher inside a PE.
|
||
- Receives commands from PE_CPU.
|
||
- Expands composite commands into sub-commands.
|
||
- Tracks dependencies and command state.
|
||
- Dispatches work to engine queues.
|
||
- Manages tile scheduling for composite commands.
|
||
|
||
**PE_DMA**
|
||
|
||
- Handles memory transfers between PE_TCM and external memory domains.
|
||
- PE_DMA has **dual egress** at the CUBE level:
|
||
- **→ XBAR**: dedicated path to HBM (local and cross-half via bridge)
|
||
- **→ NOC**: path to non-HBM destinations (shared SRAM, inter-cube UCIe, etc.)
|
||
- Supported directions include:
|
||
- HBM → PE_TCM (via XBAR)
|
||
- PE_TCM → HBM (via XBAR)
|
||
- PE_TCM → shared SRAM (via NOC)
|
||
- PE_TCM → other memory domains (via NOC, if supported by topology)
|
||
|
||
**PE_GEMM**
|
||
|
||
- Matrix multiplication engine.
|
||
- Reads activations from PE_TCM.
|
||
- May stream weights directly from HBM.
|
||
|
||
**PE_MATH**
|
||
|
||
- Element-wise computation engine.
|
||
- Reads and writes PE_TCM.
|
||
|
||
**PE_TCM**
|
||
|
||
- Local SRAM used as the staging memory for accelerator operations.
|
||
|
||
---
|
||
|
||
### D2. Command lifecycle and queues
|
||
|
||
PE_SCHEDULER maintains three logical structures.
|
||
|
||
**SubmissionQueue**
|
||
|
||
- Written by PE_CPU.
|
||
- Contains incoming PE commands waiting to be processed.
|
||
|
||
**InflightTable**
|
||
|
||
- Owned and mutated only by PE_SCHEDULER.
|
||
- Tracks:
|
||
- expanded sub-commands
|
||
- dependency state
|
||
- engine assignment
|
||
- completion status
|
||
|
||
**CompletionQueue**
|
||
|
||
- Written by PE_SCHEDULER.
|
||
- Contains final completion records for commands.
|
||
|
||
**Single-writer rule**
|
||
|
||
- Only PE_SCHEDULER is allowed to mutate command completion state.
|
||
- Engine components must report completion via explicit completion events/messages.
|
||
|
||
**Command completion**
|
||
|
||
A command becomes DONE when:
|
||
|
||
- all sub-commands complete
|
||
- PE_SCHEDULER publishes a completion record to CompletionQueue.
|
||
|
||
---
|
||
|
||
### D3. Dispatch modes
|
||
|
||
PE commands are divided into two categories.
|
||
|
||
#### D3.1 Simple command
|
||
|
||
A simple command expands to exactly one engine sub-command.
|
||
|
||
Examples include:
|
||
|
||
- DMA transfer
|
||
- GEMM compute
|
||
- MATH compute
|
||
|
||
Execution flow:
|
||
|
||
```
|
||
PE_CPU → SubmissionQueue → PE_SCHEDULER → engine queue → engine execution → completion event → PE_SCHEDULER → CompletionQueue
|
||
```
|
||
|
||
#### D3.2 Composite command (tiled pipeline)
|
||
|
||
Composite commands implement tiled pipelined execution across engines.
|
||
|
||
Each tile executes the following pipeline:
|
||
|
||
```
|
||
Input DMA (READ)
|
||
→ Compute (GEMM or MATH)
|
||
→ Output DMA (WRITE)
|
||
```
|
||
|
||
**Tiling rule**
|
||
|
||
If the DMA payload exceeds hardware tile size, PE_SCHEDULER splits the transfer into tiles.
|
||
Each tile is assigned a monotonically increasing `tile_id`.
|
||
|
||
**Tile dependency rules**
|
||
|
||
For tile `t`:
|
||
|
||
- Compute must wait for input DMA: `DMA_READ(t) → COMPUTE(t)`
|
||
- Output DMA must wait for compute: `COMPUTE(t) → DMA_WRITE(t)`
|
||
- All dependencies are enforced by PE_SCHEDULER.
|
||
|
||
**Overlap policy (Phase 0 default)**
|
||
|
||
Operations for different tiles may overlap when engine resources permit.
|
||
|
||
Allowed overlaps:
|
||
|
||
```
|
||
DMA_READ(t+1) ∥ COMPUTE(t)
|
||
DMA_WRITE(t−1) ∥ COMPUTE(t)
|
||
DMA_READ(t) ∥ DMA_WRITE(t)
|
||
```
|
||
|
||
Disallowed overlaps:
|
||
|
||
```
|
||
GEMM(t) ∥ GEMM(t′)
|
||
MATH(t) ∥ MATH(t′)
|
||
GEMM(t) ∥ MATH(t′)
|
||
```
|
||
|
||
---
|
||
|
||
### D4. Engine execution model (Phase 0 default)
|
||
|
||
Each engine behaves as a deterministic service resource.
|
||
|
||
**DMA engine**
|
||
|
||
PE_DMA contains two independent channels.
|
||
|
||
```
|
||
DMA_READ capacity = 1
|
||
DMA_WRITE capacity = 1
|
||
```
|
||
|
||
Rules:
|
||
|
||
- DMA_READ and DMA_WRITE may execute concurrently.
|
||
- Multiple READs cannot overlap.
|
||
- Multiple WRITEs cannot overlap.
|
||
|
||
Example allowed:
|
||
|
||
```
|
||
DMA_READ(t+1) ∥ DMA_WRITE(t)
|
||
```
|
||
|
||
Example not allowed:
|
||
|
||
```
|
||
DMA_READ(t) ∥ DMA_READ(t+1)
|
||
DMA_WRITE(t) ∥ DMA_WRITE(t+1)
|
||
```
|
||
|
||
**Compute engine**
|
||
|
||
Compute operations share a single compute resource.
|
||
|
||
```
|
||
PE_ACCEL capacity = 1
|
||
```
|
||
|
||
Both GEMM and MATH require this shared compute slot.
|
||
|
||
Consequences:
|
||
|
||
- GEMM ∥ GEMM not allowed
|
||
- MATH ∥ MATH not allowed
|
||
- GEMM ∥ MATH not allowed
|
||
|
||
Only one compute operation can run in a PE at a time.
|
||
|
||
**Compute opcode restriction**
|
||
|
||
Composite commands contain one compute opcode only.
|
||
|
||
Examples:
|
||
|
||
```
|
||
COMPOSITE_GEMM
|
||
COMPOSITE_MATH
|
||
```
|
||
|
||
Mixed compute pipelines such as `GEMM → MATH` are not supported in Phase 0.
|
||
|
||
**Engine completion signaling**
|
||
|
||
Every engine emits a completion event when a sub-command finishes.
|
||
Completion events are delivered to PE_SCHEDULER.
|
||
|
||
---
|
||
|
||
### D5. Dataflow model
|
||
|
||
Compute operations use a TCM-centric dataflow model.
|
||
|
||
**Input path (HBM)**
|
||
|
||
```
|
||
HBM → XBAR → PE_DMA (DMA_READ) → PE_TCM
|
||
```
|
||
|
||
**Input path (shared SRAM)**
|
||
|
||
```
|
||
Shared SRAM → NOC → PE_DMA (DMA_READ) → PE_TCM
|
||
```
|
||
|
||
**Compute stage**
|
||
|
||
Compute engines read input tensors from PE_TCM.
|
||
|
||
```
|
||
PE_TCM → GEMM / MATH
|
||
```
|
||
|
||
Weights for GEMM may optionally stream directly from HBM (via XBAR).
|
||
|
||
**Output path (HBM)**
|
||
|
||
Compute results are written to PE_TCM, then DMA writes to HBM.
|
||
|
||
```
|
||
PE_TCM → PE_DMA (DMA_WRITE) → XBAR → HBM
|
||
```
|
||
|
||
**Output path (shared SRAM)**
|
||
|
||
```
|
||
PE_TCM → PE_DMA (DMA_WRITE) → NOC → Shared SRAM
|
||
```
|
||
|
||
#### D5.1 PE_TCM partitioning and ownership boundary
|
||
|
||
The PE_TCM address space is partitioned into two logical regions.
|
||
|
||
**SchedulerReservedTCM**
|
||
|
||
- A staging region owned exclusively by PE_SCHEDULER.
|
||
- This region is used for composite command tile buffers.
|
||
- PE_SCHEDULER:
|
||
- partitions this region into tile buffers
|
||
- assigns buffers for DMA_READ, COMPUTE, and DMA_WRITE stages
|
||
- guarantees input/output buffer separation
|
||
- manages tile buffer lifetime
|
||
|
||
**AllocatableTCM**
|
||
|
||
- General-purpose region managed by PEMemAllocator.
|
||
- Used by host or DP-visible allocations.
|
||
|
||
**Visibility rule (hard isolation)**
|
||
|
||
- PEMemAllocator must not see or allocate memory inside SchedulerReservedTCM.
|
||
- SchedulerReservedTCM is excluded from allocator-managed ranges by construction.
|
||
- This prevents DP or host allocations from interfering with scheduler staging buffers.
|
||
|
||
**Tile buffer rules**
|
||
|
||
Within SchedulerReservedTCM:
|
||
|
||
- input buffers and output buffers must not overlap
|
||
- PE_SCHEDULER assigns tile buffers for DMA and compute stages
|
||
- tile buffers remain valid until the corresponding DMA_WRITE completes
|
||
- Buffer reuse is allowed only after the tile lifetime finishes.
|
||
|
||
---
|
||
|
||
### D6. Observability and trace contract
|
||
|
||
The simulator must emit deterministic trace events.
|
||
|
||
Required events include:
|
||
|
||
- `command_submitted`
|
||
- `sub_command_dispatched`
|
||
- `engine_start`
|
||
- `engine_complete`
|
||
- `tile_ready`
|
||
- `command_complete`
|
||
|
||
Trace ordering must be deterministic for identical inputs.
|
||
|
||
---
|
||
|
||
### D7. Topology representation
|
||
|
||
PE internal components are declared in `cube.pe_template`.
|
||
|
||
The template is instantiated once per PE.
|
||
|
||
PE instances are derived from `cube.pe_layout`.
|
||
|
||
External connectivity such as:
|
||
|
||
- PE_DMA → XBAR (HBM data path)
|
||
- PE_DMA → NOC (non-HBM data path: shared SRAM, inter-cube UCIe)
|
||
- NOC → PE_CPU (command path from M_CPU)
|
||
|
||
is modeled at the CUBE level (see ADR-0003 D3).
|
||
|
||
---
|
||
|
||
## Links
|
||
|
||
- SPEC R3, R4
|
||
- ADR-0003 D4 (PE-level system hierarchy)
|
||
- ADR-0005 View C (PE-level diagram)
|
||
- ADR-0008 D2 (PA-level allocation at PE scope; PEMemAllocator is the per-PE allocator instance)
|
||
- ADR-0009 D3 (kernel execution fan-out and PE_CPU dispatch)
|