kernbench2/docs/adr/ADR-0010-api-cli-surface-and-semantics.md

# ADR-0010: Command Line Interface and Execution Semantics

## Status

Accepted

## Context

The `kernbench` CLI is the user-facing entry point of the simulator. It
exposes four subcommands:

- `run` — execute a benchmark against a topology.
- `list` — enumerate registered benches.
- `probe` — diagnostic utility for latency / BW measurement.
- `web` — interactive topology viewer.

Device enumeration is centralized in the CLI; neither the runtime API
nor the simulation engine enumerates devices. Benchmarks remain
single-device by design and accept a device identifier as input.

## Decision

### D1. Benchmark contract — single-device by design

- A benchmark MUST define behavior for a single device only.
- A benchmark MUST accept a device identifier as input.
- Benchmarks MUST NOT enumerate or loop over multiple devices.

Multi-device execution is the CLI's concern (D3), not the benchmark's.

### D2. `kernbench run` — benchmark execution

Required arguments:

- `--topology <path>`: topology YAML file path. Loaded via
  `resolve_topology()`.
- `--bench <identifier>`: benchmark identifier. Resolved via
  `kernbench.benches.registry.resolve()`, which accepts either the
  registered kebab-case name (e.g., `gemm-single-pe`) or a numeric
  index from `kernbench list`.

Optional arguments:

- `--device <selector>` (default: `all`):
  - `all` — run once per discovered SIP (see D3).
  - `sip:<N>` — run only on SIP N.
  - Parsed via `resolve_device()`.
- `--verify-data` (default: off) — enable Phase 2 data verification
  (see ADR-0020). When set, `engine_factory` constructs the engine
  with `enable_data=True`. After the benchmark runs, a diagnostic
  summary of recorded ops is printed.

Each invocation runs the benchmark once within a single simulation
instance.

### D3. Multi-device execution is logically parallel

When `--device all` (or omitted) and the topology has multiple SIPs:

- Benchmark executions are submitted to a single simulation engine
  instance.
- Executions are logically parallel in simulation time.
- Inter-device contention is naturally modeled (shared fabric
  bandwidth, cross-SIP traffic, etc.).

The CLI does NOT spawn multiple OS processes or independent
simulation runs — parallelism is internal to one simulation instance.

### D4. `kernbench list` — enumerate registered benches

No arguments. Prints each registered bench's auto-assigned index,
registered name, and one-line description.

Benches register themselves via the `@bench(name=..., description=...)`
decorator (`kernbench.benches.registry`). Every non-underscore module
under `kernbench.benches/` MUST register at least one bench; a missing
decorator raises `RuntimeError` at package import time.

Indices are assigned alphabetically by name at import time. They are a
CLI convenience (shorthand for `--bench`), not a stable API — a new
bench inserted alphabetically will shift later indices.

### D5. `kernbench probe` — latency / BW diagnostic utility

Required argument:

- `--topology <path>`: topology YAML file path.

Optional argument:

- `--case <name>` (default: `all`) — run a predefined traffic
  pattern, or `all` to run every defined case.

Probe runs each pattern through the simulation engine and reports
per case:

- End-to-end latency (ns).
- Effective bandwidth (nbytes / total_ns).
- Bottleneck bandwidth (min edge BW along the chosen path).
- Utilization (effective / bottleneck).

Probe additionally validates monotonicity invariants — for example
that local-HBM access ≤ cross-PE-within-cube ≤ cross-cube ≤
cross-SIP — and reports violations. Probe is a developer tool for
verifying the latency / BW model; it is not a benchmark.

### D6. `kernbench web` — topology viewer

Optional arguments:

- `--port <N>` (default: `8765`) — HTTP port.
- `--no-open` — do not auto-open the browser.

Launches a local HTTP server that renders the compiled topology in
the browser. Distinct from the static `docs/diagrams/` artifacts:

- `docs/diagrams/` files are derived at topology-compile time
  (ADR-0006).
- `kernbench web` is interactive — pan/zoom, hover for component
  attributes, switch between SIP / CUBE / PE views.

### D7. Runtime API and simulation engine remain device-scoped

- Runtime API calls operate on one device per invocation.
- The simulation engine schedules all requests deterministically.
- Neither layer enumerates devices.

This invariant keeps each layer testable in isolation; device
enumeration and multi-device fan-out live only in the CLI's `run`
command (D3).

The `probe` implementation lives under `kernbench.probes` (separate
from `kernbench.benches`), reflecting that probes are diagnostic
utilities, not registered benches.

## Consequences

- Benchmark authors write single-device logic; multi-device behavior
  emerges from the CLI dispatching across SIPs.
- Adding a new subcommand (e.g., trace export, replay) does not
  require benchmark or runtime-API changes — the CLI is the
  extension point.
- `probe` and `web` are diagnostic / visualization tools, not
  benchmarks; they bypass the benchmark loader path.

## Links

- SPEC R7, R8, R9
- ADR-0007 (Runtime API and Simulation Engine Boundaries)
- ADR-0020 (Two-pass data execution — `--verify-data`)
- ADR-0006 (Topology compilation and diagram generation —
  background for `kernbench web`)