benches: package as kernbench.benches, add @bench registry + list subcommand

Move benches/ -> src/kernbench/benches/ and src/kernbench/cli/probe.py ->
src/kernbench/probes/probe.py. Each bench self-registers via
@bench(name=..., description=...); kernbench list enumerates benches
with auto-assigned indices, --bench accepts kebab-case name or numeric
index. Audit at package-import time fails if any non-underscore module
forgets the decorator. ADR-0010 (EN + KO) updated to reflect the new
resolver path, list subcommand, and probes package separation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-20 14:42:10 -07:00
parent 168b0c89f0
commit 049e3d8bb3
28 changed files with 398 additions and 79 deletions
@@ -7,9 +7,10 @@ Accepted
## Context
The `kernbench` CLI is the user-facing entry point of the simulator. It
exposes three subcommands:
exposes four subcommands:
- `run` — execute a benchmark against a topology.
- `list` — enumerate registered benches.
- `probe` — diagnostic utility for latency / BW measurement.
- `web` — interactive topology viewer.
@@ -33,8 +34,10 @@ Required arguments:
- `--topology <path>`: topology YAML file path. Loaded via
`resolve_topology()`.
- `--bench <name>`: benchmark name. Resolved via
`benches.loader.resolve_bench()`.
- `--bench <identifier>`: benchmark identifier. Resolved via
`kernbench.benches.registry.resolve()`, which accepts either the
registered kebab-case name (e.g., `gemm-single-pe`) or a numeric
index from `kernbench list`.
Optional arguments:
@@ -63,7 +66,21 @@ When `--device all` (or omitted) and the topology has multiple SIPs:
The CLI does NOT spawn multiple OS processes or independent
simulation runs — parallelism is internal to one simulation instance.
### D4. `kernbench probe` — latency / BW diagnostic utility
### D4. `kernbench list` — enumerate registered benches
No arguments. Prints each registered bench's auto-assigned index,
registered name, and one-line description.
Benches register themselves via the `@bench(name=..., description=...)`
decorator (`kernbench.benches.registry`). Every non-underscore module
under `kernbench.benches/` MUST register at least one bench; a missing
decorator raises `RuntimeError` at package import time.
Indices are assigned alphabetically by name at import time. They are a
CLI convenience (shorthand for `--bench`), not a stable API — a new
bench inserted alphabetically will shift later indices.
### D5. `kernbench probe` — latency / BW diagnostic utility
Required argument:
@@ -87,7 +104,7 @@ that local-HBM access ≤ cross-PE-within-cube ≤ cross-cube ≤
cross-SIP — and reports violations. Probe is a developer tool for
verifying the latency / BW model; it is not a benchmark.
### D5. `kernbench web` — topology viewer
### D6. `kernbench web` — topology viewer
Optional arguments:
@@ -102,7 +119,7 @@ the browser. Distinct from the static `docs/diagrams/` artifacts:
- `kernbench web` is interactive — pan/zoom, hover for component
attributes, switch between SIP / CUBE / PE views.
### D6. Runtime API and simulation engine remain device-scoped
### D7. Runtime API and simulation engine remain device-scoped
- Runtime API calls operate on one device per invocation.
- The simulation engine schedules all requests deterministically.
@@ -112,6 +129,10 @@ This invariant keeps each layer testable in isolation; device
enumeration and multi-device fan-out live only in the CLI's `run`
command (D3).
The `probe` implementation lives under `kernbench.probes` (separate
from `kernbench.benches`), reflecting that probes are diagnostic
utilities, not registered benches.
## Consequences
- Benchmark authors write single-device logic; multi-device behavior