eval: commit milestone bench output (track generated figures + results)

Per request, the milestone bench output is now tracked in git instead of
gitignored, so the figures/results are viewable on the remote:

- src/kernbench/benches/1H_milestone_output/gemm/  (3 PNGs + gemm_sweep.json)
- src/kernbench/benches/1H_milestone_output/ccl/   (3 per-topology PNGs,
  buffer-kind PNG+CSV, FSIM comparison PNG, topology.png, summary.csv)

Drop the .gitignore rule; update ADR-0054 D3 + Negative (EN+KO) to say the
output is committed (regenerable by rerunning the bench). Artifacts produced
by full bench runs (milestone-1h-gemm non-FAST, milestone-1h-ccl).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-22 15:37:27 -07:00
parent cc1bbd0ab7
commit b1d6fafd3a
15 changed files with 1695 additions and 9 deletions
@@ -59,8 +59,8 @@ ADR-0045 D5는 bench를 단일 구성(single-SIP, 또는 ADR-0024 multi-SIP CCL
쓴다(사용자 요청 — bench 옆 아티팩트). 디렉터리는 생성된 PNG/CSV/JSON만
보유하며(`.py`/`__init__.py` 없음), 따라서 eager-import audit(ADR-0045
첫 동작)이 무시한다 — `pkgutil.iter_modules`는 비-패키지 하위 디렉터리를
yield하지 않는다. committed `docs/diagrams/` 아티팩트와 달리
**git-ignore**된다(요청 시 재생성 가능).
yield하지 않는다. `docs/diagrams/` 아티팩트처럼 **커밋된다**(원격에서
figure를 볼 수 있도록); bench 재실행 시 제자리에서 재생성된다.
### D4. GEMM 무거운 sweep — 기본은 fresh, `MILESTONE_FAST`로 재사용
@@ -115,7 +115,8 @@ bench 실행이 곧 재생성이다. slow 경로는 `@pytest.mark.slow` bench
드로잉을 섞는다). 대부분 평가 하니스인 "bench"는 이례적이며, 본 ADR이
이를 정당화한다.
- 생성 아티팩트가 명시적 요청에 의해 source tree(`src/kernbench/benches/`)
안에 산다; 커밋을 피하려 git-ignore.
안에 살며 커밋된다(원격에서 figure를 볼 수 있도록); bench 재실행 시
재생성된다.
- `milestone-1h-ccl`(및 기본 `milestone-1h-gemm`)은 분 단위 소요 —
on-demand 마일스톤 아티팩트에는 수용 가능, 일상 실행에는 아님.
+5 -3
View File
@@ -61,8 +61,9 @@ Both benches write to `src/kernbench/benches/1H_milestone_output/{gemm,ccl}/`
(per user request — artifacts beside the bench). The directory holds only
generated PNG/CSV/JSON (never a `.py`/`__init__.py`), so the eager-import
audit (ADR-0045 first action) ignores it — `pkgutil.iter_modules` does not
yield non-package subdirectories. It is **git-ignored** (regenerable on
demand), unlike the committed `docs/diagrams/` artifacts.
yield non-package subdirectories. It is **committed** (like the
`docs/diagrams/` artifacts) so the figures are viewable on the remote;
rerunning the bench regenerates it in place.
### D4. GEMM heavy sweep — fresh by default, `MILESTONE_FAST` to reuse
@@ -118,7 +119,8 @@ ADR-0045 D1).
sweeps, and matplotlib drawing). A "bench" that is mostly an eval harness
is unusual; this ADR legitimizes it.
- Generated artifacts live inside the source tree (`src/kernbench/benches/`)
by explicit request; git-ignored to avoid committing them.
by explicit request and are committed (so the figures are viewable on the
remote); rerunning the bench regenerates them.
- `milestone-1h-ccl` (and the default `milestone-1h-gemm`) take minutes —
acceptable for an on-demand milestone artifact, not for routine runs.