ywkang/kernbench2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ADR: bilingual structure — EN canonical in adr/, KO mirror in adr-ko/

Browse Source 
Establish English as the canonical ADR language with Korean translations
held in a parallel docs/adr-ko/ tree as derived artifacts (1:1 mirror).
Promotion from adr-proposed/ to adr/ now writes English to adr/ and the
Korean to adr-ko/; bidirectional sync rule documented in CLAUDE.md.

- Migrate 30 ADRs in docs/adr/: 28 Korean-only translated to English,
  2 bilingual pairs (ADR-0020, ADR-0023) consolidated (.en.md suffix
  dropped). ADR-0023 EN regenerated against KO source which had newer
  HW Realization Notes (D16-D23) section.
- docs/adr-history/ left frozen by design (transitional state).
- CLAUDE.md (Part 2): update ADR Lifecycle for 4-folder layout, mark
  docs/adr-ko/ as a Derived Artifact, add ADR Translation Discipline
  section covering bidirectional sync, conflict resolution (EN wins),
  and proposed-language freedom.
- tools/verify_adr_lang_pairs.py: new verification tool checking pair
  completeness, filename mirroring, ADR-ID match, Status byte-equality.
  Pre-commit hook intentionally not added; run on demand or in CI.
- tests/test_verify_adr_lang_pairs.py: 11 cases including CRLF/LF
  normalization, em-dash title separator, underscore-slug edge case.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Yangwook
2026-05-20 01:38:44 -07:00
parent 687c98086d
commit a796c1d2f7
42 changed files with 10515 additions and 3422 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/adr/ADR-0024-par-sip-tp-launcher.md
+59 -52
View File
@@ -6,43 +6,46 @@ Accepted
## Context
### 목표
### Goal
`torch.distributed` collective 호출의 참여 단위(rank)를 **SIP**(device)
경계에 맞춘다. 실제 PyTorch DDP/TP 스크립트와 **호스트 레벨에서 구분 없이**
읽히는 bench 코드를 목표로 한다.
Align the participation unit (rank) of `torch.distributed` collective calls
to the **SIP** (device) boundary. The aim is bench code that, at the host
level, reads **indistinguishably** from real PyTorch DDP/TP scripts.
real PyTorch와 비교:
Comparison with real PyTorch:
| 차원 | real PyTorch | KernBench |
| Dimension | real PyTorch | KernBench |
| --- | --- | --- |
| 프로세스 모델 | N개 프로세스, 각 1 GPU | 1 프로세스, N greenlet, 각 1 SIP |
| `get_rank()` | `RANK` env var | greenlet-local 레지스트리 |
| `get_world_size()` | `WORLD_SIZE` env var | topology의 SIP 수 |
| Process model | N processes, 1 GPU each | 1 process, N greenlets, 1 SIP each |
| `get_rank()` | `RANK` env var | greenlet-local registry |
| `get_world_size()` | `WORLD_SIZE` env var | SIP count from topology |
| `torch.cuda.set_device(r)` (real) / `torch.ahbm.set_device(r)` (KernBench) | rank → GPU | rank → SIP |
| `mp.spawn` | OS 프로세스 fork | greenlet fan-out |
| `mp.spawn` | OS process fork | greenlet fan-out |
### 풀어야 할 문제
### Problems to solve
1. **공개 API에서 rank = SIP** — bench worker가 PE 개념을 알지 않도록.
2. **Greenlet-local rank/device tracking** — 1-프로세스 모델 안에서 각
worker greenlet이 자기 rank / 자기 SIP를 정확히 식별.
3. **Tensor placement = structural (sip, cube, pe)** — rank가 SIP이면
기본 텐서 배치도 구조적 좌표로 표현되어야 함.
1. **Public API where rank = SIP** — so bench workers do not have to know
about the PE concept.
2. **Greenlet-local rank/device tracking** — within the 1-process model,
each worker greenlet must correctly identify its own rank / its own SIP.
3. **Tensor placement = structural (sip, cube, pe)** — if rank is SIP,
the default tensor placement should also be expressed in structural
coordinates.
### Non-problem (이 ADR 밖)
### Non-problem (outside this ADR)
- IPCQ direction addressing → ADR-0025
- `DPPolicy.sip`/`num_sips` 제거 → ADR-0026
- Removing `DPPolicy.sip`/`num_sips` → ADR-0026
- Megatron-style TP → ADR-0027
- DTensor → ADR-0028 (future)
- Worker scheduling / `mp.spawn` / collective drain / exception cleanup
→ ADR-0027 D0/D1
- Collective algorithm 구현 (intercube_allreduce, SFR config) → ADR-0032
- Collective algorithm implementation (intercube_allreduce, SFR config)
→ ADR-0032
## Decision
### D1. rank = SIP (world_size 해석)
### D1. rank = SIP (world_size resolution)
```python
def _resolve_world_size(self) -> int:
@@ -55,8 +58,8 @@ def _resolve_world_size(self) -> int:
return int(spec.get("system", {}).get("sips", {}).get("count", 1))
```
우선순위: 알고리즘 override > defaults override > SIP count. `ccl.yaml`
override는 legacy "rank = PE" 테스트 경로로 유지.
Priority order: algorithm override > defaults override > SIP count. The
`ccl.yaml` override is retained as the legacy "rank = PE" test path.
### D2. Greenlet-local rank registry (+ debug warning)
@@ -83,11 +86,11 @@ class DistributedContext:
return int(self._rank_by_greenlet[g])
```
### D3. `torch.ahbm.set_device(rank)` — SIP 바인딩
### D3. `torch.ahbm.set_device(rank)` — SIP binding
KernBench 백엔드 이름은 `ahbm` (ADR-0023). Real PyTorch는
`torch.cuda.set_device(r)`이지만 우리는 CUDA가 아니므로 honestly-named
namespace를 사용한다.
The KernBench backend name is `ahbm` (ADR-0023). Real PyTorch uses
`torch.cuda.set_device(r)`, but since we are not CUDA we use an
honestly-named namespace.
```python
class _AhbmNamespace:
@@ -113,10 +116,12 @@ class _AhbmNamespace:
# Bench code: `torch.ahbm.set_device(rank)` mirrors `torch.cuda.set_device`.
```
**PyTorch 2.x style 병행 지원**: 최신 PyTorch는 device-agnostic한
`torch.accelerator` 네임스페이스를 지향 (`torch.accelerator.set_device_index(r)`,
`torch.accelerator.current_device_index()`). Device vendor에 종속되지 않는
코드를 쓰려는 사용자를 위해 KernBench도 이 표면을 병행 지원한다.
**PyTorch 2.x style parallel support**: Recent PyTorch is moving toward a
device-agnostic `torch.accelerator` namespace
(`torch.accelerator.set_device_index(r)`,
`torch.accelerator.current_device_index()`). To support users who want to
write code that is not tied to a specific device vendor, KernBench also
exposes this surface in parallel.
```python
class _AcceleratorNamespace:
@@ -141,23 +146,23 @@ self.ahbm = _AhbmNamespace()
self.accelerator = _AcceleratorNamespace(self.ahbm) # alias
```
Bench 작성자는 다음 중 하나를 선택 — 둘 다 내부적으로 같은 레지스트리를 보유:
Bench authors may choose either — both share the same registry internally:
```python
torch.ahbm.set_device(rank) # KernBench-native, explicit backend
torch.accelerator.set_device_index(rank) # PyTorch 2.x device-agnostic
```
### D4. Tensor placement = structural (sip, cube, pe) 좌표
### D4. Tensor placement = structural (sip, cube, pe) coordinates
`resolve_dp_policy`가 `target_sip`을 직접 받아 구조적 좌표로 placement 생성.
세부는 ADR-0026.
`resolve_dp_policy` takes `target_sip` directly and produces placement in
structural coordinates. Details in ADR-0026.
```python
# RuntimeContext._create_tensor
current_sip = self.ahbm.current_device() # (D3 naming)
if current_sip is None:
current_sip = 0 # single-driver fallback (D2와 일관)
current_sip = 0 # single-driver fallback (consistent with D2)
placement = resolve_dp_policy(
dp, shape=shape_2d, itemsize=itemsize,
num_pe=eff_num_pe, num_cubes=eff_num_cubes,
@@ -165,29 +170,29 @@ placement = resolve_dp_policy(
)
```
Post-hoc `pe_index` shifting 없음 — ShardSpec이 `(sip, cube, pe)` 구조적
좌표를 직접 보유. ShardSpec 상세는 ADR-0026.
No post-hoc `pe_index` shifting — ShardSpec carries the `(sip, cube, pe)`
structural coordinates directly. ShardSpec details in ADR-0026.
---
## Dependencies
- **ADR-0023** (IPCQ): backend `ahbm` namespace의 기원.
- **ADR-0026** (DPPolicy intra-device): D4의 `resolve_dp_policy` 시그니처와
ShardSpec의 구조적 좌표 표현.
- **ADR-0027** (Megatron TP + scheduler): worker scheduling, `mp.spawn`,
collective drain, exception cleanup의 구현 기준.
- **ADR-0023** (IPCQ): origin of the backend `ahbm` namespace.
- **ADR-0026** (DPPolicy intra-device): the `resolve_dp_policy` signature
used by D4 and the structural-coordinate representation of ShardSpec.
- **ADR-0027** (Megatron TP + scheduler): the implementation baseline for
worker scheduling, `mp.spawn`, collective drain, and exception cleanup.
---
## Non-goals
- **IPCQ protocol 수정**: ADR-0023 유지.
- **DPPolicy 필드 정리**: ADR-0026.
- **Modifying the IPCQ protocol**: ADR-0023 remains as-is.
- **Cleaning up DPPolicy fields**: ADR-0026.
- **Megatron-style TP**: ADR-0027.
- **Worker scheduling / spawn / drain / exception cleanup**: ADR-0027 D0/D1.
- **Collective algorithm 구현**: ADR-0032.
- **Multi-node (프로세스 간)**: 단일 프로세스.
- **Collective algorithm implementation**: ADR-0032.
- **Multi-node (cross-process)**: single process only.
---
@@ -195,12 +200,14 @@ Post-hoc `pe_index` shifting 없음 — ShardSpec이 `(sip, cube, pe)` 구조적
### Positive
- **Bench = real PyTorch DDP** (공개 API 관점).
- **Greenlet-local rank**: 1-프로세스 모델에서 cross-rank correctness 가능.
- **Structural placement 좌표**: ADR-0026 / ADR-0027 / ADR-0032의 다른 ADR이
`(sip, cube, pe)` 3튜플 위에서 일관되게 동작.
- **Bench = real PyTorch DDP** (from the public-API point of view).
- **Greenlet-local rank**: enables cross-rank correctness within the
1-process model.
- **Structural placement coordinates**: lets the other ADRs (ADR-0026 /
ADR-0027 / ADR-0032) operate consistently on top of the `(sip, cube, pe)`
3-tuple.
### Neutral
- IPCQ PE-level protocol (ADR-0023) 불변.
- IO_CPU 역할 불변 (기존 transit 그대로).
- IPCQ PE-level protocol (ADR-0023) is unchanged.
- IO_CPU role is unchanged (existing transit behavior preserved).