Add PE-level IPCQ collective infra + unified ccl_allreduce bench (ADR-0023)

Major changes:

PE-level IPCQ infrastructure:
- New PE_IPCQ component: ring-buffer control plane with 4-direction
  neighbor mapping, head/tail pointers, backpressure (poll/sleep).
- PE_DMA extended with vc_comm channel for IPCQ outbound/inbound DMA,
  including in-flight data snapshot (D9) and op_log recording at
  outbound time for Phase 2 replay correctness.
- IpcqDmaToken piggyback model: data + metadata travel together,
  atomic visibility at receiver (invariant I6).
- Credit return fast path: bottleneck-BW latency, no fabric vc_comm.

Phase 2 data execution (ADR-0020 integration):
- op_log extended: DmaWriteCmd now captures src_space/src_addr for
  Phase 2 dma_write copy; ipcq_copy ops recorded at outbound time.
- DataExecutor replays dma_write + ipcq_copy in t_start order.
- Engine._flush_data_phase: incremental cursor-based replay after
  each engine.wait() so host reads see post-Phase-2 data.
- KernelRunner Phase 1 writes disabled when op_log is active to
  prevent stale data from corrupting the MemoryStore snapshot.

TLContext / kernel API:
- tl.send(dir, src=TensorHandle), tl.recv(dir, shape, dtype),
  tl.recv_async, tl.wait(RecvFuture), copy_to_dst mode.
- TensorHandle operator overloading (add/sub/mul/div) via thread-local
  active TLContext → MathCmd dispatch through PE_MATH.
- PE-local scratch allocator for math output handles.
- tl.load returns space="hbm" handles for correct Phase 2 addressing.
- Additional math functions: maximum, minimum, fma, clamp, softmax, cdiv.

Unified ccl_allreduce bench (PyTorch-compat host code):
- Single benches/ccl_allreduce.py with run() + worker(rank, ws, torch)
  split matching real PyTorch DDP worker pattern.
- torch.distributed facade: init_process_group, get_world_size,
  get_rank, get_backend, all_reduce, barrier — only real PyTorch names.
- AhbmCCLBackend: eager install_ipcq at init, all_reduce dispatches
  kernel via tensor shard metadata (n_elem from shards[0].nbytes).
- world_size derived from topology spec (sips × cubes × pes_per_cube)
  with optional algorithm-level override in ccl.yaml.

Tensor API (PyTorch-compat surface):
- Tensor.numpy(): gather-aware (all shards via VA-based addressing).
- Tensor.copy_(source): scatter from host tensor into sharded target.
- RuntimeContext.from_numpy(arr): host-side staging tensor.
- Tensor.data property fixed to use numpy() (was shards[0]-only).

Algorithm modules moved to src/kernbench/ccl/algorithms/:
- ring_allreduce, mesh_allreduce, tree_allreduce, hello_send.
- Each module exports kernel_args(world_size, n_elem) helper.
- ccl.yaml module paths updated to kernbench.ccl.algorithms.*.

Dead code removed:
- 7 per-variant bench files (ccl_allreduce_{tcm,hbm,sram}, etc.).
- _run_ccl_bench greenlet-per-SIP scheduler.
- benches.loader.is_ccl_bench + run_rank detection.
- benches/ccl/ directory.

Tests:
- New test_ccl_allreduce_matrix.py: 7 parametrized cases
  (ring×3 buffers, ring 8/16, mesh 4, tree 7).
- New test_runtime_api_tensor.py: copy_/numpy/from_numpy unit tests.
- Existing tests updated for new import paths + world_size_override.

Docs:
- Korean ccl-author-guide.md and ADR-0023 paths updated.
- New English versions: ccl-author-guide.en.md, ADR-0023.en.md.

502 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

tests/test_pe_ipcq.py

@@ -0,0 +1,317 @@
+"""Tests for PE_IPCQ component (ADR-0023 D1, D2, D9, D14).
+
+These tests use a mock setup: PeIpcqComponent is instantiated directly,
+its in_ports/out_ports are wired to plain SimPy Stores, and IpcqInitMsg
+is delivered via a simple dummy transaction wrapper. PE_DMA is mocked
+as a Store that we drain manually.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any
+
+import pytest
+import simpy
+
+from kernbench.common.ipcq_types import (
+    IpcqCreditMetadata,
+    IpcqDmaToken,
+    IpcqEndpoint,
+    IpcqInitEntry,
+    IpcqInvalidDirection,
+    IpcqMetaArrival,
+    IpcqRecvCmd,
+    IpcqRequest,
+    IpcqSendCmd,
+)
+from kernbench.components.builtin.pe_ipcq import PeIpcqComponent
+from kernbench.runtime_api.kernel import IpcqInitMsg
+from kernbench.topology.types import Node
+
+
+# ── Fakes / fixtures ─────────────────────────────────────────────────
+
+
+@dataclass
+class _FakeTxn:
+    request: Any
+    done: simpy.Event
+    result_data: dict[str, Any] = field(default_factory=dict)
+
+
+def _make_pe_ipcq(env: simpy.Environment, pe_prefix: str = "sip0.cube0.pe0") -> PeIpcqComponent:
+    """Create a PeIpcqComponent with mocked ports.
+
+    Returns the component with:
+      - in_ports["host"] for posting IpcqInitMsg / IpcqRequest
+      - out_ports["__pe_dma__"] for outgoing IpcqDmaToken (drain manually)
+      - The component is started.
+    """
+    node = Node(
+        id=f"{pe_prefix}.pe_ipcq",
+        kind="pe_ipcq",
+        impl="builtin.pe_ipcq",
+        attrs={},
+        pos_mm=None,
+    )
+    comp = PeIpcqComponent(node, ctx=None)
+    comp.in_ports["host"] = simpy.Store(env)
+    comp.out_ports[f"{pe_prefix}.pe_dma"] = simpy.Store(env)
+    comp.start(env)
+    return comp
+
+
+def _install_two_neighbors(env: simpy.Environment, comp: PeIpcqComponent) -> tuple[simpy.Store, simpy.Store]:
+    """Install E and W neighbor entries with peer_credit_stores.
+
+    Returns (peer_e_credit_store, peer_w_credit_store) — i.e. the stores
+    that the component will put credits into when it receives data.
+    """
+    peer_e_credit = simpy.Store(env)
+    peer_w_credit = simpy.Store(env)
+
+    ep_e = IpcqEndpoint(
+        sip=0, cube=0, pe=1,
+        buffer_kind="tcm",
+        rx_base_pa=0x10_000, rx_base_va=0,
+        n_slots=4, slot_size=4096,
+    )
+    ep_w = IpcqEndpoint(
+        sip=0, cube=0, pe=2,
+        buffer_kind="tcm",
+        rx_base_pa=0x20_000, rx_base_va=0,
+        n_slots=4, slot_size=4096,
+    )
+    init_msg = IpcqInitMsg(
+        correlation_id="t", request_id="t",
+        target_sips=(0,), target_cubes=(0,), target_pe=0,
+        entries=(
+            IpcqInitEntry(
+                direction="E", peer=ep_e,
+                my_rx_base_pa=0x30_000, my_rx_base_va=0,
+                n_slots=4, slot_size=4096,
+                peer_credit_store=peer_e_credit,
+            ),
+            IpcqInitEntry(
+                direction="W", peer=ep_w,
+                my_rx_base_pa=0x40_000, my_rx_base_va=0,
+                n_slots=4, slot_size=4096,
+                peer_credit_store=peer_w_credit,
+            ),
+        ),
+        backpressure_mode="sleep",
+        buffer_kind="tcm",
+        credit_size_bytes=16,
+    )
+    done = env.event()
+    comp.in_ports["host"].put(_FakeTxn(request=init_msg, done=done))
+    env.run(until=done)
+    return peer_e_credit, peer_w_credit
+
+
+# ── send: forward token to PE_DMA ────────────────────────────────────
+
+
+def test_send_forwards_token_to_pe_dma():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+    pe_dma = comp.out_ports["sip0.cube0.pe0.pe_dma"]
+
+    cmd = IpcqSendCmd(
+        direction="E", src_addr=0x500, src_space="tcm",
+        nbytes=128, shape=(8, 8), dtype="f16", handle_id="s1",
+    )
+    done = env.event()
+    comp.in_ports["host"].put(IpcqRequest(command=cmd, done=done))
+    env.run(until=done)
+
+    # Token should be in PE_DMA's mock store
+    assert len(pe_dma.items) == 1
+    token = pe_dma.items[0]
+    assert isinstance(token, IpcqDmaToken)
+    assert token.dst_addr == 0x10_000  # peer.rx_base_pa + 0
+    assert token.nbytes == 128
+    assert token.sender_seq == 0
+    assert token.src_direction == "E"
+
+
+def test_send_advances_my_head_and_slot_addresses():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+    pe_dma = comp.out_ports["sip0.cube0.pe0.pe_dma"]
+
+    for i in range(3):
+        cmd = IpcqSendCmd(
+            direction="E", src_addr=0x500 + i,
+            src_space="tcm", nbytes=64,
+            shape=(8,), dtype="f16", handle_id=f"s{i}",
+        )
+        done = env.event()
+        comp.in_ports["host"].put(IpcqRequest(command=cmd, done=done))
+        env.run(until=done)
+
+    tokens = pe_dma.items
+    assert [t.sender_seq for t in tokens] == [0, 1, 2]
+    # slot addresses: peer.rx_base_pa (0x10_000) + i * slot_size (4096)
+    assert [t.dst_addr for t in tokens] == [0x10_000, 0x11_000, 0x12_000]
+
+
+def test_send_invalid_direction_raises():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+
+    cmd = IpcqSendCmd(
+        direction="N", src_addr=0x100, src_space="tcm",
+        nbytes=64, shape=(8,), dtype="f16", handle_id="s_bad",
+    )
+    done = env.event()
+    comp.in_ports["host"].put(IpcqRequest(command=cmd, done=done))
+
+    with pytest.raises(IpcqInvalidDirection):
+        env.run(until=done)
+
+
+# ── recv: wait for data and return slot address ─────────────────────
+
+
+def test_recv_waits_until_metadata_arrives():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+
+    recv_cmd = IpcqRecvCmd(
+        direction="W", shape=(8,), dtype="f16", handle_id="r1",
+    )
+    recv_req = IpcqRequest(command=recv_cmd, done=env.event())
+    comp.in_ports["host"].put(recv_req)
+
+    # Run a bit — recv should not complete yet (no data)
+    env.run(until=10)
+    assert not recv_req.done.triggered
+
+    # Simulate metadata arrival from peer (W direction = sender pe=2)
+    fake_token = IpcqDmaToken(
+        src_addr=0, src_space="tcm",
+        dst_addr=0x40_000, dst_endpoint=comp._queue_pairs["W"]["peer"],
+        nbytes=64, handle_id="x",
+        shape=(8,), dtype="f16",
+        sender_seq=0,
+        src_sip=0, src_cube=0, src_pe=2, src_direction="E",
+    )
+    comp.in_ports["host"].put(IpcqMetaArrival(token=fake_token))
+    env.run(until=recv_req.done)
+
+    assert recv_req.result_data["src_addr"] == 0x40_000  # my_rx_base_pa for W
+    assert recv_req.result_data["direction"] == "W"
+
+
+def test_recv_returns_immediately_if_data_already_present():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+
+    # Pre-arrive metadata
+    fake_token = IpcqDmaToken(
+        src_addr=0, src_space="tcm",
+        dst_addr=0x40_000, dst_endpoint=comp._queue_pairs["W"]["peer"],
+        nbytes=64, handle_id="x",
+        shape=(8,), dtype="f16",
+        sender_seq=0,
+        src_sip=0, src_cube=0, src_pe=2, src_direction="E",
+    )
+    comp.in_ports["host"].put(IpcqMetaArrival(token=fake_token))
+    env.run(until=5)
+
+    recv_cmd = IpcqRecvCmd(
+        direction="W", shape=(8,), dtype="f16", handle_id="r1",
+    )
+    recv_req = IpcqRequest(command=recv_cmd, done=env.event())
+    comp.in_ports["host"].put(recv_req)
+    env.run(until=recv_req.done)
+
+    assert recv_req.result_data["src_addr"] == 0x40_000
+
+
+def test_recv_round_robin_picks_arrived_direction():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+
+    # Pre-arrive metadata only on W direction
+    fake_token = IpcqDmaToken(
+        src_addr=0, src_space="tcm",
+        dst_addr=0x40_000, dst_endpoint=comp._queue_pairs["W"]["peer"],
+        nbytes=64, handle_id="x",
+        shape=(8,), dtype="f16",
+        sender_seq=0,
+        src_sip=0, src_cube=0, src_pe=2, src_direction="E",
+    )
+    comp.in_ports["host"].put(IpcqMetaArrival(token=fake_token))
+    env.run(until=5)
+
+    # recv() with no direction → round-robin
+    recv_cmd = IpcqRecvCmd(
+        direction=None, shape=(8,), dtype="f16", handle_id="r_rr",
+    )
+    recv_req = IpcqRequest(command=recv_cmd, done=env.event())
+    comp.in_ports["host"].put(recv_req)
+    env.run(until=recv_req.done)
+
+    assert recv_req.result_data["direction"] == "W"
+
+
+# ── backpressure: send blocks when full ──────────────────────────────
+
+
+def test_send_blocks_when_peer_slot_full():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+
+    # n_slots = 4, so 4 sends should succeed; 5th blocks
+    for i in range(4):
+        cmd = IpcqSendCmd(
+            direction="E", src_addr=0x500, src_space="tcm",
+            nbytes=64, shape=(8,), dtype="f16", handle_id=f"s{i}",
+        )
+        done = env.event()
+        comp.in_ports["host"].put(IpcqRequest(command=cmd, done=done))
+        env.run(until=done)
+
+    # 5th send: should not complete
+    cmd5 = IpcqSendCmd(
+        direction="E", src_addr=0x500, src_space="tcm",
+        nbytes=64, shape=(8,), dtype="f16", handle_id="s5",
+    )
+    req5 = IpcqRequest(command=cmd5, done=env.event())
+    comp.in_ports["host"].put(req5)
+    env.run(until=20)
+    assert not req5.done.triggered
+
+    # Send a credit return: peer (E direction, pe=1) consumed slot 0
+    credit = IpcqCreditMetadata(
+        consumer_seq=1,  # peer consumed up to my_tail=1
+        src_sip=0, src_cube=0, src_pe=1, src_direction="W",  # peer's view
+    )
+    comp.credit_inbox.put(credit)
+    env.run(until=req5.done)
+    assert req5.done.triggered
+
+
+# ── Init test ────────────────────────────────────────────────────────
+
+
+def test_init_installs_neighbors():
+    env = simpy.Environment()
+    comp = _make_pe_ipcq(env)
+    _install_two_neighbors(env, comp)
+
+    assert "E" in comp._queue_pairs
+    assert "W" in comp._queue_pairs
+    assert comp._queue_pairs["E"]["peer"].pe == 1
+    assert comp._queue_pairs["W"]["peer"].pe == 2
+    assert comp._queue_pairs["E"]["my_head"] == 0
+    assert comp._queue_pairs["E"]["peer_tail_cache"] == 0