ADR-0027: Megatron TP API + worker-wait generalization + mp.spawn

Implements ADR-0027 Phase 2 end-to-end. All 559 tests pass (was 523 +
1 xfail; ring_default_ws strict-xfail is now resolved).

D0 — Worker-wait generalization (context.py):
- _pending_worker_waits queue on RuntimeContext.
- ctx.wait(h) in worker context defers to main via g.parent.switch().
  Fast-path for already-completed handles.
- Worker API is unchanged: tensor deploy, launch, etc. still look
  synchronous; they're transparently cooperatively scheduled.
- Solves ADR-0024 Phase B kernel-greenlet orphan bug (env.run now
  only ever drives from main; kernel _parent is always main).

D0.5 — Host-read barrier (tensor.py):
- Explicit _HOST_READ_BARRIERS registry (T5.g closed-set via code
  review, not reflection-magic).
- numpy/data/__getitem__/__repr__ drain pending worker-waits before
  host-observable read.
- copy_: source-side barrier via source.numpy(). Target-side write
  barrier is intentionally NOT applied — global pending target barrier
  prematurely drains cross-rank collectives → deadlock.
- Collective pending is excluded from barrier drain condition
  (collective is cross-rank; its own yield in all_reduce covers the
  invariant naturally).

D1 — torch.multiprocessing.spawn (runtime_api/multiprocessing.py):
- API signature parity with real PyTorch spawn; execution is
  cooperative greenlet scheduler (process isolation etc. are explicit
  non-goals per D1.0).
- _drain_pending drains worker-waits then collectives in one barrier,
  loop-until-empty.
- Round-based exception handling with SystemExit sibling abort +
  SpawnException(errors) wrapping root-cause ranks.
- RuntimeContext attaches ctx.multiprocessing in __post_init__.
- benches/ccl_allreduce.py hand-rolled loop collapses to one
  torch.multiprocessing.spawn call.

D2–D6 — kernbench.tp package:
- parallel_state: initialize_model_parallel, get_*_rank,
  get_*_world_size, with weak active-ctx registry in context.py.
- layers: ColumnParallelLinear, RowParallelLinear (shape-only
  primitives — fp16 gemm via tl.load + tl.dot + tl.store).
- kernels: _gemm_kernel used by TP layers (self-contained; no bench
  dependency).
- primitives / mappings stubs per D6/D8.

Data-path fixes (surfaced by TP gemm + all_reduce sequence):
- sim_engine/op_log.py: dma_write snapshot is skipped for TCM
  sources (PE scratch is repopulated by Phase 2 math/gemm replay —
  capturing Phase-1-time snapshot picked up STALE data from prior
  kernel's output aliased at the same scratch addr, causing the later
  kernel's dma_write to overwrite Phase 2 result with stale value).
- sim_engine/op_log.py + sim_engine/data_executor.py: per-operand
  space recorded on GemmCmd and composite gemm records so HBM-resident
  operands (tl.load output) don't default to TCM during replay.
- runtime_api/context.py: ctx.zeros writes zero-init to MemoryStore
  at VA keys so kernels reading via VA see deterministic init even
  without explicit copy_().

Tests (Phase 1 + Phase 2):
- test_worker_wait_drain (T3): orphan invariant + resume + multi-rank
  drain + idempotency + exception propagation.
- test_mp_spawn (T4): spawn shape + bind + SpawnException scope.
- test_host_read_barrier (T5): barrier contract per entry-point +
  closed-set registry check.
- test_tp_parallel_state (T1): initialize + rank lookup.
- test_tp_layers (T2): shape + deterministic numerical correctness
  (concat-matmul equality for RowParallel, not mean-only).
- test_tp_mlp (T6): full 2-layer MLP with deterministic weight
  numerical match + rank-consistency post all-reduce.
- test_ccl_allreduce_matrix: ring_default_ws xfail removed (T7).

Regression: 523 pre + 35 new + 1 ex-xfail = 559 passed, 1 intentional
skip (T3.e historical failure documentation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

tests/test_ccl_allreduce_matrix.py

@@ -70,29 +70,14 @@ CASES = [
     # Default fallback — no world_size override → ADR-0024 D1 derives
     # from topology (SIP count = 2). Exercises the new SIP-level TP
     # launcher + cross-SIP ring.
-    # XFAIL — architectural blocker (ADR-0024 Phase B, future redesign):
-    # Bench workers call torch.zeros / copy_ which internally drive
-    # env.run in the WORKER-greenlet context. Any KernelLaunchMsg already
-    # pending in the SimPy queue gets stepped inside that worker context,
-    # which in turn spawns kernel_runner + kernel greenlet with parent =
-    # worker (not main). When the worker later yields / finishes, the
-    # kernel greenlet is orphaned; its next switch_to_simpy raises
-    # GreenletExit mid-add, producing rank 0 mean=1 (expected 3).
-    # Fix requires redesigning worker semantics so env.run only ever
-    # drives from main (options: lazy-deploy tensor API, coroutine
-    # worker, or setup/verify split). Not a single-PR change — parked
-    # until ADR-0027 (Megatron TP) starts, at which point a proper
-    # architectural solution lands together with TP use cases.
+    # ADR-0027 D0+D1 landed the architectural fix (worker-wait
+    # generalization + torch.multiprocessing.spawn scheduler drain), so
+    # this case now passes normally. Keeping it as the topology-default
+    # smoke.
     pytest.param(
         "ring_allreduce_tcm", "kernbench.ccl.algorithms.ring_allreduce",
         "ring_1d", "tcm", None, 8, 2,
         id="ring_default_ws",
-        marks=pytest.mark.xfail(
-            reason="ADR-0024 Phase B: worker-greenlet env.run captures "
-                   "kernel greenlet as child → orphaned on worker yield. "
-                   "Needs architectural redesign (see test comment).",
-            strict=True,
-        ),
     ),
     # Buffer variants at 8-rank (fast — same kernel, different slot space).
     pytest.param(