Add Tensor indexing + hierarchical 3-level all-reduce kernel

Tensor.__setitem__ / __getitem__:
- Shard-aligned slice assignment and read on deployed tensors.
- Scalar broadcast and numpy array assignment supported.
- Cross-shard slices raise NotImplementedError (use copy_ for that).
- 3 new tests: single-PE, multi-PE, cross-shard error case.

Hierarchical all-reduce kernel (src/kernbench/ccl/algorithms/):
- 3-level reduce: intra-cube (E/W) → inter-cube (N/S) → inter-SIP (parent).
- Bidirectional ring reduce at each level: ceil((N-1)/2) rounds.
  Left half sends via dir_dec, right half via dir_inc (wrap).
  Representative receives from both sides.
- Chain broadcast for reverse path: cube 0 PE 0 → all PE 0s → all PEs.
- Registered in ccl.yaml as "hierarchical_allreduce" with topology: none
  (neighbors() override builds the full 3-level neighbor map).
- kernel_args derives pes_per_cube/cubes_per_sip/num_sips from world_size.
- Mock-verified at 8/16/32/64/128 ranks.

Mock runtime fixes:
- Direction pairing: explicit N↔S, E↔W, parent↔parent instead of
  "first matching reverse". Fixes 2-element rings where N and S both
  point to the same peer.
- Deadlock detection: send-counter based (not just queue-depth-total)
  to catch chain reductions where send+recv pairs net to zero.
- Multi-cube program_id: pes_per_cube parameter enables
  program_id(axis=0) = PE within cube, program_id(axis=1) = cube id.
  Legacy single-cube tests unaffected (default = world_size).

504 tests pass in 12s.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

tests/test_runtime_api_tensor.py

@@ -134,3 +134,73 @@ def test_copy_shape_mismatch_raises():
             t.copy_(torch.from_numpy(src))
 
     _run_with(body)
+
+
+# ── __setitem__ / __getitem__ (shard-aligned) ───────────────────────
+
+
+def test_setitem_getitem_single_pe():
+    """Scalar and slice assignment on a single-PE tensor round-trips."""
+
+    def body(torch):
+        dp = DPPolicy(sip="replicate", cube="replicate", pe="replicate",
+                      num_sips=1, num_cubes=1, num_pes=1)
+        t = torch.zeros((1, 8), dtype="f16", dp=dp, name="t")
+
+        # Scalar broadcast
+        t[0, 0:4] = 3.0
+        assert np.allclose(t[0, 0:4], 3.0)
+        assert np.allclose(t[0, 4:8], 0.0)  # untouched
+
+        # Array assignment
+        t[0, 4:8] = np.array([10, 20, 30, 40], dtype=np.float16)
+        assert np.array_equal(t[0, 4:8], [10, 20, 30, 40])
+
+        # Full read-back via numpy()
+        expected = np.array([[3, 3, 3, 3, 10, 20, 30, 40]], dtype=np.float16)
+        assert np.array_equal(t.numpy(), expected)
+
+    _run_with(body)
+
+
+def test_setitem_getitem_multi_pe_shard_aligned():
+    """Shard-aligned slice assignment on an 8-PE column-wise tensor."""
+
+    def body(torch):
+        n_pe = 8
+        n_elem = 4  # per shard
+        dp = DPPolicy(sip="replicate", cube="replicate", pe="column_wise",
+                      num_sips=1, num_cubes=1, num_pes=n_pe)
+        t = torch.zeros((1, n_pe * n_elem), dtype="f16", dp=dp, name="t")
+
+        # Write each shard with its rank value
+        for r in range(n_pe):
+            t[0, r * n_elem : (r + 1) * n_elem] = float(r + 1)
+
+        # Read back each shard
+        for r in range(n_pe):
+            expected = float(r + 1)
+            arr = t[0, r * n_elem : (r + 1) * n_elem]
+            assert np.allclose(arr, expected), f"shard {r}: {arr} != {expected}"
+
+        # Full gather
+        full = t.numpy().reshape(-1)
+        for r in range(n_pe):
+            assert np.allclose(full[r * n_elem : (r + 1) * n_elem], float(r + 1))
+
+    _run_with(body)
+
+
+def test_setitem_cross_shard_raises():
+    """Slice spanning two shards raises NotImplementedError."""
+
+    def body(torch):
+        n_pe = 4
+        n_elem = 4
+        dp = DPPolicy(sip="replicate", cube="replicate", pe="column_wise",
+                      num_sips=1, num_cubes=1, num_pes=n_pe)
+        t = torch.zeros((1, n_pe * n_elem), dtype="f16", dp=dp, name="t")
+        with pytest.raises(NotImplementedError, match="spans multiple shards"):
+            t[0, 2:6] = 1.0  # crosses shard 0 (0:4) and shard 1 (4:8)
+
+    _run_with(body)