diff --git a/docs/report/1H-codesign-paper/build/main.pdf b/docs/report/1H-codesign-paper/build/main.pdf index 15e630d..9c86446 100644 Binary files a/docs/report/1H-codesign-paper/build/main.pdf and b/docs/report/1H-codesign-paper/build/main.pdf differ diff --git a/docs/report/1H-codesign-paper/figures/cube_architecture.pdf b/docs/report/1H-codesign-paper/figures/cube_architecture.pdf new file mode 100644 index 0000000..b9b91d3 Binary files /dev/null and b/docs/report/1H-codesign-paper/figures/cube_architecture.pdf differ diff --git a/docs/report/1H-codesign-paper/figures/latency_model.png b/docs/report/1H-codesign-paper/figures/latency_model.png new file mode 100644 index 0000000..6712c15 Binary files /dev/null and b/docs/report/1H-codesign-paper/figures/latency_model.png differ diff --git a/docs/report/1H-codesign-paper/figures/pe_architecture.png b/docs/report/1H-codesign-paper/figures/pe_architecture.png new file mode 100644 index 0000000..33b72a5 Binary files /dev/null and b/docs/report/1H-codesign-paper/figures/pe_architecture.png differ diff --git a/docs/report/1H-codesign-paper/figures/sip_architecture.pdf b/docs/report/1H-codesign-paper/figures/sip_architecture.pdf new file mode 100644 index 0000000..86e878a Binary files /dev/null and b/docs/report/1H-codesign-paper/figures/sip_architecture.pdf differ diff --git a/docs/report/1H-codesign-paper/main.tex b/docs/report/1H-codesign-paper/main.tex index 5d3fa3f..8219248 100644 --- a/docs/report/1H-codesign-paper/main.tex +++ b/docs/report/1H-codesign-paper/main.tex @@ -15,6 +15,8 @@ \usepackage{caption} \captionsetup{font=small,labelfont=bf} \usepackage{microtype} +\usepackage{tikz} +\usetikzlibrary{arrows.meta,positioning,calc,fit} \graphicspath{{figures/}} diff --git a/docs/report/1H-codesign-paper/sections/02-platform.tex b/docs/report/1H-codesign-paper/sections/02-platform.tex index 2afe6d7..4ccc503 100644 --- a/docs/report/1H-codesign-paper/sections/02-platform.tex +++ b/docs/report/1H-codesign-paper/sections/02-platform.tex @@ -3,11 +3,10 @@ All results in this report are produced on \emph{KernBench}, a system-level discrete-event simulator for LLM kernels running on AHBM. -This section explains why the platform exists, how it executes a -kernel, how its latency model works---specifically how the hardware is -viewed as a graph, how that graph is driven by a discrete-event engine, -and how congestion is captured---and the concrete hardware configuration -used for every experiment that follows. +This section explains why the platform exists, the device and +execution model it presents to a kernel writer, how its latency model +turns that execution into a number, and the concrete hardware +configuration used for every experiment that follows. \subsection{Why KernBench: source-level kernels without a software stack} \label{sec:why} @@ -33,51 +32,140 @@ without the confound of compiler maturity or framework overhead. The cost is that KernBench numbers are \emph{not} E2E latencies; they are the achievable-kernel latencies an ideal software stack would expose. -\subsection{Execution model} +\subsection{Device and execution model} \label{sec:exec} +KernBench models AHBM as a hierarchy of SIPs, CUBEs, and processing +elements (PEs). At the system level, multiple SIPs are connected +through inter-package links, while each SIP contains a collection of +CUBEs joined by an on-package interconnect +(Fig.~\ref{fig:sip-arch}). + +\begin{figure}[t] +\centering +\includegraphics[width=\linewidth]{sip_architecture.pdf} +\caption{Modeled hardware graph at the \emph{SIP} level (one +example configuration; specific parameters in \S\ref{sec:hw} / +Table~\ref{tab:hw}). The SIP holds a $4{\times}4$ mesh of CUBEs and +an IO chiplet, with each line a directed link labelled by its +physical distance and bandwidth.} +\label{fig:sip-arch} +\end{figure} + +Each CUBE contains eight PEs, shared SRAM, HBM controllers, an +\textsf{M\_CPU} control processor, and an intra-CUBE router mesh +(Fig.~\ref{fig:cube-arch}). Together these components form the +execution substrate for all kernels evaluated in this report. + +\begin{figure}[t] +\centering +\includegraphics[width=\linewidth]{cube_architecture.pdf} +\caption{Modeled hardware graph at the \emph{CUBE} level---a +zoom-in on one CUBE node from Fig.~\ref{fig:sip-arch}. The CUBE +holds 8 PEs (each with its own HBM channels), the \textsf{M\_CPU} +control processor, a shared SRAM, an intra-CUBE NoC router mesh, and +UCIe links to neighbouring CUBEs.} +\label{fig:cube-arch} +\end{figure} + +Within a PE, commands are dispatched by \textsf{PE\_CPU} to +\textsf{PE\_SCHED}, which routes work to specialized execution +engines---DMA, FETCH/STORE, GEMM, vector-math, and IPCQ +(Fig.~\ref{fig:pe-arch}). Commands come in two flavours. \emph{Atomic} +commands target a single engine---a plain DMA read, a GEMM tile, a +vector-math op, an IPCQ send/receive---and are the natural unit for +short or special-purpose work; the \textsf{PE\_CPU} itself also runs +control-plane work directly. \emph{Composite} commands, by contrast, +carry an ordered pipeline of operations across multiple engines +(\textsf{DMA\_READ} $\rightarrow$ \textsf{FETCH} $\rightarrow$ +\textsf{GEMM}/\textsf{MATH} $\rightarrow$ \textsf{STORE} $\rightarrow$ +\textsf{DMA\_WRITE}) that the \textsf{PE\_SCHED} tiles and streams +without per-tile redispatch. The composite form is the substrate for +the GEMM optimization (\S\ref{sec:gemm}) and, combined with on-PE +collectives, for fused attention (\S\ref{sec:gqa}). + +\begin{figure*}[t] +\centering +\includegraphics[width=\linewidth]{pe_architecture.png} +\caption{Modeled PE architecture used in this report---one example +configuration whose specific parameters are listed in +\S\ref{sec:hw} (Table~\ref{tab:hw}); KernBench is not tied to this +particular decomposition and supports arbitrary PE block layouts +provided each component is given a port and bandwidth model. +\textsf{PE\_CPU} dispatches commands to the \textsf{PE\_SCHED}, which +routes tile-token streams through the \textsf{PE\_DMA}, +\textsf{PE\_FETCH\_STORE}, and \textsf{GEMM}/\textsf{MATH} engines +along on-chip links; the \textsf{PE\_IPCQ} provides the control plane +for on-device collective communication.} +\label{fig:pe-arch} +\end{figure*} + KernBench is layered along the flow of a request: \begin{itemize} \item The \textbf{runtime API} is host-facing and topology-agnostic---it deploys tensors and launches kernels but knows nothing about routing or interconnect. - \item The \textbf{simulation engine} schedules discrete events, routes - every request through the modeled graph, and tracks completion via - per-request correlation IDs. + \item The \textbf{simulation engine} schedules discrete events and + routes every request through the modeled graph. \item The \textbf{components} are device-side nodes that model - hardware behavior: the per-PE blocks (scheduler, DMA, GEMM and + hardware behaviour: the per-PE blocks (scheduler, DMA, GEMM and vector-math engines, TCM, IPCQ), the NoC routers, the HBM controllers, and the inter-chiplet links. \end{itemize} -Within a PE, work is expressed as \emph{composite commands}: a single -command carries an ordered pipeline of operations -(\textsf{DMA\_READ} $\rightarrow$ \textsf{FETCH} $\rightarrow$ -\textsf{GEMM}/\textsf{MATH} $\rightarrow$ \textsf{STORE} $\rightarrow$ -\textsf{DMA\_WRITE}) that the PE scheduler tiles and streams. This -composite mechanism is the substrate for the GEMM optimization -(\S\ref{sec:gemm}) and, combined with on-PE collectives, for fused -attention (\S\ref{sec:gqa}). Data and timing are handled in two passes, -so that a kernel's numeric results and its latency are computed -consistently but independently. +Data and timing are handled in two passes, so that a kernel's numeric +results and its latency are computed consistently but independently. -\subsection{Latency model: a graph traversed by events} +\subsection{Latency model: graph traversal and contention} \label{sec:latency} -\paragraph{The hardware as a graph.} KernBench views the modeled -hardware as a directed graph. \emph{Nodes} are the components listed -above; \emph{edges} are the interconnect links between them, each -carrying bandwidth (\si{\giga\byte\per\second}) and propagation -(\si{\nano\second}) attributes. The topology is compiled once at +The modeled hardware hierarchy described above is represented +internally as a directed graph. Nodes correspond to hardware +components---PEs, routers, memory controllers, SRAM blocks, IO +chiplets---while edges represent communication links with associated +bandwidth (\si{\giga\byte\per\second}) and propagation +(\si{\nano\second}) attributes. Every operation in KernBench---DMA +transfers, remote memory accesses, collective communication, command +dispatch---is modelled as a traversal through this graph. End-to-end +latency is decomposed into four contributions accumulated along the +traversal path (Fig.~\ref{fig:latency-model}): \textbf{per-node +overhead} at each component, \textbf{per-edge transmission} on each +wire, \textbf{drain} (per-flit service occupancy) at the destination, +and \textbf{queuing delay} at the shared FIFOs that the wire and the +destination share between concurrent transactions. + +\begin{figure*}[t] +\centering +\includegraphics[width=\textwidth]{latency_model.png} +\caption{Conceptual schematic of the latency model. Two source nodes +(Requester A/B) inject flits through a chain of routers into a +destination node; on the shared edge between routers, flits from the +two transactions are interleaved flit-by-flit by the wire's FIFO +arrival order. End-to-end latency is the sum of four contributions: +\textbf{per-node overhead} (the switch's fixed processing cost, +shown in light yellow---the same colour as the destination's +processing-logic block), \textbf{per-edge transmission} +($\textit{flit\_size}/\textit{BW}$ on each wire), +\textbf{drain} (per-flit service occupancy at the destination's +channel), and \textbf{queuing delay} (waiting in a FIFO when a +shared resource is busy). The places where queuing actually +accumulates are highlighted in green: the router output queue and +the destination's input queue. This is the model, not a +measurement; specific bandwidths and overheads are listed in +\S\ref{sec:hw} (Table~\ref{tab:hw}).} +\label{fig:latency-model} +\end{figure*} + +\paragraph{The hardware as a graph.} The topology is compiled once at configuration time into this graph and is never mutated during a run. -Every routed request---a DMA, a remote read, an IPCQ message, a -kernel-launch command---is a \emph{traversal} of this graph from a -source node to a destination service, hopping through routers and -links along the way. There are no hidden shortcuts, implicit bypasses, -or magic paths: if a request reaches its destination, the path it took -is explicit in the graph, and the latency it incurred is the sum of -the per-node and per-edge costs paid along that path. +There are no hidden shortcuts, implicit bypasses, or magic paths: if a +request reaches its destination, the path it took is explicit in the +graph, and the latency it incurred is the sum of the per-node and +per-edge costs paid along that path. The same graph representation +applies recursively at every hierarchy level---system, SIP +(Fig.~\ref{fig:sip-arch}), CUBE (Fig.~\ref{fig:cube-arch}), and PE +(Fig.~\ref{fig:pe-arch}). \paragraph{From graph to discrete-event simulation.} The graph is driven by a discrete-event engine. Two kinds of events advance @@ -85,14 +173,14 @@ simulation time: \emph{node events} (component switching overhead, service completions such as an HBM channel commit or a GEMM tile finish) and \emph{edge events} (the flit-by-flit serialization of a payload across a bandwidth-limited link). The engine maintains a -priority queue of pending events ordered by their scheduled time, -fires them one at a time, and treats ties under a deterministic -ordering policy so that the same kernel on the same topology always -yields the same trace. Per-request correlation IDs are stamped at -injection and carried through every hop, so the trace is recoverable -from injection to completion. Every nanosecond in a reported latency -traces back to exactly one of these events on exactly one node or -edge. +priority queue of pending events ordered by their scheduled time and +fires them one at a time, with ties broken under a deterministic +policy so that the same kernel on the same topology always yields the +same trace. Per-request correlation IDs are stamped at injection and +carried through every hop, so the path from injection to completion +is fully traceable. Every nanosecond in a reported latency +corresponds to exactly one of these events on exactly one node or +edge---there is no slack in the budget. \paragraph{Latency contributions.} Three kinds of latency accumulate along a traversal: (i) \emph{per-node fixed overhead}---each component @@ -113,11 +201,11 @@ edges. \emph{Every directed edge has a FIFO}: an arriving flit takes its bandwidth-limited transfer time on top of whatever earlier flits are still being served, so a busy link queues later traffic behind earlier traffic rather than transferring everything at peak BW. -\emph{HBM is modeled with per-pseudo-channel parallelism}: a stateless -array of channel-availability timestamps with address-based channel -selection captures the bank-level concurrency that real HBM exposes, -so 64 channels per CUBE deliver real parallelism on uniform addresses -but a hot channel surfaces as the bottleneck on skewed ones. +\emph{HBM is modelled with per-pseudo-channel parallelism}: a +stateless array of channel-availability timestamps with address-based +channel selection captures the bank-level concurrency that real HBM +exposes, so 64 channels per CUBE deliver real parallelism on uniform +addresses but a hot channel surfaces as the bottleneck on skewed ones. \emph{Every component has a serial worker}: a router carrying two heavy streams interleaves them at flit granularity in arrival order rather than fanning out for free, so two concurrent collectives sharing @@ -128,7 +216,7 @@ actually relieve them---which is exactly the question the codesign work in this report turns on. \paragraph{Control-plane (issue) cost model.} The cost of \emph{issuing} -a command is modeled structurally rather than with a per-operation +a command is modelled structurally rather than with a per-operation calibration table. The PE control processor charges, per command, \[ d_{\text{cmd}} = \textsf{FIXED} + b_{\text{logical}} \cdot R, @@ -171,10 +259,13 @@ than a model error. The known simplifications---FIFO router arbitration (instead of round-robin), HBM scheduler without write-buffer reordering, no bank conflict, no refresh or thermal effects, and no upstream backpressure---are the price of a -deterministic, inspectable model. They bound the absolute accuracy but -do not distort the \emph{relative} comparisons (tiling A vs.\ B, -topology X vs.\ Y, with vs.\ without composite command, mesh vs.\ -torus) that this report is built on. +deterministic, inspectable model. These simplifications bound the +absolute accuracy, but the agreement with both analytic models and the +external full-system simulator data above indicates that KernBench is +sufficiently accurate for evaluating the \emph{relative} +hardware--software design trade-offs (tiling A vs.\ B, topology X +vs.\ Y, with vs.\ without composite command, mesh vs.\ torus) that +are the primary objective of this work. \subsection{Modeled hardware configuration} \label{sec:hw} diff --git a/scripts/paper/paper_latency_model_diagram.py b/scripts/paper/paper_latency_model_diagram.py new file mode 100644 index 0000000..f09cc77 --- /dev/null +++ b/scripts/paper/paper_latency_model_diagram.py @@ -0,0 +1,535 @@ +#!/usr/bin/env python3 +"""Conceptual latency-model diagram for the KernBench paper (v5). + +Generic naming (Requester Node A/B, Router, Destination Node), with +Router internals visible (two input ports -> switch -> output queue -> +output port) so the queuing delay can be located *at the out port* +rather than at the box edge. The Destination Node shows queue -> drain +slot -> processing logic. + +Highlights: + * Same-size Routers, each with explicit switching logic + output queue. + * Edge labels removed -- the wires speak for themselves. + * Annotation lines are strictly vertical; arrow heads use a larger + mutation_scale so no thin line protrudes past the tip; `shrinkA` + pulls the tail away from the text so they no longer overlap. + * "flit-level interleaving on wires" (not "shared FIFO"). + * Destination: queue -> drain -> processing logic (no "served"). + +Output: + docs/report/1H-codesign-paper/figures/latency_model.png + +Per /paper isolation: this is a report-only harness under scripts/paper/. +""" + +import math +from pathlib import Path + +import matplotlib.patches as patches +import matplotlib.pyplot as plt + +OUT = Path( + "/Users/ywkang/kernbench/docs/report/1H-codesign-paper/figures/latency_model.png" +) +OUT.parent.mkdir(parents=True, exist_ok=True) + + +# --- Colours -------------------------------------------------------------- +C_A = "#E07B5E" +C_B = "#5E9BD1" +C_NODE = "#FFFFFF" +C_BORD = "#222222" +C_WIRE = "#222222" +C_ANN = "#333333" +C_DIM = "#777777" +C_SWITCH = "#FAFAFA" +C_OQUEUE = "#EFEFEF" +C_PROC = "#F4ECDC" +C_QUEUE = "#3CB371" # medium-sea-green: distinctive vs A/B flit colours + + +# --- Canvas --------------------------------------------------------------- +fig = plt.figure(figsize=(17.0, 5.0)) +ax = fig.add_subplot(111) +ax.set_xlim(0, 33) +ax.set_ylim(2.2, 11.2) +ax.axis("off") + + +# --- Top header: end-to-end latency formula ----------------------------- +ax.text( + 16.5, 10.6, + r"End-to-end latency = $\Sigma$ per-node overhead + " + r"$\Sigma$ per-edge transmission + drain + " + r"queuing delay", + ha="center", fontsize=12, color=C_ANN, weight="bold", +) +ax.plot([1.5, 31.5], [10.05, 10.05], color=C_DIM, lw=0.6) + + +# --- Helpers -------------------------------------------------------------- +def box(cx, cy, w, h, label, fs=11, fweight="normal"): + ax.add_patch(patches.FancyBboxPatch( + (cx - w / 2, cy - h / 2), w, h, + boxstyle="round,pad=0.05", + facecolor=C_NODE, edgecolor=C_BORD, linewidth=1.6, + )) + if label: + ax.text(cx, cy, label, ha="center", va="center", + fontsize=fs, weight=fweight) + + +def draw_flit_aligned(cx, cy, angle_rad, w, h, color, label): + """Draw a flit polygon rotated to align with the wire angle.""" + cos_a, sin_a = math.cos(angle_rad), math.sin(angle_rad) + corners = [] + for dx, dy in [(-w / 2, -h / 2), (w / 2, -h / 2), + (w / 2, h / 2), (-w / 2, h / 2)]: + rx = dx * cos_a - dy * sin_a + ry = dx * sin_a + dy * cos_a + corners.append((cx + rx, cy + ry)) + ax.add_patch(patches.Polygon( + corners, facecolor=color, edgecolor="black", linewidth=0.5, + )) + ax.text(cx, cy, label, ha="center", va="center", + fontsize=9, color="white", weight="bold") + + +def draw_wire_with_flits(x0, y0, x1, y1, n_flits, label_char, + color=None, flit_w=0.66, flit_h=0.95, gap=0.10): + """Wire (line) + endpoint arrowhead + flits centred on the line. + + Wires are drawn in the neutral C_WIRE colour: the wire itself is + the place where the *transmission* delay accumulates (flit_size / + BW), not where flits queue. Queuing happens before the wire, at + the FIFO at the egress side -- coloured separately. + """ + head_back = 0.30 + dx, dy = x1 - x0, y1 - y0 + wire_len = math.hypot(dx, dy) + ux, uy = dx / wire_len, dy / wire_len + x_line_end = x1 - ux * head_back + y_line_end = y1 - uy * head_back + ax.plot([x0, x_line_end], [y0, y_line_end], + color=C_WIRE, lw=1.6, zorder=1) + head_len, head_half = 0.30, 0.16 + bx = x1 - ux * head_len + by = y1 - uy * head_len + px, py = -uy, ux + tri = [ + (x1, y1), + (bx + px * head_half, by + py * head_half), + (bx - px * head_half, by - py * head_half), + ] + ax.add_patch(patches.Polygon(tri, facecolor=C_WIRE, + edgecolor=C_WIRE, linewidth=0.0)) + # Flit train + angle_rad = math.atan2(dy, dx) + train_len = n_flits * flit_w + (n_flits - 1) * gap + centre_p = 0.5 + start_p = centre_p - (train_len / 2) / wire_len + step_p = (flit_w + gap) / wire_len + if isinstance(label_char, str): + labels = [label_char] * n_flits + cols = [color] * n_flits + else: + labels = [c[0] for c in label_char] + cols = [c[1] for c in label_char] + for i in range(n_flits): + p = start_p + (i + 0.5) * step_p + cx = x0 + p * dx + cy = y0 + p * dy + draw_flit_aligned(cx, cy, angle_rad, + flit_w, flit_h, cols[i], labels[i]) + + +def draw_router(cx, cy, w, h, two_inputs=True): + """Same-size Router with explicit internal switching logic and an + output queue. Returns the wire-attachment (x,y) for each input + port and the single output port: + ((in1_x, in1_y), (in2_x, in2_y) or None, (out_x, out_y)) + """ + # Outer box + box(cx, cy, w, h, "") + ax.text(cx, cy + h / 2 - 0.32, "Router", + ha="center", fontsize=10, weight="bold") + + # Input ports (small circles on the left edge) + in_x = cx - w / 2 + in_x_internal = in_x + 0.25 + if two_inputs: + in_y_top = cy + 0.55 + in_y_bot = cy - 0.55 + for iy in (in_y_top, in_y_bot): + ax.add_patch(patches.Circle( + (in_x_internal, iy), 0.12, + facecolor="white", edgecolor=C_BORD, linewidth=0.9, + )) + else: + in_y_top = None + in_y_bot = cy + ax.add_patch(patches.Circle( + (in_x_internal, in_y_bot), 0.12, + facecolor="white", edgecolor=C_BORD, linewidth=0.9, + )) + + # Switch (small box, centre-left-ish). sw_cy = cy so that the + # output queue and (single-input) input port are at the same y as + # the router centre -- this keeps all router-to-router edges + # strictly horizontal. + sw_w, sw_h = 0.85, 1.10 + sw_cx = cx - 0.55 + sw_cy = cy + # The switch is the router's processing logic -- colour it the + # same as the Destination Node's processing-logic block so the + # two read as the same "processing" concept. + ax.add_patch(patches.Rectangle( + (sw_cx - sw_w / 2, sw_cy - sw_h / 2), sw_w, sw_h, + facecolor=C_PROC, edgecolor=C_BORD, linewidth=0.8, + )) + ax.text(sw_cx, sw_cy, "switch", ha="center", va="center", + fontsize=7.5, style="italic") + + # Short feeder lines (no arrowhead) from input ports to the + # switch. We deliberately drop arrowheads here: the head + port + # circle were too small at this scale and read as an overlap. + sw_left_x = sw_cx - sw_w / 2 + if two_inputs: + for iy in (in_y_top, in_y_bot): + ax.plot( + [in_x_internal + 0.12, sw_left_x], + [iy, sw_cy + 0.25 * ((iy - cy) / 0.55)], + color=C_DIM, lw=0.7, zorder=1, + ) + else: + ax.plot( + [in_x_internal + 0.12, sw_left_x], + [in_y_bot, sw_cy], + color=C_DIM, lw=0.7, zorder=1, + ) + + # Output queue (small queue holding a couple of flits). This *is* + # a real queueing location, so its fill takes the C_QUEUE family. + oq_w, oq_h = 0.95, 0.55 + oq_cx = cx + 0.55 + oq_cy = sw_cy + ax.add_patch(patches.Rectangle( + (oq_cx - oq_w / 2, oq_cy - oq_h / 2), oq_w, oq_h, + facecolor="#D9F0E1", edgecolor=C_QUEUE, linewidth=0.9, + )) + ax.text(oq_cx, oq_cy + oq_h / 2 + 0.18, "FIFO", + ha="center", fontsize=7, color=C_DIM, style="italic") + # Two small flits inside (A and B) hinting at the in-flight contents + mini_w, mini_h = 0.22, 0.34 + for i, (col, lab) in enumerate([(C_A, "A"), (C_B, "B")]): + mx = oq_cx - 0.30 + i * (mini_w + 0.06) + ax.add_patch(patches.Rectangle( + (mx, oq_cy - mini_h / 2), mini_w, mini_h, + facecolor=col, edgecolor="black", linewidth=0.3, + )) + ax.text(mx + mini_w / 2, oq_cy, lab, + ha="center", va="center", + fontsize=5.5, color="white", weight="bold") + + # Short feeder line (no arrowhead) from switch into out queue + ax.plot( + [sw_cx + sw_w / 2, oq_cx - oq_w / 2], + [sw_cy, oq_cy], + color=C_DIM, lw=0.7, zorder=1, + ) + # Short feeder line from out queue to out port + out_x_internal = cx + w / 2 - 0.25 + ax.plot( + [oq_cx + oq_w / 2, out_x_internal - 0.12], + [oq_cy, oq_cy], + color=C_DIM, lw=0.7, zorder=1, + ) + # Output port circle on the right edge (a port marker, not a + # queue -- the queue is on the wire that follows, not the port). + ax.add_patch(patches.Circle( + (out_x_internal, oq_cy), 0.12, + facecolor="white", edgecolor=C_BORD, linewidth=0.9, + )) + + return ( + (in_x_internal, in_y_top) if two_inputs else None, + (in_x_internal, in_y_bot), + (out_x_internal, oq_cy), + oq_cx, # also return the out-queue centre so callouts can target it + oq_cy, + ) + + +# --- Box layout --------------------------------------------------------- +# Requester centres set to match Router 1's input-port y so that +# Edge 1A and Edge 1B are strictly horizontal. Box heights are kept +# small enough that a visible gap separates the two Requester boxes. +R1_TMP_CY = 7.0 +ReqA = (2.8, R1_TMP_CY + 0.55) # = 7.55, matches in_y_top of R1 +ReqB = (2.8, R1_TMP_CY - 0.55) # = 6.45, matches in_y_bot of R1 +box(*ReqA, 3.0, 0.85, "Requester\nNode A", fs=10) +box(*ReqB, 3.0, 0.85, "Requester\nNode B", fs=10) + +R_W, R_H = 3.0, 2.8 +R1 = (10.5, 7.0) +R2 = (20.0, 7.0) + +r1_in_top, r1_in_bot, r1_out, r1_oq_cx, r1_oq_cy = draw_router( + *R1, R_W, R_H, two_inputs=True, +) +_, r2_in_bot, r2_out, r2_oq_cx, r2_oq_cy = draw_router( + *R2, R_W, R_H, two_inputs=False, +) + +# Destination Node (same height as Router; wide enough so queue + +# drain + processing-logic all fit on a single horizontal row). +Dst = (28.4, 7.0) +Dst_W, Dst_H = 6.0, R_H # match the Router height +box(*Dst, Dst_W, Dst_H, "") +ax.text(Dst[0], Dst[1] + Dst_H / 2 - 0.25, "Destination Node", + ha="center", fontsize=10, weight="bold") + + +# --- Edges: requester -> router 1 (Edge 1A & 1B, with flits) ------------ +draw_wire_with_flits( + ReqA[0] + 1.6, ReqA[1], + r1_in_top[0] - 0.02, r1_in_top[1], + n_flits=4, color=C_A, label_char="A", +) +draw_wire_with_flits( + ReqB[0] + 1.6, ReqB[1], + r1_in_bot[0] - 0.02, r1_in_bot[1], + n_flits=4, color=C_B, label_char="B", +) + +# Edge 2: router1 out -> router2 in (horizontal, interleaved) +labels_e2 = [("A", C_A) if i % 2 == 0 else ("B", C_B) for i in range(8)] +E2_y = r1_out[1] +draw_wire_with_flits( + r1_out[0] + 0.02, r1_out[1], + r2_in_bot[0] - 0.02, r2_in_bot[1], + n_flits=8, label_char=labels_e2, +) + +# Edge 3: router2 out -> destination (horizontal, interleaved) +labels_e3 = [("A", C_A) if i % 2 == 0 else ("B", C_B) for i in range(4)] +draw_wire_with_flits( + r2_out[0] + 0.02, r2_out[1], + Dst[0] - Dst_W / 2, r2_out[1], + n_flits=4, label_char=labels_e3, +) + + +# --- Destination internals: queue -> drain -> processing logic ---------- +# Light-green halo behind the queue area marks it as a queueing point. +C_QUEUE_BG = "#D9F0E1" + +dy_main = Dst[1] - 0.10 +Dst_left = Dst[0] - Dst_W / 2 + +# Queue (4 flits, FIFO; rightmost is the next to drain, so the +# order is set so the serve sequence after the in-flight A alternates +# A (in drain) -> B -> A -> B -> A -- giving a clean BABA queue. +qW, qH, qGap = 0.42, 0.62, 0.07 +q_labels = ["A", "B", "A", "B"] +q_cols = [C_A, C_B, C_A, C_B] +q_total = len(q_labels) * qW + (len(q_labels) - 1) * qGap +q_x_start = Dst_left + 0.35 + +# Green halo behind the queue boxes (the queue itself is a queueing +# location -- mark it with the C_QUEUE colour family) +ax.add_patch(patches.FancyBboxPatch( + (q_x_start - 0.10, dy_main - qH / 2 - 0.08), + q_total + 0.20, qH + 0.16, + boxstyle="round,pad=0.02", + facecolor=C_QUEUE_BG, edgecolor=C_QUEUE, + linewidth=0.9, zorder=1.5, +)) + +for i, (lab, col) in enumerate(zip(q_labels, q_cols)): + qx = q_x_start + i * (qW + qGap) + ax.add_patch(patches.Rectangle( + (qx, dy_main - qH / 2), qW, qH, + facecolor=col, edgecolor="black", linewidth=0.4, + zorder=2, + )) + ax.text(qx + qW / 2, dy_main, lab, + ha="center", va="center", + fontsize=8, color="white", weight="bold", zorder=3) +# Common y for the "queue" / "drain" labels. Matches the FIFO label +# spacing inside the Router (0.18 above the box top) and uses the same +# font size for visual consistency. +DST_LABEL_Y = dy_main + qH / 2 + 0.18 +ax.text(q_x_start + q_total / 2, DST_LABEL_Y, + "queue", + ha="center", fontsize=7, style="italic", color=C_DIM) + +# Drain slot (height matched to queue boxes; font matched to FIFO/queue +# labels for visual consistency) +dr_W, dr_H = 0.85, qH +dr_x = q_x_start + q_total + 0.35 +dr_cx = dr_x + dr_W / 2 +dr_cy = dy_main +ax.add_patch(patches.FancyBboxPatch( + (dr_x, dr_cy - dr_H / 2), dr_W, dr_H, + boxstyle="round,pad=0.03", + facecolor=C_A, edgecolor="black", linewidth=1.0, +)) +ax.text(dr_cx, dr_cy, "A", ha="center", va="center", + fontsize=9, color="white", weight="bold") +ax.text(dr_cx, DST_LABEL_Y, + "drain", + ha="center", fontsize=7, style="italic", color=C_DIM) + +# Drain-time double-arrow underneath the drain slot. Move the "drain +# time" text a touch further down so the descender does not collide +# with the arrowhead. +dt_y = dr_cy - dr_H / 2 - 0.28 +ax.annotate( + "", xy=(dr_x + dr_W, dt_y), xytext=(dr_x, dt_y), + arrowprops=dict(arrowstyle="<->", lw=0.9, color=C_ANN), +) +ax.text(dr_cx, dt_y - 0.42, "drain time", + ha="center", fontsize=8.5, color=C_ANN, style="italic") + +# Processing-logic block (after drain) — wider so the two-line text +# does not collide with the box edges +pl_W, pl_H = 1.75, 1.00 +pl_x = dr_x + dr_W + 0.40 +pl_cx = pl_x + pl_W / 2 +pl_cy = dr_cy +ax.add_patch(patches.FancyBboxPatch( + (pl_x, pl_cy - pl_H / 2), pl_W, pl_H, + boxstyle="round,pad=0.03", + facecolor=C_PROC, edgecolor=C_BORD, linewidth=1.0, +)) +ax.text(pl_cx, pl_cy, "processing\nlogic", + ha="center", va="center", fontsize=9) + +# Small gray arrows along the queue -> drain -> processing chain +ax.annotate( + "", xy=(dr_x - 0.04, dr_cy), + xytext=(q_x_start + q_total + 0.13, dr_cy), + arrowprops=dict(arrowstyle="-|>", lw=0.7, + color=C_DIM, mutation_scale=7), +) +ax.annotate( + "", xy=(pl_x - 0.04, dr_cy), + xytext=(dr_x + dr_W + 0.04, dr_cy), + arrowprops=dict(arrowstyle="-|>", lw=0.7, + color=C_DIM, mutation_scale=7), +) + + +# --- Annotations (strictly vertical, no diagonals, no overlap) --------- + +def vcallout(text, xy, x_text_top_y, fs=10, clearance=0.55, + marker_color=None): + """Vertical dotted callout. Optional `marker_color` paints a small + coloured circle just left of the text -- used to tie the label to + a colour code in the diagram. + """ + text_y = x_text_top_y + if text_y < xy[1]: + line_top_y = text_y + clearance + else: + line_top_y = text_y - clearance + ax.plot([xy[0], xy[0]], [xy[1], line_top_y], + color=C_ANN, lw=0.9, linestyle=":", zorder=2) + if marker_color is not None: + # Text-width estimate at fs=10 (~0.18 data units per char) so + # the marker is placed clearly to the left of the text. + text_w = len(text) * 0.18 + marker_x = xy[0] - text_w / 2 - 0.30 + ax.add_patch(patches.Circle( + (marker_x, text_y), 0.14, + facecolor=marker_color, edgecolor=C_BORD, linewidth=0.6, + zorder=3, + )) + ax.text(xy[0], text_y, text, + ha="center", va="center", + fontsize=fs, color=C_ANN, zorder=3) + + +# Transmission delay -> flit on Edge 2 (text BELOW the wire). +# Extra clearance ~ 2 x text height so the line stops well clear of +# the label. +e2_total = 8 * 0.66 + 7 * 0.10 +e2_centre = (r1_out[0] + r2_in_bot[0]) / 2 +e2_start = e2_centre - e2_total / 2 +trans_idx = 5 +trans_cx = e2_start + (trans_idx + 0.5) * (0.66 + 0.10) +vcallout( + "transmission delay = flit_size / BW", + xy=(trans_cx, E2_y - 0.55), + x_text_top_y=E2_y - 2.6, + clearance=0.45, +) + +# Flit-level interleaving -> ABOVE the wire, anchored on a flit near +# the centre/right of the wire. Text y matches the queuing-delay +# callout so the two labels sit on the same horizontal row, separated +# horizontally so they don't collide. +int_idx = 4 +int_cx = e2_start + (int_idx + 0.5) * (0.66 + 0.10) +vcallout( + "flit-level interleaving on wires", + xy=(int_cx, E2_y + 0.55), + x_text_top_y=R1[1] + R_H / 2 + 0.85, +) + +# Queuing delay -> at Router 1 out-queue. Text positioned just above +# the Router-1 box. +vcallout( + "queuing delay", + xy=(r1_oq_cx, r1_oq_cy + 0.30), + x_text_top_y=R1[1] + R_H / 2 + 0.85, + marker_color=C_QUEUE, +) + +# Drain -> below the drain slot (text is below). +# Extra clearance ~ 2.5 x text height so the line stops further from +# the label. +vcallout( + "drain = per-flit service occupancy", + xy=(dr_cx, dt_y - 0.66), # just *inside* the Destination Node + # bottom edge -- the dotted line then + # penetrates the box slightly rather + # than hanging below it + x_text_top_y=E2_y - 2.6, + clearance=0.60, +) + +# Per-node overhead -> points at Router 1's switch block (where the +# component's fixed processing cost lives). Text on the same row as +# transmission delay and drain so the three "below the wire" callouts +# sit on one horizontal baseline. +R1_SW_CX = R1[0] - 0.55 # sw_cx for Router 1 +R1_SW_BOTTOM_Y = R1[1] - 0.55 # bottom of sw box +vcallout( + "per-node overhead", + xy=(R1_SW_CX, R1_SW_BOTTOM_Y), + x_text_top_y=E2_y - 2.6, + clearance=0.60, + marker_color=C_PROC, +) + + +# --- Legend (close to the diagram) -------------------------------------- +LX, LY = 1.0, 2.6 +ax.add_patch(patches.Rectangle((LX, LY), 0.7, 0.45, + facecolor=C_A, edgecolor="black", + linewidth=0.4)) +ax.text(LX + 0.95, LY + 0.22, "Transaction A flit", + va="center", fontsize=10) + +ax.add_patch(patches.Rectangle((LX + 4.8, LY), 0.7, 0.45, + facecolor=C_B, edgecolor="black", + linewidth=0.4)) +ax.text(LX + 5.75, LY + 0.22, "Transaction B flit", + va="center", fontsize=10) + + +# --- Save ---------------------------------------------------------------- +fig.savefig(OUT, dpi=140, bbox_inches="tight", facecolor="white") +print(f"Wrote {OUT}")