Optimizing the AI Fabric: How Multipath Reliable Connection (MRC) is Redefining High-Performance Networking over Ethernet

The rapid evolution of artificial intelligence and machine learning has placed an unprecedented strain on the fundamental architecture of the modern data center. As clusters grow to accommodate tens of thousands of GPUs, the underlying network—the fabric that connects these processing powerhouses—has become a primary bottleneck. In the latest episode of the Heavy Networking podcast, hosts Ethan Banks and Drew Conry-Murray delve into a technical solution poised to address these challenges: Multipath Reliable Connection (MRC).

Joined by a trio of distinguished engineers from Broadcom—Rip Sohan, Eric Davis, and Eric Spada—the discussion centers on the necessity of evolving transport protocols to meet the demands of massive AI workloads. The conversation highlights how MRC is designed to optimize Remote Direct Memory Access (RDMA) traffic across massive scale-out fabrics, particularly when running over standard, "lossy" Ethernet environments.

The Evolution of Transport for AI Workloads

For years, the networking industry has relied on established protocols to move data, but the unique nature of AI training traffic has exposed the limitations of traditional methods. Unlike standard web traffic, which consists of many small, independent flows, AI workloads often involve "elephant flows"—massive, sustained data transfers that occur during collective communication phases like All-Reduce or All-to-All.

In a traditional network using Equal Cost Multipath (ECMP) routing, traffic is typically distributed based on a hash of the packet headers. This "flow-based" approach ensures that all packets in a single stream take the same path, preventing out-of-order delivery. However, in an AI environment, this can lead to "hot spots" where multiple elephant flows are hashed to the same physical link, while other links remain underutilized. This imbalance creates congestion, increases latency, and ultimately slows down the entire AI training process.

As Rip Sohan, Eric Davis, and Eric Spada explained during the session, MRC aims to solve this by moving away from flow-based pinning and toward a more granular approach: packet spraying. By distributing individual packets across all available paths in the fabric, MRC ensures a perfectly even distribution of traffic, maximizing the utilization of the available bandwidth.

Solving the "Lossy" Ethernet Dilemma

One of the most significant hurdles in high-performance networking has been the requirement for "lossless" Ethernet. To run RDMA over Converged Ethernet (RoCEv2), network administrators have traditionally relied on Priority Flow Control (PFC) to prevent packet loss. While effective in small clusters, PFC can lead to complex issues like head-of-line blocking and congestion spreading (often referred to as "pause frames" causing a ripple effect across the fabric) when scaled to thousands of nodes.

The Broadcom engineers highlighted that MRC is specifically engineered to operate over "plain old lossy Ethernet." By building reliability directly into the transport layer, MRC can handle packet loss and out-of-order delivery more gracefully than its predecessors. This shift allows data center operators to build massive AI fabrics without the brittle complexity of PFC, making the network more resilient and easier to manage at scale.