Optimizing the AI Fabric: How Multipath Reliable Connection (MRC) is Redefining High-Performance Networking over Ethernet

The rapid evolution of artificial intelligence and machine learning has placed an unprecedented strain on the fundamental architecture of the modern data center. As clusters grow to accommodate tens of thousands of GPUs, the underlying network—the fabric that connects these processing powerhouses—has become a primary bottleneck. In the latest episode of the Heavy Networking podcast, hosts Ethan Banks and Drew Conry-Murray delve into a technical solution poised to address these challenges: Multipath Reliable Connection (MRC).

Joined by a trio of distinguished engineers from Broadcom—Rip Sohan, Eric Davis, and Eric Spada—the discussion centers on the necessity of evolving transport protocols to meet the demands of massive AI workloads. The conversation highlights how MRC is designed to optimize Remote Direct Memory Access (RDMA) traffic across massive scale-out fabrics, particularly when running over standard, "lossy" Ethernet environments.

The Evolution of Transport for AI Workloads

For years, the networking industry has relied on established protocols to move data, but the unique nature of AI training traffic has exposed the limitations of traditional methods. Unlike standard web traffic, which consists of many small, independent flows, AI workloads often involve "elephant flows"—massive, sustained data transfers that occur during collective communication phases like All-Reduce or All-to-All.

In a traditional network using Equal Cost Multipath (ECMP) routing, traffic is typically distributed based on a hash of the packet headers. This "flow-based" approach ensures that all packets in a single stream take the same path, preventing out-of-order delivery. However, in an AI environment, this can lead to "hot spots" where multiple elephant flows are hashed to the same physical link, while other links remain underutilized. This imbalance creates congestion, increases latency, and ultimately slows down the entire AI training process.

As Rip Sohan, Eric Davis, and Eric Spada explained during the session, MRC aims to solve this by moving away from flow-based pinning and toward a more granular approach: packet spraying. By distributing individual packets across all available paths in the fabric, MRC ensures a perfectly even distribution of traffic, maximizing the utilization of the available bandwidth.

Solving the "Lossy" Ethernet Dilemma

One of the most significant hurdles in high-performance networking has been the requirement for "lossless" Ethernet. To run RDMA over Converged Ethernet (RoCEv2), network administrators have traditionally relied on Priority Flow Control (PFC) to prevent packet loss. While effective in small clusters, PFC can lead to complex issues like head-of-line blocking and congestion spreading (often referred to as "pause frames" causing a ripple effect across the fabric) when scaled to thousands of nodes.

The Broadcom engineers highlighted that MRC is specifically engineered to operate over "plain old lossy Ethernet." By building reliability directly into the transport layer, MRC can handle packet loss and out-of-order delivery more gracefully than its predecessors. This shift allows data center operators to build massive AI fabrics without the brittle complexity of PFC, making the network more resilient and easier to manage at scale.

This resilience is not merely about handling dropped packets; it is about maintaining performance in the face of hardware reality. In a massive data center, link failures and fabric glitches are inevitable. MRC’s ability to dynamically route around failures and handle the resulting reordering of packets ensures that an AI training job—which might cost millions of dollars in compute time—is not derailed by a single faulty cable or transceiver.

The Role of Packet Spraying and Congestion Control

The technical heart of the MRC discussion focused on the mechanics of packet spraying. When packets are "sprayed" across multiple paths, they often arrive at the destination out of order because different paths through the fabric may have slightly different latencies. Traditional transport protocols struggle with this, often interpreting out-of-order packets as a sign of congestion or loss, which triggers a reduction in transmission speed.

MRC, however, is designed with a specialized reorder buffer and a robust congestion control mechanism. This allows the receiver to reconstruct the original data stream efficiently. By decoupling the reliability of the connection from the specific path taken through the network, MRC allows for a much more fluid and responsive data transfer.

During the podcast, the guests emphasized that this is not just about raw speed, but about "tail latency." In AI training, the entire cluster often has to wait for the slowest member to finish its communication phase before the next round of computation can begin. By eliminating hot spots and ensuring even traffic distribution, MRC significantly reduces the "tail" of the latency distribution, ensuring that no single congested link holds back the entire GPU cluster.

AdSpot Sponsor: Itential

As the complexity of these networks grows, the need for sophisticated management tools becomes paramount. This episode’s sponsor, Itential, is addressing this through their FlowAI platform. Itential’s FlowAI delivers "agentic operations" for infrastructure, allowing engineers to build AI agents that are governed, deterministic, and ready for production environments.

By integrating intelligent automation into network operations, Itential aims to remove the "AI chaos" often associated with modern infrastructure management. Their approach focuses on providing tools that work the way engineers need them to, ensuring that as the underlying fabric becomes more complex with technologies like MRC, the operational layer remains manageable and reliable. More information on these capabilities can be found at itential.com/flowai.

Industry Standardization and the Open Compute Project

A critical aspect of the MRC development is its status as an open standard. The guests from Broadcom noted that the Multipath Reliable Connection Specification has been contributed to the Open Compute Project (OCP). This move is intended to foster a broader ecosystem where different hardware vendors and software developers can implement and benefit from the protocol.

The OCP-MRC 1.0 specification provides a detailed framework for how these connections should be established, maintained, and optimized. By making this an open project, the contributors hope to prevent the fragmentation of AI networking and ensure that the industry can move toward a unified, high-performance transport layer.

The importance of this openness is further underscored by the research community’s involvement. The podcast referenced a significant paper, "Resilient AI Supercomputer Networking using MRC and SRv6," available on Arxiv. This research explores how MRC can be combined with Segment Routing over IPv6 (SRv6) to create even more flexible and resilient network architectures. The integration of SRv6 allows for sophisticated traffic engineering and path steering, which, when combined with MRC’s packet spraying, provides a powerful toolkit for building the next generation of AI supercomputers.

The Expertise Behind the Innovation

The depth of the discussion was driven by the extensive experience of the Broadcom engineering team. Rip Sohan, a Distinguished Engineer, Eric Davis, a Master Engineer, and Eric Spada, a Technical Director and Distinguished Engineer, represent the front lines of silicon and protocol design. Their collective work at Broadcom focuses on ensuring that the physical chips—the switches and network interface cards (NICs)—possess the hardware-level logic required to execute MRC’s complex reordering and congestion control tasks at line rate.

The transition to MRC represents a significant engineering challenge, as it requires coordination between the NIC at the edge of the network and the switches in the core. The guests explained that for MRC to be truly effective, the hardware must be able to handle the bookkeeping of thousands of concurrent connections and millions of packets, all while maintaining the nanosecond-level latencies required by AI applications.

Looking Ahead: The Future of AI Fabrics

As the episode of Heavy Networking concluded, the overarching theme was one of necessary transformation. The industry is moving away from general-purpose networking and toward a specialized, purpose-built "AI Fabric." In this new paradigm, the network is no longer just a pipe; it is a critical component of the compute stack.

Technologies like Multipath Reliable Connection are essential for this transition. By solving the problems of load balancing, congestion, and reliability over standard Ethernet, MRC provides a roadmap for scaling AI clusters to sizes that were previously thought impossible. For network architects and data center operators, the message is clear: the "heavy" lifting of AI networking requires a fundamental rethink of how we move data, and MRC is at the forefront of that evolution.

The full discussion, including deeper dives into the OCP specification and the practicalities of implementing packet spraying, can be viewed on the Packet Pushers’ YouTube channel. For those interested in the technical minutiae, the OCP-MRC specification and the associated GitHub repository provide the foundational documentation for what may soon become a standard feature in high-performance data centers worldwide.