- OpenAI and tech giant partners unveiled a new data center networking protocol called MRC
- MRC is designed to reduce congestion and prevent failures via something called packet spraying
- The Ethernet-based tech is a win for the Ethernet community, which previously worried about Infiniband's data center dominance
OpenAI has spent years perfecting AI networking, and now it’s making its secret sauce – something called Multipath Reliable Connection, or MRC – available to the masses. As it turns out, the move is a huge win for the Ethernet community.
Developed with Microsoft, Nvidia, Intel, AMD and Broadcom, MRC is a protocol designed to boost the performance and resilience of AI workloads in data centers by tackling the causes of network congestion and failures. It does this via something called “packet spraying.”
Mark Handley, a member of the core networking team at OpenAI, told Fierce that packets in traditional networks are essentially assigned a single path through the network at random. This means that some paths on the network can end up with less traffic while others become overloaded. This overloading can cause congestion, which is a major problem for AI training workloads.
Instead of sending each traffic flow across one path in the network, MRC sprays each flow through hundreds of paths on the network, Handley said. This not only distributes traffic across the network more evenly – thereby avoiding congestion – but also creates redundancy that mitigates the impact if a single path fails. And the protocol is capable of scaling to clusters of 100,000 or more GPUs.
In addition to packet spraying, something called packet trimming is also employed for more efficient failure recovery, Handley said.
He added OpenAI and its partners have spent “years” working on this, but only got the first hardware that could run it last summer. It has been operational for at least six months, he said.
Nvidia disclosed in a blog that Microsoft's Fairwater data center in Wisconsin and Oracle's in Abilene, Texas, both "rely on MRC." AMD similarly said it has deployed the technology with an unnamed "leading cloud provider."
Building on Ultra Ethernet work
At this point in the story, anyone familiar with the work of the Ultra Ethernet Consortium might be thinking this all sounds a little too familiar. And that’s because it is.
Handley, who previously worked at Broadcom, was actually involved in the Ultra Ethernet Consortium. He said the technology developed by the group “is very good, but it does more than we need.”
“The techniques that we’ve taken out of it are everything that we need, but we didn’t need everything that Ultra Ethernet was doing,” he explained. “So, MRC takes some techniques which were effectually standardized in Ultra Ethernet…by being able to take this smaller step, we’re getting just the really key features that we want. We think MRC is kind of the sweet spot here.”
Ethernet revived
Though derivative, the move is a win for the Ethernet community, which used to worry that Infiniband would remain the dominant technology for data center networking.
“This does not mean InfiniBand disappears overnight. But it does show that Ethernet is rapidly evolving from a general-purpose data center technology into a serious foundation for the largest AI supercomputers,” Dell’Oro Group VP Sameh Boujelbene wrote. “A stronger Ethernet ecosystem could reduce dependency on a single networking approach, expand vendor participation, and give cloud providers and AI labs more flexibility in how they design infrastructure.”
Indeed, OpenAI isn’t keeping MRC for itself. The company and its development partners just announced the donation of MRC to the Open Compute Project. The move is partly self-serving – Handley noted that it will help accelerate the development of a vendor ecosystem. But it also will allow other players to take advantage of the tech and contribute to its future development.
Asked who else might find the technology useful, Handley pointed to hyperscalers and neoclouds as well as “anybody” doing RDMA-based applications. “I don’t think it’s limited to AI,” he said. He added MRC can be used within and between data center buildings locally, but isn’t designed for wide area networks.