The ONOS Project and the Open Networking Foundation have collaborated to create the first open-source leaf-spine fabric, based on the ONOS SDN controller, OpenFlow 1.3 and bare-metal switching hardware, targeting service provider data centers, among other use cases. It is being combined with the ONOS CORD initiative to create an integrated network functions virtualization solution for service providers as well.
This is a potentially significant step for telecom operators trying to transform their data centers to be more efficient, and to convert some of their central offices into data centers. The open source approach is replacing the traditional spanning tree, with its one-to-one redundancies, from vendors such as Cisco. It uses familiar networking protocols including Ethernet and IP-MPLS, a new more efficient approach to redundancy, and bare metal switching on which a well-defined open source software stack will run.
The first use case for this new fabric is the Central Office Re-architected as a Data Center (CORD) initiative from ON.Lab, which developed ONOS , the open source software-defined networking controller used in this new open source fabric as well.
"We are already talking [to large service providers] and one of the beauties of this is that, because it is integrated with CORD, they are not just getting a leaf-spine fabric, they are getting a fully integrated solution that includes an overlay network that is doing virtualizing, service chaining and orchestrating virtual machines and instantiating VNFs inside them," notes Saurav Das, principal network architect with ONF, in an interview. "Essentially CORD is a complete NFV solution. They are almost trying out the leaf-spine fabric as a side-effect of trying out CORD."
The new fabric draws on work by the Open Networking Foundation in two projects: Atrium and SPRING-OPEN. It also incorporates work from the Open Compute Project, building on Edgecore bare-metal hardware OCP's Open Network Linux. And it uses Broadcom's OpenFlow Data Plane Abstraction (OF-DPA) API.
The spanning tree approach worked in the previous era, but can't handle the east-west traffic that dominates the data centers today, Das says. "Everyone has already recognized that the leaf-spine architecture is the one to use moving forward, just because it affords us that horizontal scaling and higher performance for east-west traffic in modern data centers," he says.
Bare metal switches are another unmistakable trend as the industry accepts the need for hardware to evolve separately from software, running some version of Linux on top of that bare metal silicon, Das says. Using SDN to drive this enables simpler control with greater flexibility, faster addition of new services and lower total cost of ownership, Das says.
The leaf and spine switches used in this new fabric are built on Accton gear, but could use any Original Design Manufacture equipment, because it uses the same software stack regardless, as shown below, that was developed as part of ONF's Project Atrium. Key to that is the open APIs from Broadcom.
"This has been used to create a leaf-spine fabric by creating a new application on top of ONOS," Das comments. Communication within the racks uses Ethernet, while communications across racks is at Layer 3. The new leaf-spine fabric also uses OpenFlow 1.3, which provides control over all the forwarding tables in switching silicon such as Broadcom's, Das says. He maintains that where traditional networking vendors are using OpenFlow 1.3, they are still using it in the same manner as they used the previous version, to address one table. By using OF-DPA, this new fabric can handle programming of all flow-tables and port groups.
This new collaboration also takes a different approach to the SDN controller, as the fabric is designed so that dataplane packets never have to go to the controller, and the controller is considered part of the network, as another network element. Using ONOS as the SDN controller, and its approach to spreading master/backup responsibility for switches around, enables redundancy to be achieved more efficiently on a one-to-three or one-to-five basis, not the traditional one-to-one.
All of that helps with the high availability, scalability and performance of the control plane, Das notes. "This is unique to ONOS -- no other SDN controller has this redundancy and scale built into it," he says. "This is what has plagued classic SDN solutions in the past, but we have overcome these challenges using ONOS in the control plane and OF-DPA in the data plane."
CORD use case
The leaf-spine fabric serves as the underlay network for CORD, but it shares the ONOS controller with the overlay network that is part of the CORD architecture as well and that common control enables a number of services, Das says.
For example, in a virtual router application, the CORD infrastructure interoperates with the outside world via a software stack such as Quagga, to talk traditional routing protocols with the metro network, "but then everything we learn is handed off to both the underlay control to program the switching hardware and to the overlay control to program the overlay network, such that this entire infrastructure behave as one giant distributed router," he says.
Other NFV solutions use a software router supporting virtual network functions, but it can't scale to the degree that the CORD option can, Das says. "That is one big benefit of the way we have designed CORD."
In a multicast solution for a residential IPTV service, the combined underlay-overlay control enables faster response and avoids unnecessary traffic through other software stacks.
On the immediate roadmap for the new open leaf-spine initiative are features such as dual-homing, support for MEF-based e-line services and VxLAN Hardware VTEP. And the community being created is expected to also develop monitoring and other software tools.
More collaborators are coming on board, including Ciena, Dell and others.
— Carol Wilson, Editor-at-Large, Light Reading