& cplSiteName &

SDN & the Problem With Data Set Transfers

Bruce Gregory
12/9/2015
50%
50%

Large data set transfers, such as large file transfers between research centers and supercomputing facilities, cause congestion that can wreak havoc on networks. Such transfers disrupt the normal traffic flow in a network, and it is difficult to accurately predict how long it will take to transmit the data (although it is usually a very long time).

Here is the issue: Since these large data set transfers are typically TCP flows that live for a long time, they fill available buffers across the network and disrupt the buffering and queuing of "regular" day-to-day traffic of a campus. This regular traffic is random, short-lived flows that can be more latency sensitive than the large flows. They don't require a lot of bandwidth, but they do need to complete in a timely fashion.

Perfect SDN application
Large data set transfers turn out to be a perfect application for SDN. First, they require the ability to provide either bandwidth reservation or dynamic bandwidth allocation that meets SLA requirements across a heterogeneous network. Second, the network still needs to accommodate the small flows of regular traffic acceptably.

This is most easily served by a logically centralized control plane that understands application requirements, network topology, capacity and current state. In other words, a software-defined network. The SDN control plane then provisions bandwidth where and when it is needed.

Not just any SDN
Any true SDN solution can provide the intelligent network control described above. Typically, a large transfer is not a random event. You know when you are about to launch a migration and can signal that information, either programmatically or through a GUI, to the SDN controller. The controller can then configure the forwarding plane to act appropriately.

The trick is doing so at the performance levels required for large production networks. For example, companies often prototype a solution using OVS, but find the solution doesn't scale when they try to roll it out to their production WAN.

Making SDN work on a 100G WAN is really hard. SDN solutions that work fine in top-of-rack applications run into scalability and manageability issues in such a demanding environment. They simply are not designed for this level of performance.

Designing SDN for performance networks
The key is to implement a programmable hardware approach that allows an SDN control plane access to a data plane that has been tailored to exactly the functionality the control plane requires. The system should leverage metering and QoS capabilities in order to enable operators to deal with these types of large flows. Operators then have the ability to define per-flow classification requirements as needed. They can also set appropriate meter rates and markers and police traffic.

When handling large flows, operators can also set up the committed and peak information rates and burst sizes to reflect the priority of the large flows and to provide a bandwidth guarantee. The large flow completes in a known time, and without disrupting the delivery of smaller flows within the regular traffic.

This is not a theoretical model; the technology exists today to design and implement an SDN system for performance networks that can pick out an individual flow from a 100G river of traffic and provide the necessary control for that flow's specific needs. This is analogous to picking a grain of sand from the middle of the desert.

We hear one thing over and over from operators: They want the ability to have very fine-grained visibility and control over their traffic physical device behavior, and they want it via open interfaces on their physical devices to allow them to progress at the pace of the industry as a whole, rather than the pace of a single equipment vendor.

The customers have spoken. Are we all listening?

— Bruce Gregory, CEO, Corsa Technology Inc.

(0)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
More Blogs from Column
As 5G networks begin to roll out, it is worth looking at how service providers can deliver high-value, high-margin security services in a way that no one else in the IoT ecosystem can match.
Operators that mainly serve prepaid customers must figure out a strategy for tackling churn.
The optical networking industry is at another inflection point, writes James Kisner, and the chalice some companies drink from next could point the way to everlasting revenue growth.
Like Lucy always does to Charlie Brown, 5G is likely another new technology that teased optical networking vendors with massive, sustained spending, then quickly yanked back expectations.
More acquisition activity in the optical space is inevitable, but it might not happen for a while until some newly coupled companies get to know each other just a little bit more.
Featured Video
Flash Poll
Upcoming Live Events
September 17-19, 2019, Dallas, Texas
October 1, 2019, New Orleans, Louisiana
October 2-22, 2019, Los Angeles, CA
October 10, 2019, New York, New York
November 5, 2019, London, England
November 7, 2019, London, UK
December 3-5, 2019, Vienna, Austria
December 3, 2019, New York, New York
All Upcoming Live Events
Partner Perspectives - content from our sponsors
Transform Beyond Borders to Lead the Innovation
By Ben Zhou, CEO, Whale Cloud
Reject Limits. Build the Future.
By David Wang, Huawei
China Telecom & Huawei Jointly Complete the World's First End-to-End 5G SA Voice & Video Call
By Jay Liu, Senior Marketing Manager, Cloud Core Product Line, Huawei Technologies
All Partner Perspectives