Light Reading

Are You Really Getting 400 Gbit/s Performance?

Sterling Perrin
9/11/2013
0%
100%

Driven by continuing IP traffic growth, a new generation of 400 Gbit/s network processors is beginning to emerge. Such processors have been announced by a handful of vendors, with more to come.

However, scaling from 100 Gbit/s to 400 Gbit/s processors is not an easy feat. Recently, we witnessed some 400 Gbit/s network processor performance testing in a lab and were surprised by some of the results. Based on this data, we believe that there may be some wide variations between what is claimed on spec sheets and the real world performance that is achievable.

We believe that service providers evaluating new higher capacity systems must perform extensive testing on their own in order to ensure that the high-end systems perform as advertised under a variety of conditions that mirror real-world traffic mixes and traffic growth. Otherwise, service providers may be stuck with true performance that is far below the advertised 400G sticker. In fact, under certain conditions, true performance may not even be better than the legacy routers that are being replaced.

One important test is the packet sweep test. The packet sweep tests router performance over a spectrum of different packet sizes, from small (64 bytes) to large (1,500 bytes or greater), in order to ensure consistent performance across the full range of incoming packet variants. It is somewhat counterintuitive, but the greatest demands on the processor occur at the smallest packet sizes. The reason is that small packet sizes force greater amounts of table look-ups per second, and this burdens the processor.

The test we reviewed showed a 400 Gbit/s processor consistently dropping packets at all sizes, both small and large. As the packet size increased, the performance did not improve. On a few occasions, packets dropped by as much as 50 percent, and we never saw more than 80 percent throughput at any point in the test.

A second test, known as the Internet mix (or IMIX) test, was reviewed. Just as there is no typical packet size, there is no typical service provider IP traffic mix. It varies greatly from provider to provider. The test we reviewed used a couple of different real-world IMIX samples provided by service providers, based on their network scenarios. The results showed poor performance across a range of IMIX profiles. The processor appeared to suffer from problems related to storing packets for lookup, which resulted in severe performance impact on all traffic types tested. Significantly, when comparing two generations of processor performance, the new 400 Gbit/s processor demonstrated anywhere from 20 percent to 50 percent less performance based on serviceable bandwidth.

A third test measured the performance of a processor while enabling service level agreements (SLAs). Here, the results showed that enabling SLAs on connections reduced the actual throughput of the processor from 400 Gbit/s to 100 Gbit/s, meaning that only 50 percent of the advertised 100 GigE ports would be available with SLAs turned on.

The tests above are tied closely to the conventional views of scale, meaning the ability to handle more and more bits through the processor and on the network. As the Internet evolves from person-to-person communications to the machine-to-machine dominated Internet of things, another component of scale is becoming increasingly important: the ability to handle more and more flows through the network.

For next-generation routers, this means they must not only process massive amounts of bits, but they must also be able to process massive numbers of flows. Routers equipped with 400 Gbit/s processors must be tested in their abilities to handle tens of thousands of flows. As a rule of thumb, a 10 GigE port will typically serve 500 customers. If we assume just four classes of service per customer (a low assumption), this creates 2,000 different flows per 10 GigE port. With 20 x 10 GigE ports per line card (the state-of-the-art in existing designs), this creates 40,000 different flows per card. As we move to an Internet of things, it is very possible that these next-gen routers could hit performance limits on supported flows long before they hit their maximum capacities in bit/s. It is another dimension of scale that must be accounted for and tested in evaluating core routers.

The final point we will touch upon is efficiency in power consumption and footprint. It is well understood that these opex factors are of critical importance to service providers. In some regions, where power costs are well above global averages or where equipment is deployed in dense urban areas where space is severely limited, power and space requirements may make or break a buying decision. Even in the US, however, we have had discussions with large service providers who place power and space at the top of their lists.

Here again, service providers need to dig deeper than the power and space requirements that are advertised. To get a true understanding of space and power, these advertised specs must be placed in the context of the overall performance of the system -- as determined by packet sweep tests, IMIX tests, flow limitations, and any other tests that are performed on the system.

For example, if performance testing and evaluations show that a system will perform at 80 percent of its capacity limits under real-world network conditions, then this needs to be taken into account for space and power consumption. In this case, 20 percent more space and 20 percent greater power consumption would be needed to achieve 400 Gbit/s of capacity. Failing to account for any performance limitations sets service providers up for unwelcome opex surprises when these new systems start to fill up in the network.

In summary, vendors rolling out new 400 Gbit/s silicon promise routers that are smarter, faster, and greener than previous generations, but some in-depth test results we've seen indicate that this may not be the case. Service providers cannot afford to rely on vendor claims in this area. Rather, extensive testing -- of the kind we’ve described in this article -- is a must. The added time and costs of upfront testing will pay dividends for years to come.

This blog was commissioned by Cisco Systems. The blog was created independently of Cisco, and Heavy Reading is wholly responsible for its contents.

— Sterling Perrin, Senior Analyst, Heavy Reading


Interested in learning more on this topic? Then come to Ethernet & SDN Expo, a Light Reading Live event that takes place on October 2-3, 2013 at the Javits Center in New York City. Co-located with Interop, Light Reading's Ethernet & SDN Expo will focus on how the convergence of Carrier Ethernet 2.0 with emerging carrier software-defined networking (SDN) and network functions virtualization (NFV) technologies could change the whole telecom landscape for service providers. For more information, or to register,
click here
.


(15)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
Page 1 / 2   >   >>
mfaisalk
50%
50%
mfaisalk,
User Rank: Lightning
9/14/2013 | 11:05:32 PM
400G performance
We need to be careful with our judgements here. It is one vendor testing another vendor. What good you can expect here ?

 

Faisal
David Burns
50%
50%
David Burns,
User Rank: Light Beer
9/13/2013 | 2:05:16 PM
Check the math
Quote (snipped for conciseness):
if performance testing and evaluations show that a system will perform at 80 percent of its capacity limits under real-world network conditions ... 20 percent more space and 20 percent greater power consumption would be needed to achieve 400 Gbit/s of capacity.

Wrong. If actual performance is 20% lower, you need to provision 25% more equipment to achieve the targeted performance. Simple example: 8 x 400 Gbit/s should give you 3200 Gbit/s. If processors perform at 80%, actual performance of one processor is 320 GBit/s, so you need 10 to reach your target - 25% more than 8.
desiEngineer
50%
50%
desiEngineer,
User Rank: Light Beer
9/13/2013 | 1:17:08 PM
Re: Which network processor tests?
Sterling,

If you can guarantee that the tests were fair and impartial, then yes, enquiring minds want to know.

If not, don't attach your name (and LightReading's), and then hide behind some "this is newsworthy" BS.  And I don't care how sophisticated and expensive the lab is, I am not ingenuous enough to believe that the lab was created for the purpose of the general public.  Clue#1: I haven't seen one report come out of that lab that showed a single Cisco product in a negative light.

No vendor has a vested interest in impartiality.  Knowing how much marketing is attached to all claims, it behooves the operators to do their due diligence, but to pretend that Cisco can do that due diligence for all of us is naive at best.

-desi
Surfer Blue
50%
50%
Surfer Blue,
User Rank: Light Beer
9/13/2013 | 12:54:07 PM
Re: Which network processor tests?
Service providers do extensive testing before deploying any new hardware in their networks.  FP3 cards have been available for a couple of years and are widely deployed – clearly they have been thouroughly vetted in the field.  Personally I'd take real world testing and deployment experience over contrived competitor lab testing any day.  

 
sterlingperrin
50%
50%
sterlingperrin,
User Rank: Lightning
9/13/2013 | 9:36:34 AM
Re: Which network processor tests?
Yarn,

You are correct - I have no interest in standing in the middle. Here's the thing. Cisco has a test lab that would be the envy of all but the very largest tier 1 operators. They conduct very thorough and very expensive testing of their competitors' products, which they've been doing for 13 years (probably not something they want publicized). In this case, they came to us - and some other media as well - with information from their labs that was compelling and newsworthy.

Are we saying that Cisco's results are conclusive about ALU, or any particular competitor? Absolutely not!! But we do believe there was enough in there to raise broad questions and write this piece.

I have received calls from people at ALU who are understandably upset. They are getting calls from customers who are asking questions. It's not an easy thing to go through, I'm sure, but as media, it is our job to bring the right topics of debate to the forefront and spark the dialogue. We made a judgment that 400G performance testing is a legitimate topic and that questions should be asked of all 400G NP players - ALU, Cisco, Huawei, Juniper, and any other market entrants that emerge.

Sterling

 
mendyk
50%
50%
mendyk,
User Rank: Light Sabre
9/12/2013 | 7:28:34 PM
Re: Which network processor tests?
We note the sponsorship of the blog for full-disclosure purposes. We also note that Heavy Reading is wholly responsible for the content. The post by Sterling calls attention to the fact that performance claims often do not pass scrutiny, which means operators should do their own evaluations before making purchase decisions.
tb100
50%
50%
tb100,
User Rank: Moderator
9/12/2013 | 6:01:11 PM
Re: Which network processor tests?
Wait a minute. Did you just say, and I quote: "It is Cisco's testing of Alcatel-Lucent's 400G processor"? (and 'blog was commissioned by Cisco Systems')?

So Cisco paid you to write an article that shows the 'results' of their testing of a competitor's product? Aren't the results pretty much predetermined, no matter what the capabilities of the product?

 

I am at a loss for words....
yarn
100%
0%
yarn,
User Rank: Light Sabre
9/12/2013 | 2:06:22 PM
Re: Which network processor tests?
Sterling,

Why lend your name to help Cisco get credibility for this type of quasi-technical FUD bs articles they're so well known for? Now you're caught in the middle and have to stand up for their crap, and tarnish your own credibility in the process.  
mendyk
50%
50%
mendyk,
User Rank: Light Sabre
9/12/2013 | 11:53:53 AM
Re: Which network processor tests?
All product testing -- including tests from independent labs -- should come with a big "grain of salt" warning. And in the case of vendor-run tests, results are clearly subject to question, especially from competing suppliers who had no influence on how the testing was done. From the point of view of prospective customers, though, it's important to understand when performance claims come with an asterisk or two.
sterlingperrin
50%
50%
sterlingperrin,
User Rank: Lightning
9/12/2013 | 11:11:34 AM
Re: Which network processor tests?
Manish,

Correct, we thought there was value in the piece as a call for due diligence.

If service providers are testing these systems at their limits across scenarios applicable to their network (including growth and possible changes to traffic patterns, etc.) then they are covered.

400 Gbit/s is very new. I don't how much testing has been done by anyone yet.

Sterling

 
Page 1 / 2   >   >>
More Blogs from Heavy Lifting Analyst Notes
Most telecom and network equipment manufacturers are now shipping products with 100G ports.
In part 2 in a series of blogs on 'Accelerating NFV Implementation,' Caroline Chappell looks at the long-term vision for NFV and how operators can get there.
In part 1 in a series of blogs on 'Accelerating NFV Implementation', Caroline Chappell looks at why OpenStack is set to play a key role in the NFV Management and Orchestration (MANO) stack.
What are the biggest SDN questions facing service providers today? Heavy Reading's Sterling Perrin highlights the major talking points as he prepares for a full day's debate in Denver.
As the quest toward the Gigabit City continues, it is becoming clearer that public-private initiatives may be the key to success.
Flash Poll
From The Founder
Last week I dropped in on "Hotlanta," Georgia to moderate Light Reading's inaugural DroneComm conference – a unique colloquium investigating the potential for drone communications to disrupt the world's telecom ecosystem. As you will see, it was a day of exploration and epiphany...
LRTV Documentaries
Cable Eyeing SDN for Headend, Home Uses

5|26|15   |   05:57   |   (1) comment


CableLabs is looking at virtualizing CMTS and CCAP devices in the headend, as well as in-home devices, says CableLabs' Karthik Sundaresan.
LRTV Documentaries
Verizon's Emmons: SDN Key to Cost-Effective Scaling

5|22|15   |   03:53   |   (0) comments


For Verizon and other network operators to ramp up available bandwidth cost effectively, they need to move to SDN and agree on how to do that.
LRTV Documentaries
Lack of Universal SDN a Challenge

5|21|15   |   04:51   |   (3) comments


Heavy Reading Analyst Sterling Perrin talks about how uncertainty about SDN standards and approaches may be slowing deployment.
LRTV Custom TV
Steve Vogelsang Interview: Carrier SDN

5|20|15   |   05:02   |   (0) comments


Sterling Perrin speaks to Steve Vogelsang, Alcatel-Lucent CTO for IP Routing & Transport business, about the new Carrier SDN-enabling Network Services Platform and the operator challenges it solves.
LRTV Custom TV
Carrier SDN: On-Demand Networks for an On-Demand World

5|20|15   |   20:52   |   (0) comments


Steve Vogelsang, Alcatel-Lucent CTO for IP Routing & Transport business, talks about requirements and benefits of Carrier SDN during the keynote address at the Light Reading Carrier SDN event May 2015.
LRTV Documentaries
The Security Challenge of SDN

5|19|15   |   02:52   |   (0) comments


CenturyLink VP James Feger discusses concerns that virtualization could create new vulnerabilities unless network operators build in safeguards.
LRTV Custom TV
NFV Elasticity – Highly Available VNF Scale-Out Architectures for the Mobile Edge

5|18|15   |   5:50   |   (0) comments


Peter Marek and Paul Stevens from Advantech Networks and Communications Group talk about their NFV Elasticity initiative and the company's latest platforms for deploying virtual network functions at the edge of the network. Packetarium XL and the new Versatile Server Module: 'designed to reach parts of the network that other servers cannot reach.'
LRTV Huawei Video Resource Center
Bay Area Spark Meetup 2015

5|14|15   |   3:54   |   (0) comments


Developed in 2009, Apache Spark is a powerful open source processing engine built around speed, ease of use and sophisticated analytics. This spring, Huawei hosted a meetup for Spark developers and data scientists in Santa Clara, California. Light Reading spoke with organizers and attendees about Huawei's code contributions and long-term commitment to Spark.
LRTV Custom TV
The Transport SDN Buzz

5|12|15   |   06:01   |   (1) comment


Sterling Perrin, senior analyst at Heavy Reading, speaks with Peter Ashwood-Smith of Huawei and Guru Parulkar of ON.Lab about the evolution of transport SDN and the integration of technologies.
LRTV Custom TV
Next-Generation CCAP: Cisco cBR-8 Evolved CCAP

5|5|15   |   04:49   |   (0) comments


John Chapman, Cisco's CTO of Cable Access Business Unit and Cisco Fellow, explained the innovation design of Cisco's cBR-8, the industry's first Evolved CCAP, including DOCSIS 3.1 design from ground-up, distributed CCAP with Remote PHY and path to virtualization. Cisco's cBR-8 Evolved CCAP is the platform that will last through the transitions.
LRTV Custom TV
Meeting the Demands of Bandwidth & Service Group Growth

5|1|15   |   5:35   |   (0) comments


Jorge Salinger, Comcast's Vice President of Access Architecture, explains how DOCSIS 3.1 and multi-service CCAP can meet the demands of the bandwidth and service group growth.
LRTV Custom TV
DOCSIS 3.1: Transforming Cable From Hardware-Defined Network to Software-Defined Network

4|29|15   |   03:48   |   (0) comments


John Chapman, Cisco's CTO of Cable Access Business Unit and Cisco Fellow, explains how DOCSIS 3.1 can transform cable HFC network to a more agile software-defined network.
Upcoming Live Events
June 8, 2015, Chicago, IL
June 9, 2015, Chicago, IL
June 9-10, 2015, Chicago, IL
June 10, 2015, Chicago, IL
September 29-30, 2015, The Westin Grand Müchen, Munich, Germany
October 6, 2015, The Westin Peachtree Plaza, Atlanta, GA
October 6, 2015, Westin Peachtree Plaza, Atlanta, GA
All Upcoming Live Events
Infographics
Procera has gathered facts, stats and customer experience feedback from a survey of 540 users from across the globe.
Hot Topics
10 Alternate Uses for Tablets
Eryn Leavens, Copy Desk Editor, 5/22/2015
Bidding War for TWC Looks Likelier
Alan Breznick, Cable/Video Practice Leader, 5/22/2015
Eurobites: Alcatel-Lucent Trials 400G in Czech Republic
Paul Rainford, Assistant Editor, Europe, 5/26/2015
Comcast Targets 6 New Gigabit Markets
Mari Silbey, Senior Editor, Cable/Video, 5/21/2015
Potholes Lurk in Indian Smart City Project
Gagandeep Kaur, Contributing Editor, 5/22/2015
Like Us on Facebook
Twitter Feed
BETWEEN THE CEOs - Executive Interviews
On May 29th 10 AM ET, Steve Saunders, founder and CEO of Light Reading, will be drilling into the "pains and gains" of NFV with Saar Gillai, SVP & GM for NFV at Hewlett-Packard Co. (NYSE: HPQ) (HP). He has defined a four-step NFV model describing a sequence of technology innovation. It's a must-read doc for any network architect looking to get to grips with their NFV migration strategy. Join us for the interview, and the chance to ask Saar your NFV questions directly!
With 200 customers in 60 countries, Stockholm-based Net Insight has carved out a solid leadership position in one of the hottest vertical markets going in comms right now: helping service providers and broadcasters deliver video and other multimedia traffic over IP networks. How has Net Insight managed to achieve this success in the face of immense competition from the industry giants?
My ongoing interview tour of the leading minds of the telecom industry recently took me to Richardson, Texas, where I met with Rod Naphan, CTO and SVP, Solutions, ...
Cats with Phones
Too Fluffy to Talk Click Here
Elmer found that his bountiful fur got in the way of meaningful conversation.