& cplSiteName &

10G Ethernet Switches Pass the Test

Craig Matsumoto
News Analysis
Craig Matsumoto
1/31/2011
50%
50%

Some recent tests show that 10Gbit/s switch chips can handle high-end data center requirements. So, how long might it be before systems vendors stop doing their own ASICs?

Nick Lippis, principal analyst for Lippis Enterprises , thinks that's a valid question. His company is releasing data from a test of seven vendors' 10Gbit/s switches, and the results seem to confirm that merchant semiconductors do just fine in terms of throughput, latency, power consumption and action under duress (that is, working at 150 percent of capacity).

Lippis will be presenting his findings via WebEx on Tuesday, Feb. 1 at noon EDT; here's the link for the event.

Cisco Systems Inc. (Nasdaq: CSCO) is the vendor that's most famously stuck with its own ASICs, but Alcatel-Lucent (NYSE: ALU) and Juniper Networks Inc. (NYSE: JNPR) tend to use their own chips as well. "When those who spin their own ASICs start to see what's being done with merchant chips, we'll have to see whether they'll start to go with the Broadcom Corp. (Nasdaq: BRCM), Marvell Technology Group Ltd. (Nasdaq: MRVL) or Fulcrum Microsystems Inc. kinds of fabrics," he tells Light Reading.

Lippis had invited Light Reading for a peek at the tests, which were conducted in December at an Ixia (Nasdaq: XXIA) facility called iSimCity. Lippis had limited resources available, so some big names such as Brocade Communications Systems Inc. (Nasdaq: BRCD) and Cisco got left out, but the tests still gave an indication of how well this new generation of switches performs. (See Friday Show & Tell: Testing the New Ethernet.)

One unexpected twist in the results is that the U.S.-based companies' switches performed better than the others. For instance, the highest latency, in most test cases, went to the Voltaire Inc. (Nasdaq: VOLT) Vantage 6048.

And while every switch scored 100 percent on Layer 3 throughput tests, the Hitachi Cable Ltd. Apresia 15000-64XL-PSR was the only one to score less than 100 on Layer 2 throughput. It dipped as low as 97.3 percent throughput when dealing with 128-byte frames. Apresia was also the only box to show performance degradation during congestion tests.

Overall, though, Lippis says he was impressed by the switches' performance, especially when it came to power consumption.

Lippis's tests included switches from AlcaLu, Arista Networks Inc. , and Juniper, and top-of-rack switches from Force10 Networks Inc. , Hitachi, IBM Corp. (NYSE: IBM) (with switches from Blade Network Technologies) and Voltaire.

Lippis has a second round of testing planned for the week of April 4. Brocade, which Lippis says was interested in the December test but didn't respond in time, is a likely candidate -- and Lippis isn't shy about saying who else he'd like to include. "I'd love to get Cisco top-of-rack switches in there," he says.

— Craig Matsumoto, West Coast Editor, Light Reading

(9)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
BigBro
50%
50%
BigBro,
User Rank: Moderator
12/5/2012 | 5:14:10 PM
re: 10G Ethernet Switches Pass the Test


Sure, the shape tells you a lot about the underlying switch ASIC.


One would expect the latency of a store-and-forward switch to increase with packet size, because the ASIC has to receive the entire packet before it makes its forwarding decision.


On a cut-through switch, you'd expect the latency to be basically flat across packet size, because as soon as the ASIC has received enough of the header, it can make its forwarding decision, and start forwarding the packet (as long as the output port is not busy).


I'm not sure I understand why larger packets would have *less* latency than smaller ones. That's got to be an artifact of the ASIC, or perhaps even the test equipment: once you start getting down into the sub-microsecond range, the test equipment itself becomes a variable in your test that you shouldn't ignore. What MAC and PHY-layer hardware is at that end, for example?

spc_vancem
50%
50%
spc_vancem,
User Rank: Light Beer
12/5/2012 | 5:14:10 PM
re: 10G Ethernet Switches Pass the Test


In the article 97.3 % throughput is said to be low. However, in practical use, is this really a low number? Almost any other traffic types than constant bitrate traffic will result in long queues and full buffers when loads are rising this high. My question is: When is such a high performance actually needed? Any comments anyone?  

Pete Baldwin
50%
50%
Pete Baldwin,
User Rank: Light Beer
12/5/2012 | 5:14:10 PM
re: 10G Ethernet Switches Pass the Test


One thing that surprised me -- and I'd commented to Nick Lippis about this -- was the variety of profiles in the latency testing.  I'm not talking about the actual latency figures, but about the *shapes* of the graphs.


Lippis tested each switch on a variety of packet sizes -- 128-byte packets, 256-byte packets, etc. Some switches had good, low latency for small packet sizes and bad latency for bigger packets. Others were the other way around. Some, IIRC, were consistently flat.


It was interesting to me. It seems to imply that latency effects are rather unpredictable from switch to switch.


As for what causes these different profiles, Lippis was saying a lot of it might be the fingerprint of the chipset being used. Each vendor's different software plays a role, too.

Pete Baldwin
50%
50%
Pete Baldwin,
User Rank: Light Beer
12/5/2012 | 5:14:09 PM
re: 10G Ethernet Switches Pass the Test


> I'm not sure I understand why larger packets would have *less* latency than smaller ones.


Same here. I guess latency is just a tricky beast to wrestle.


I should specify:  Most of the switches either show a mostly flat profile (very good latency for small packets, flatly "less good" for all other sizes) or a sharp, sharp spike for ridiculously large packets (9,216 bytes).  So, some of the visual differences in the graphs come from corner cases. 


But it's true that a couple of boxes showed worse latency with small packets. I found that interesting.

Pete Baldwin
50%
50%
Pete Baldwin,
User Rank: Light Beer
12/5/2012 | 5:14:09 PM
re: 10G Ethernet Switches Pass the Test


You know, that question did occur to me.  But when everybody else is scoring perfect 100s... 97.3 is certainly low by comparison!


You've got a valid question, though, considering not all these datacenter operators will be looking for five 9s kind of performance. Anybody have any real-world experience to apply here?

cross
50%
50%
cross,
User Rank: Light Beer
12/5/2012 | 5:14:07 PM
re: 10G Ethernet Switches Pass the Test


Hi Steinarb,


The average packet size in the Internet is not that small indeed, and keeps growing due to increased video traffic and other bulk applications (see http://www.caida.org/research/.... It is certainly larger than 128 bytes in all cases - the averages measured are somewhere between 150-300 bytes.


"The" worldwide average packet size does not exist, though - it all depends where one measures and which applications dominate in each part of the world - and the situation in data centers is certainly even less uniform. That said, data centers often see an increased fraction of video and storage applications so the average packet size is going up as well according to our findings (but we have no representative proof). Some remote desktop applications (Citrix, for example) and Voice over IP generate small packet sizes around 128 bytes or even less; however, I am not aware of networks where remote desktop or VoIP traffic would drive 10GE ports to full utilization. If a remote desktop application sends bulk screen updates, it uses larger packet sizes as well.


We typically measure the 128-byte single packet size line rate throughput only if a customer explicitly requests it - since the old RFC2544 mentions this measurement as a reference.  The result is of limited value.  Sometimes service providers say they would like to check the chipset's design limitations. In fact, however, today's chipset challenges are more related to bursty traffic of variable packet sizes simultaneously sent, mixed with multicast traffic, coming from many sources and going to many destinations in an imbalanced, meshed traffic pattern.


My alarm bell rather goes off if I see a system reaching 100 % line rate at even the smallest single packet size (64 bytes for IPv4) since it is likely the chipset has been optimized for RFC2544, which does not guarantee its perfection otherwise. We have seen and published such test results in the past.  A good and competitively priced chipset needs to balance throughput requirements of artificial RFC2544 tests with those seen in real-life, complex networks. The art of lab testing is to replicate such real-life scenarios.


Best regards, Carsten (EANTC)


 

tmmarvel
50%
50%
tmmarvel,
User Rank: Light Beer
12/5/2012 | 5:13:57 PM
re: 10G Ethernet Switches Pass the Test


The major throughput and latency problems arise at multi-node network scale, rather than at individual switch scale.

An individual packet switch can be engineered to perform fine on most conditions, but is there a good way for network scale QoS control and throughput optimization in cases of meshed packet streams across multiple packet switches? The operators tend to have to resort to low average network utilization to be able to provide QoS guarantees for services over packet layer shared network 'clouds' serving multiple application/customer contracts.

The major latency, jitter and packet loss problems arise when the volumes of multiple uncoordinated packet streams exceed the capacity of shared physical links, and individual packet switch performance cannot solve these network scale QoS problems inherent to services over multi-client packet layer shared networks.

Which brings up the point of why use packet switched networks as the infra across different service contracts where QoS guarantees are a requirement? WDM/TDM as a mechanism to divide the physical infrastructure between different packet switched contracts certainly eliminates the major network scale congestion control issues. WDM/TDM also supports any packet length mixes up to continued 100% throughput transparently. Within such isolated L1/0 clouds, which can internally provide packet-switched connectivity, the packet layer QoS control will be much more manageable as it can be handled purely from each individual client's edge devices.
 
It would appear that such customer/application level isolation will deal with the bulk of the QoS and throughput problems with packet switched network services. The remaining economic issue is network scale throughput optimization for meshed packet streams, but again, 'better' packet switches do not appear to be the solution there either, as the matter to be focused on is the network scale performance.



stochasticprocess
50%
50%
stochasticprocess,
User Rank: Light Beer
12/5/2012 | 5:13:55 PM
re: 10G Ethernet Switches Pass the Test


In the report, they mention that "For store and forward DUT switches latency is defined in RFC 1242 as the time interval starting when the last bit of the input frame reaches the input port and ending when the first bit of the output frame is seen on the output port."  The latency is not increasing with frame size on store and forward switches because of how the latency is measured (essentially subtracting out the length).  Lippis couldn't have used this measurement method with the cut through switches because this would give you negative latency for jumbo frames (the first bit of output frame would be received before the last bit on the input frame).  Instead you need to do first bit in- first bit out.  Was it the case that Lippis and IXIA measured latency differently for cut through and store and forward switches?

BigBro
50%
50%
BigBro,
User Rank: Moderator
12/5/2012 | 5:11:57 PM
re: 10G Ethernet Switches Pass the Test


This post explains why latency goes down for larger packets:


http://www.fulcrummicro.com/bl...


"Frame processing time is masked for larger packets. As can be seen in some of the results, the latency gets lower as the size of the packet gets larger. Since the latency clock starts after the last bit arrives, large packets have plenty of time to process the frame header before the first bit is seen on the output port.  With small packets, even after the last bit arrives, the output must wait until frame header processing is complete before the first bit is seen on the output port."

Light Reading’s Upskill U is a FREE, interactive, online educational resource that delivers must-have education on themes that relate to the overall business transformation taking place in the communications industry.
NEXT COURSE
Wednesday, August 3, 1:00PM EDT
The Central Office Re-Architected as a Data Center
Guru Parulkar, Executive Director, Open Networking Research Center, Open Networking Lab
UPCOMING COURSE SCHEDULE
Wednesday, August 10, 1:00PM EDT
Telcos & Open Source 101
Phil Robb, Senior Technical Director, OpenDaylight
Friday, August 12, 1:00PM EDT
The Role of Open Source in NFV
Jim Fagan, Director, Cloud Practice, Telstra
Wednesday, August 17, 1:00PM EDT
Using Open Source for Data Centers and Cloud Services
Roz Roseboro, Senior Analyst, Heavy Reading
in association with:
From The Founder
Light Reading today starts a new voyage as part of a larger Enterprise.
Flash Poll
Live Streaming Video
Charting the CSP's Future
Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
LRTV Custom TV
Reinventing Operations for a Virtual, Software-Defined World

7|28|16   |   5:23   |   (0) comments


Heavy Reading Senior Analyst Jim Hodges speaks with Accenture's Larry Socher and Matt Anderson about what service providers must do to transform their business to get the benefits of SDN and NFV including: leveraging DevOps, introducing real-time OSS and implementing analytics.
Women in Comms Introduction Videos
Fujitsu Sales Leader Shares Lessons Learned

7|27|16   |   5:12   |   (0) comments


As Fujitsu's only female sales leader, Annie Bogue knows the importance of asking for what you want, being flexible (she's been relocated five times), keeping a meticulous calendar, 'leaning in,' working harder than everyone else around you, being aware and more.
Telecom Innovators Video Showcase
VeEX Test & Measurement Solutions

7|25|16   |   08:57   |   (0) comments


Cyrille Morelle, president and CEO of VeEX Inc., talks test and measurement with Light Reading's Steve Saunders at BCE 2016. This includes innovative products such as VeSion Cloud-Based platform for network monitoring; MTTplus Modular Test platform for Access, Business, Carrier Ethernet, Transport and Core services; and OPX-BOX+ for Fiber Optics.
LRTV Custom TV
VeEX: Live From BCE 2016

7|25|16   |   03:20   |   (0) comments


VeEX's Senior Director of Business Development, Perry Romano, explains how VeEX provides tools to help install, maintain, monitor and manage network infrastructure efficiently and effectively. The portfolio of products on display include the RXT-6000, MTTplus and TX300s.
LRTV Custom TV
Real-Time Telemetry & Analytics for Intelligent SDN Orchestration

7|25|16   |   03:09   |   (0) comments


Packet Design CEO Scott Sherwood discusses how real-time network telemetry and analytics are enabling a new breed of SDN orchestration applications.
From the Founder
The Russo Report: Driving Disruption

7|25|16   |   07:44   |   (0) comments


In the first episode of a four-part series, Light Reading Founder and CEO Steve Saunders and Calix President and CEO Carl Russo drive around town discussing the disruptive mega-changes in the communications industry and where hope lies for service providers to meet the escalating demands of the cloud.
LRTV Custom TV
NetScout: Maximizing Enterprise Cloud for Digital Transformation

7|20|16   |   04:53   |   (0) comments


Light Reading Editor Mitch Wagner talks to NetScout CMO Jim McNiel about maximizing the benefits of enterprise cloud and digital transformation while minimizing potential pitfalls with a proper monitoring and instrumentation strategy.
Women in Comms Introduction Videos
Ciena's VP Offers a Career Crash Course

7|20|16   |   4:14   |   (2) comments


How did Ciena's Vice President of Sales, Angela Finn, carve out her career path? Simple, she tells WiC. She stayed true to her company, customers and principles. She shares her advice for women on how to be authentic and credible, as well as for companies that want to make a real change to their culture and practices.
LRTV Custom TV
NFV in 2016: Part 2 – Climbing the Virtualization Maturity Curve

7|19|16   |   06:56   |   (0) comments


Many of the initial use case implementations are single-vendor and self-contained. The industry is still climbing the virtualization maturity curve, needing further clarity and stability in the NFV infrastructure (NFVi) and greater availability and choice of virtualized network functions (VNFs). Interoperability between NFVis and VNFs from different vendors ...
Telecom Innovators Video Showcase
Versa Networks' Kumar Mehta on SD-WAN Managed Services

7|19|16   |     |   (0) comments


In Silicon Valley, Steve Saunders sits down with Versa's Kumar Mehta for an interview focused on why service providers are building SD-WAN managed services, and how Versa's telco customers are innovating.
LRTV Custom TV
Juniper Networks & The Evolution of NFV

7|19|16   |   06:01   |   (0) comments


Senior Juniper Networks executives talk to Light Reading Founder & CEO Steve Saunders about NFV developments and the recent independent evaluation by test lab EANTC of Juniper's Cloud CPE solution.
LRTV Interviews
CenturyLink Goes Beyond Managed WiFi

7|19|16   |     |   (0) comments


CenturyLink's managed WiFi allows enterprises, such as retailers and resorts, to track guest WiFi usage in order to help them better communicate with customers.
Upcoming Live Events
September 13-14, 2016, The Curtis Hotel, Denver, CO
November 3, 2016, The Montcalm Marble Arch, London
November 30, 2016, The Westin Times Square, New York City
December 6-8, 2016,
May 16-17, 2017, Austin Convention Center, Austin, TX
All Upcoming Live Events
Infographics
Five of the Top 10 most targeted countries in Check Point Software Technologies' global Malware & Threat Index for Q1 2016 are in Africa.
Hot Topics
Verizon Sports Big Plans for Yahoo
Alan Breznick, Cable/Video Practice Leader, Light Reading, 7/26/2016
Light Reading Beyond
Steve Saunders, CEO and founder, Light Reading, 7/28/2016
Yahoo Signing Off in $4.83B Sale to Verizon
Mari Silbey, Senior Editor, Cable/Video, 7/25/2016
Are IP/Optical Integration Initiatives Moving to Silos?
Faisal Khan, Network Operator Technology Planning Dept., 7/28/2016
Ericsson Board Has Been Asleep at the Wheel – Consultant
Ray Le Maistre, Editor-in-chief, 7/25/2016
Like Us on Facebook
Twitter Feed
BETWEEN THE CEOs - Executive Interviews
There's no question that, come 2020, 5G technology will turn the world's conception of what mobile networking is on its head. Within the world of 5G development, Dr. ...
I've enjoyed interviewing many interesting people since I rejoined Light Reading, but William A. "Bill" Owens certainly takes the biscuit, as we say where I come from.
Animals with Phones
Live Digital Audio

Our world has evolved through innovation from the Industrial Revolution of the 1740s to the information age, and it is now entering the Fourth Industrial Revolution, driven by technology. Technology is driving a paradigm shift in the way digital solutions deliver a connected world, changing the way we live, communicate and provide solutions. It can have a powerful impact on how we tackle some of the world’s most pressing problems. In this radio show, Caroline Dowling, President of Communications Infrastructure & Enterprise Computing at Flex, will join Women in Comms Director Sarah Thomas to discuss the impact technology has on society and how it can be a game-changer across the globe; improving lives and creating a smarter world. Dowling, a Cork, Ireland, native and graduate of Harvard Business School's Advanced Management Program, will also discuss her experience managing an international team focused on innovation in an age of high-speed change.