& cplSiteName &

40-Gig Router Test Results

Light Reading
11/30/2004
50%
50%

$500 million. Four years of R&D. 500 people.

If it took all of this for Cisco Systems Inc. (Nasdaq: CSCO) to build its CRS-1 router, what in tarnation would it take for us to test it?

Roughly speaking, this was the question that occupied the staff of European Advanced Networking Test Center AG (EANTC) when we first received the commission from Light Reading, earlier this year, to test Cisco’s (and the world’s) first ever 40-gig-capable core router.

And it turns out that our trepidations were nothing if not well founded. Testing IP traffic at 40 gigabits per second is not a straightforward task, to put it mildly. One contributor to the project described it as “trying to measure lightning bolts. In mid-air” – which we think is as good a way as any to characterize the sheer speed and power of the equipment under evaluation. Consider:

  • Developing the test methodology alone took over two months.
  • Testing partner Agilent Technologies Inc. (NYSE: A) dedicated five person-months worth of engineering in their labs in Australia and Canada to create individual test procedures matching the test plan requirements.
  • The test itself took place during a monumental, two-week, 140-hour session, involving a total of 11 engineers.
  • More than 50 Agilent OC-192 (10 Gbit/s) plus two extremely rare OC-768 (40 Gbit/s) test ports were lined up for the test. (Securing the latter was a feat in itself, since Agilent is the only analyzer vendor manufacturing them today, and fewer than ten of these modules exist worldwide.)
  • The raw EANTC test documentation consisted of Excel sheets totaling more than 700,000 lines.
In the end, all of the effort (and angst) that went into this independent verification of Cisco’s brand new CRS-1 core router was well worth it. This test represents several industry firsts: the first test of 40-Gbit/s Sonet/SDH interfaces; the first to achieve more than 1.2-Tbit/s emulated Internet packet throughput; and the first to publish independent test results of the Cisco CRS-1 at all.

So how did Cisco’s router fare? The Reader's Digest version is this: The CRS-1 performed extraordinarily well, demonstrating that it can scale to meet the requirements of service providers far into the future. It scaled to terabits-per-second of bandwidth, millions of routes, and tens of millions of IPv4 and IPv6 flows. Software upgrades interrupted the traffic for only nanoseconds – even on a fully loaded, live chassis. Its high-availability features are simply superb. And we even confirmed that its throughput capabilities scale to multi-chassis configurations.

In the opinion of EANTC, the CRS-1 proved itself to be the next milestone in router performance and scaleability.

But why test 40-gig at all?

While the IT industry may not be growing as rapidly as it once was, the Internet still is. Many large service providers have been delaying investments in new core network infrastructure in the past years – but now that much of the equipment installed during the Internet boom has been written off, and as capacity requirements continue to ramp, they have little choice but to think about deploying the next generation of backbone routers.

This is generating a demand for new monster boxes – ones that reach new levels of forwarding speed and interfaces. Products that invoke packet-over-Sonet (POS), ultra-broadband, 40-gig hardware potentially offer a solution to service providers’ needs today that is easier to manage and uses fiber infrastructure more efficiently than products that deploy 10-Gbit/s interfaces in clumps. And 40-gig has the added benefit of future-proofing carrier networks for tomorrow. While the addressable market for such technology is small today, the history of growth in telecom networks shows that it will inevitably grow in size and importance over time.

Cisco’s solution for this market, the CRS-1, employs parallel packet processing technology implemented in part within highly specialized application-specific integrated circuits (ASICs). It claims the result is an architecture that ensures non-blocking, parallel, high-availability packet processing in each card and throughout the fabric. (See Cisco’s CRS-1 chassis description for details.)

We were eager to verify Cisco's claims.

It should be mentioned that Light Reading’s original goal in undertaking this test was not to test the CRS-1 in particular, but rather to test all 40-gig-capable routing platforms, from any vendor.

However, an analysis of the available 40-gig products produced a short-list of exactly one: the Cisco CRS-1 Carrier Routing System. (Note: Avici and Juniper, Cisco’s principal competitors, both say they have 40-gig “in the works.”)

Similarly, working out which device to use to test the CRS-1 was pretty simple. The number of testers that can analyze 40-gig can also be counted on the fingers of one, well, finger: the N2X multiservice test solution from Agilent.

While identifying which product to test and which tester to use were straightforward tasks, convincing Cisco Systems to allow Light Reading to conduct a completely impartial evaluation of its Crown Jewel was anything but.

This is to be expected. Much is at stake. The CRS-1 is hugely important to Cisco and its customers. Not only is it the first 40-Gbit/s router, it also is the first Cisco router to use a completely new “carrier-class” iteration of Cisco’s IOS router code (code that ultimately will be retrofitted across other Cisco router platforms). And it also holds huge significance as Cisco’s most important attempt to date to convince carriers that it truly understands their needs and can deliver the linchpin supporting advanced telecom networks carrying next-generation voice, video, and data service traffic.

Protracted United Nations-like negotiations were undertaken over a period of several months between Cisco and Light Reading. In the great tradition of American arbitration, voices were raised, feelings were hurt, but an agreement was finally struck based on some agreed-upon ground rules, the most important of which were:

  • It was agreed that the test would take place at Cisco’s facility in San Jose, Calif., rather than EANTC’s location in Berlin (a point we ceded for the logical reason that the CRS-1 requires a big rig and a small army of technicians to move it from place to place).
  • As with all Light Reading tests, Cisco would not contribute to the funding of the test project, at all, in order to ensure its independence.
  • Prior to publication, Cisco would be allowed to review the raw test data and to point up any anomalies, but it would not be permitted to read EANTC’s analysis of its product’s performance, and it would have no “right of veto” over the publication of any results.


Most importantly of all, all parties agreed that the methodology for the test would undergo peer review by a group of leading service providers, ensuring that both the tests and the results they produce are realistic and reflective of both real-world network conditions and carrier requirements.

The complete test methodology can be downloaded here; and the service providers' feedback can be viewed here. The end result is a methodology that is not tailored or customized around the Cisco CRS-1 in particular, but can be run with any equipment matching the basic physical layer requirements.

Based on the service providers' feedback, we identified eight key areas to test that focus on both current and future service provider requirements for IP core backbones:

  • IPv4 (the current Internet protocol) forwarding performance, emulating 15 million users with realistic traffic patterns;
  • Mixed IPv4 and IPv6 (new Internet protocol) forwarding performance, verifying the router’s preparedness for the transition to the next-generation Internet protocol;
  • The impact on forwarding speed when adding security services like access control lists, logging of unauthorized traffic streams, and quality-of-service classification;
  • Routing update performance when connected to other backbone routers via eBGP (exterior border gateway protocol), exchanging more than 1.5 million unique routes;
  • Scaleability for multicast applications like TV broadcasting and enterprise point-to-multipoint applications; loading the router with a massive 70 percent to 30 percent multicast/unicast packet mix in 300 multicast groups;
  • Evaluation of the class-of-service implementation, again servicing millions of emulated users;
  • The reality of high availability (hitless) software and hardware exchange claims – a must for a core router today;
  • Finally, a Multiprotocol Label Switching (MPLS) tunnel scalability test to evaluate the capability of the system to work as a Label Switch Router inside an MPLS core network.
Of course, there were plenty of other aspects of the Cisco CRS-1 that would have been nice to test, but were not practical to undertake. In particular, testing a network of multiple Cisco CRS-1s, rather than a single unit, would have provided valuable insights into its performance in real-world conditions. But the resources needed to do this – in terms of available CSR-1s, test equipment, and time and money – made this impossible. As already noted, testing a single CRS-1 pushed limits as it was.

So how did the Cisco CRS-1 perform in all these tests? Frankly, we were amazed by its speed, scaleability, flexibility, and the all-around robustness of the system. Based on our results, the CRS-1 is truly a 100 percent carrier-grade router.

Key findings include:

  • Wire-speed IPv4 forwarding performance with 15 million emulated users (flows) on both OC-192 and OC-768 interfaces, with a total of 929 million packets per second (640 Gbit/s) in a single chassis configuration and 1.838 billion packets per second (1.28 Tbit/s) in a multi-chassis configuration (see Test Case 1)
  • Wire-speed forwarding performance with mixed 85 percent IPv4 and 15 percent IPv6 traffic (see Test Case 2)
  • Forwarding performance unaffected by adding 5,000 access lists per port, access list logging, and QOS classification (see Test Case 3)
  • Only 18 seconds to reconverge after rerouting 500,000 routes to different ports, out of a total of 1.5 million IPv4 and IPv6 routes (see Test Case 4)
  • Zero-loss multicast forwarding performance at 400-Gbit/s total multicast throughput (see Test Case 5)
  • Premium-class (strict priority) and weighted fair queuing up to the specification in a wire-speed COS test, both with fixed bandwidths and with traffic bursts (see Test Case 6)
  • Only 120 nanoseconds service interrupt during a software package upgrade; hitless hardware exchange for all but fabric card exchanges; 0.19 seconds recovery time after a fatal error of one of the eight back-plane fabrics (see Test Case 7)
  • Wire-speed forwarding over 87,000 MPLS tunnels in total (1,500 per port); including correct traffic engineering for explicit forwarding (EF) and best-effort differentiated services (see Test Case 8)
So did the CRS-1 perform flawlessly? Not quite. An instance of IPv6 packet loss during testing needed to be corrected via a micro-code fix, but from a technical point of view this is a real pick. Also, in terms of form factor, we believe the CRS-1 chassis is impractically large for use at installations where only one or two linecards are required. Industry scuttlebutt has it that Cisco is working on a half-size version of the product that will nix this issue.

This test is significant for a number of technology “firsts,” but it represents another type of “first” also: It’s the first time that Cisco has allowed Light Reading to test one of its products since a – ahem – rather controversial router test undertaken by the publication back in 2001. We believe the results of this test confirm the wisdom of this bold move on Cisco’s part. We also hope that its decision encourages other router vendors to step up to the plate for Light Reading’s 2005 program of tests, which will be announced in January.

The following pages describe the tests, and the results for each, in more detail.

Next Page: Test Setup

— Carsten Rossenhövel is Managing Director of the European Advanced Networking Test Center AG (EANTC), an independent test lab in Berlin, Germany. EANTC offers vendor-neutral network test facilities for manufacturers, service providers, and enterprises. In this role, Carsten is responsible for the design of test methods and applications. He heads EANTC's manufacturer testing and certification group.

(8)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
brahmos
50%
50%
brahmos,
User Rank: Light Beer
12/5/2012 | 1:02:08 AM
re: 40-Gig Router Test Results
I am sure many people will pick holes in the tests
but as usual csco has proved its dangerous to underestimate it.
could Alca, Lu or NT of similar size have done it ?
does anyone other than jnpr have a hope of matching
this beast?
beowulf888
50%
50%
beowulf888,
User Rank: Light Beer
12/5/2012 | 1:02:08 AM
re: 40-Gig Router Test Results
Heh, heh. I guess people are still absorbing the "shock and awe" of it all ;-).

Of course, the CRS-1 is still an HFR (sorry, please don't fine me, Michael Powell!). They're definitely not going to do volume sales with this puppy. I wonder if the real threat from Cisco to other vendors is their taking some of this technology down to their other product lines.

cheers,
--Beo
trzwuip
50%
50%
trzwuip,
User Rank: Light Beer
12/5/2012 | 1:01:59 AM
re: 40-Gig Router Test Results
hmmm, my 1$ worth here

Juniper can't make a competitive router in my opinion, QOS and m/cast on T640 are not good, 2 x 20g fabric does not constitute the ability for a real 40g port.

Sure, if you want a core box with no QOS / MCast etc then jnpr is fine, start adding these features combined with ACL's etc and watch your traffic rates die die die.

Read the recent report about 50% packet loss on the ERX
Belzebutt
50%
50%
Belzebutt,
User Rank: Light Beer
12/5/2012 | 1:01:45 AM
re: 40-Gig Router Test Results
And what does the ERX have to do with the T640.
russ4br
50%
50%
russ4br,
User Rank: Light Beer
12/5/2012 | 1:01:41 AM
re: 40-Gig Router Test Results
Read the recent report about 50% packet loss on the ERX

The ERX BRAS (ex-Unisphere) has its unique architecture and OS (JUNOSe). It has nothing in common with the T640, but for the Juniper brand.

-russ
beowulf888
50%
50%
beowulf888,
User Rank: Light Beer
12/5/2012 | 1:01:31 AM
re: 40-Gig Router Test Results
trzwuip:
Is JNPR still being dogged by that packet reordering issue? (-- discussed in previous LightReading threads) If so, they've been plagued with that problem for a couple of years now.

JNPR has been way ahead of CSCO in their support for IPv6. I gather that IPv6 forwarding in hardware is still over a year out on some of CSCO's mainline platforms. But the CRS-1 changes this.

This might just change the dynamic in the Asian markets where IPv6 support is very very important.

--Beo
ragho
50%
50%
ragho,
User Rank: Light Beer
12/5/2012 | 1:01:19 AM
re: 40-Gig Router Test Results
Actually, that's more like 1 cent worth than 1 $, if you ask me.
<ol>
<li>The competitiveness of Juniper is determined by customers, Juniper has numerous to prove it.
</li><li>I'm yet to see any vendor deliver a good multicast story. Cisco is no better, period.
</li><li>If you have some real data to share with regarding poor QoS in Junos, I'd be very interested in seeing that.
</li><li>If Cisco really made a box with 40G fabric ports, I'm sure we'll know about it. A 40G card doesn't make 40G port; most 10G cards are made today with 4x 2.5/3.125Gbps links into the fabric. 6.25/12.5Gbps serdes are still sampling these days (except for RaSer, which I know Cisco isn't using).
</li><li>The E-series is a different architecture than the M- and T-series. Please don't compare apples to oranges, or even cream cheese.
</li></ol>
My probable guess is that you haven't even touched a Juniper box, but want to claim otherwise. The ACLs in the M/T series are enforced at line rate in hardware, so your point about lowered traffic rates is just plain B/S. Get some real data and some wheaties.
Honestly
50%
50%
Honestly,
User Rank: Light Beer
12/5/2012 | 12:58:46 AM
re: 40-Gig Router Test Results
Well,
lets see. Test was done at Cisco, Agilent was trying to get anyone to sign a PO for a CRS for Cisco at Supercom and Cisco now has a program CALLED "STRATEGIC DONATION". True, They are the industry Santa Claus and are sending CRS-1's to carriers for FREE. Hey its a lot easier for Volpi to say the CRS is at carriers. Call and ask maybe they will raffle one off on LR. This then brings up the question. What did Cisco serve for dinner during the test, and was the wine good. ? Maybe we can get Zagat to give Cisco a 30, Extraodanary spending on anything that creates wishful buzz. After the CRS has run in a real production network with real traffic call me.
From The Founder
Download our complete guide to de-risking NFV deployment in 2016, including:
  • An eight-step strategy to deploying NFV safely, based on input from the companies that have already started virtualizing their production networks.
  • Interviews with leading executives at Colt, AT&T, Deutsche Telekom, Cisco, Nokia, ZTE, Ericsson and Heavy Reading.
  • Flash Poll
    Live Streaming Video
    Prepping for the Future: Upskill U Explained
    During this short kick-off video, Doug Webster, Vice President of Service Provider Marketing, Cisco, and Light Reading’s CEO & Founder Steve Saunders give an overview of Upskill U.
    LRTV Interviews
    AT&T Expert on the Key Pillars of UC

    4|29|16   |   03:58   |   (0) comments


    Vishy Gopalakrishnan, AVP of product marketing at AT&T, talks about the three developments that are making unified communications and collaboration secure and reliable for enterprise users.
    LRTV Documentaries
    LRTV Report: Mobile Core Innovation

    4|28|16   |   25:32   |   (0) comments


    Hear from multiple industry experts from Deutsche Telekom, SK Telecom, Heavy Reading, Huawei, Cisco, Ericsson, Nokia, NEC and many more about developments in the mobile core as operators virtualize their IMS and evolved packet core systems and prepare for a 5G world.
    LRTV Huawei Video Resource Center
    NFV World Congress Highlight

    4|26|16   |     |   (0) comments


    The highlight of the NFV World Congress contains exciting telecom news. Join us for an inside look at Huawei's ICT 2020 plan and its latest collaboration with industry leaders.
    LRTV Interviews
    Unified Comms Finds Its Voice

    4|25|16   |   03:44   |   (0) comments


    Peter Quinlan, VP of UCC Product Management at Tata Communications, talks about the evolution of the unified communications and collaboration services sector and how voice is now a big part of current developments.
    LRTV Documentaries
    So... What Do We Do Now?

    4|25|16   |   03:24   |   (0) comments


    After a long hiatus, Max Dingman, the CEO of a GeeGhiz, returns for a motivational board room pep talk.
    LRTV Documentaries
    NAB 2016 Highlights

    4|21|16   |     |   (0) comments


    Light Reading's Cable/Video Practice Leader Alan Breznick climbs down from the slots to tell us about the latest news in broadcast technology at NAB 2016 in Las Vegas.
    Between the CEOs
    CEO Chat: Deepfield's Craig Labovitz

    4|21|16   |     |   (0) comments


    In this latest installment of the CEO Chat series, Craig Labovitz, co-founder and CEO of Deepfield, sits down with Light Reading's Steve Saunders in Light Reading's New York City office to discuss how Deepfield fits in with the big data trend and more.
    Shades of Ray
    Leading Lights 2016: Shortlists Announced

    4|20|16   |   0:53   |   (0) comments


    The judging is over and the Leading Lights 2016 shortlists have been published -- you can see who made the cut by clicking on this link.
    LRTV Custom TV
    Introducing MulteFire – Qualcomm at MWC 2016

    4|18|16   |   3.29   |   (0) comments


    MulteFire is the latest option for using LTE in unlicensed spectrum. As oppose to its close 'siblings', LAA and LTE-U, MulteFire operates solely in unlicensed spectrum, which enables it to offer the best of two worlds – LTE-like performance with WiFi-like deployment simplicity. In this interview, Sanjeev Athalye, Sr. Director, Product Management at Qualcomm ...
    Between the CEOs
    CEO Chat: Grant Van Rooyen of Cologix

    4|18|16   |     |   (0) comments


    Grant van Rooyen, president and CEO of Cologix, sits down with Steve Saunders, founder and CEO of Light Reading, in the vendor's New Jersey facility to offer an inside look at the company's success story and discuss the importance of security in the telecom industry.
    LRTV Huawei Video Resource Center
    ONS 2016 – Demonstration of Huawei's NetMatrix Multi-Vendor SDN Orchestrator

    4|15|16   |     |   (0) comments


    This demonstration shows how Huawei's NetMatrix SDN Orchestrator (SDN-O) addresses an operator's core service agility needs for services spanning multi-domain, multivendor networks: it includes a demonstration of:
    - Rapid New Service Design: using YANG to model a complex example of multi-domain, multivendor L3VPN network connectivity service that ...
    LRTV Custom TV
    AT&T Wants to Own North Carolina

    4|15|16   |     |   (1) comment


    Venessa Harrison, president of North Carolina for AT&T, tells how the company will expand its GigaPower service beyond the seven N.C. cities it already serves.

  • This blog, sponsored by AT&T, is the second part of a ten-part series examining next-generation broadband technologies titled "Behind the Speeds."
  • Upcoming Live Events
    May 23, 2016, Austin, TX
    May 23, 2016, Austin Convention Center
    May 24-25, 2016, Austin Convention Center, Austin, TX
    September 13-14, 2016, The Curtis Hotel, Denver, CO
    December 6-8, 2016,
    June 16-18, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    A new survey conducted by Heavy Reading and TM Forum shows that CSPs around the world see the move to digital operations as a necessary part of their overall virtualization strategies.
    Hot Topics
    Ultra-Broadband Summit, Hong Kong
    Iain Morris, News Editor, 4/27/2016
    WiCipedia: Woman Cards & Bitch Switches
    Sarah Thomas, Director, Women in Comms, 4/29/2016
    FCC Poised to Re-Regulate Wholesale Access
    Carol Wilson, Editor-at-large, 4/28/2016
    Mitel Asks: What Time of Day Do You Shower?
    Mitch Wagner, West Coast Bureau Chief, Light Reading, 4/25/2016
    GoT Fans Curse HBO (Not Right) Now
    Mari Silbey, Senior Editor, Cable/Video, 4/25/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    In this latest installment of the CEO Chat series, Craig Labovitz, co-founder and CEO of Deepfield, sits down with Light Reading's Steve Saunders in Light Reading's New York City office to discuss how Deepfield fits in with the big data trend and more.
    Grant van Rooyen, president and CEO of Cologix, sits down with Steve Saunders, founder and CEO of Light Reading, in the vendor's New Jersey facility to offer an inside look at the company's success story and discuss the importance of security in the telecom industry.
    Animals with Phones
    Live Digital Audio

    Of all the tech companies in the Valley, Intel has made the most aggressive commitment to building a diverse and inclusive workplace culture. It's doing so by taking concrete, measurable steps, making a large financial investment and through a commitment to complete transparency about its progress. In this radio show, WiC Director Sarah Thomas will be joined by Shlomit Weiss, Intel's Vice President, Data Center Group, and General Manager of Networking Engineering, who will share with us why Intel is tackling this huge challenge, how and to what effect. She will also discuss her unique experiences leading development of Client SOC development in the past and today leading development of all of the chipmaker's silicon hardware for networking IPs and discrete devices and managing a team of 600 engineers across Israel, Europe and the US.