40-Gig Router Test Results

$500 million. Four years of R&D. 500 people.

If it took all of this for Cisco Systems Inc. (Nasdaq: CSCO) to build its CRS-1 router, what in tarnation would it take for us to test it?

Roughly speaking, this was the question that occupied the staff of European Advanced Networking Test Center AG (EANTC) when we first received the commission from Light Reading, earlier this year, to test Cisco’s (and the world’s) first ever 40-gig-capable core router.

And it turns out that our trepidations were nothing if not well founded. Testing IP traffic at 40 gigabits per second is not a straightforward task, to put it mildly. One contributor to the project described it as “trying to measure lightning bolts. In mid-air” – which we think is as good a way as any to characterize the sheer speed and power of the equipment under evaluation. Consider:

• Developing the test methodology alone took over two months.
• Testing partner Agilent Technologies Inc. (NYSE: A) dedicated five person-months worth of engineering in their labs in Australia and Canada to create individual test procedures matching the test plan requirements.
• The test itself took place during a monumental, two-week, 140-hour session, involving a total of 11 engineers.
• More than 50 Agilent OC-192 (10 Gbit/s) plus two extremely rare OC-768 (40 Gbit/s) test ports were lined up for the test. (Securing the latter was a feat in itself, since Agilent is the only analyzer vendor manufacturing them today, and fewer than ten of these modules exist worldwide.)
• The raw EANTC test documentation consisted of Excel sheets totaling more than 700,000 lines.

In the end, all of the effort (and angst) that went into this independent verification of Cisco’s brand new CRS-1 core router was well worth it. This test represents several industry firsts: the first test of 40-Gbit/s Sonet/SDH interfaces; the first to achieve more than 1.2-Tbit/s emulated Internet packet throughput; and the first to publish independent test results of the Cisco CRS-1 at all.

So how did Cisco’s router fare? The Reader's Digest version is this: The CRS-1 performed extraordinarily well, demonstrating that it can scale to meet the requirements of service providers far into the future. It scaled to terabits-per-second of bandwidth, millions of routes, and tens of millions of IPv4 and IPv6 flows. Software upgrades interrupted the traffic for only nanoseconds – even on a fully loaded, live chassis. Its high-availability features are simply superb. And we even confirmed that its throughput capabilities scale to multi-chassis configurations.

In the opinion of EANTC, the CRS-1 proved itself to be the next milestone in router performance and scaleability.

But why test 40-gig at all?

While the IT industry may not be growing as rapidly as it once was, the Internet still is. Many large service providers have been delaying investments in new core network infrastructure in the past years – but now that much of the equipment installed during the Internet boom has been written off, and as capacity requirements continue to ramp, they have little choice but to think about deploying the next generation of backbone routers.

This is generating a demand for new monster boxes – ones that reach new levels of forwarding speed and interfaces. Products that invoke packet-over-Sonet (POS), ultra-broadband, 40-gig hardware potentially offer a solution to service providers’ needs today that is easier to manage and uses fiber infrastructure more efficiently than products that deploy 10-Gbit/s interfaces in clumps. And 40-gig has the added benefit of future-proofing carrier networks for tomorrow. While the addressable market for such technology is small today, the history of growth in telecom networks shows that it will inevitably grow in size and importance over time.

Cisco’s solution for this market, the CRS-1, employs parallel packet processing technology implemented in part within highly specialized application-specific integrated circuits (ASICs). It claims the result is an architecture that ensures non-blocking, parallel, high-availability packet processing in each card and throughout the fabric. (See Cisco’s CRS-1 chassis description for details.)

We were eager to verify Cisco's claims.

It should be mentioned that Light Reading’s original goal in undertaking this test was not to test the CRS-1 in particular, but rather to test all 40-gig-capable routing platforms, from any vendor.

However, an analysis of the available 40-gig products produced a short-list of exactly one: the Cisco CRS-1 Carrier Routing System. (Note: Avici and Juniper, Cisco’s principal competitors, both say they have 40-gig “in the works.”)

Similarly, working out which device to use to test the CRS-1 was pretty simple. The number of testers that can analyze 40-gig can also be counted on the fingers of one, well, finger: the N2X multiservice test solution from Agilent.

While identifying which product to test and which tester to use were straightforward tasks, convincing Cisco Systems to allow Light Reading to conduct a completely impartial evaluation of its Crown Jewel was anything but.

This is to be expected. Much is at stake. The CRS-1 is hugely important to Cisco and its customers. Not only is it the first 40-Gbit/s router, it also is the first Cisco router to use a completely new “carrier-class” iteration of Cisco’s IOS router code (code that ultimately will be retrofitted across other Cisco router platforms). And it also holds huge significance as Cisco’s most important attempt to date to convince carriers that it truly understands their needs and can deliver the linchpin supporting advanced telecom networks carrying next-generation voice, video, and data service traffic.

Protracted United Nations-like negotiations were undertaken over a period of several months between Cisco and Light Reading. In the great tradition of American arbitration, voices were raised, feelings were hurt, but an agreement was finally struck based on some agreed-upon ground rules, the most important of which were:

• It was agreed that the test would take place at Cisco’s facility in San Jose, Calif., rather than EANTC’s location in Berlin (a point we ceded for the logical reason that the CRS-1 requires a big rig and a small army of technicians to move it from place to place).
• As with all Light Reading tests, Cisco would not contribute to the funding of the test project, at all, in order to ensure its independence.
• Prior to publication, Cisco would be allowed to review the raw test data and to point up any anomalies, but it would not be permitted to read EANTC’s analysis of its product’s performance, and it would have no “right of veto” over the publication of any results.

The Cisco CRS-1 (center rack) is rather large, but the stacks of
Agilent N2X (left and right racks) match its size