Optical/IP Networks

Internet Core Router Test

Juniper Wins!

It’s worth repeating: Juniper wins.

But it’s also worth noting: its archrival, Cisco Systems, ran a close second.

Here's the sound bite: After 10 years at the top, Cisco Systems no longer has to worry about the competition catching up. Now it has a new challenge: Playing catch-up to the performance of routers from rival Juniper Networks.

That’s the simple conclusion to be drawn from six months of arduous testing that pitted Juniper’s flagship M160 against Cisco’s brand-new 12416 in the first head-to-head comparison of 10-Gbit/s routers. It’s also the first time Cisco has agreed to let any of its gear be evaluated in an independent public test.

Actually, “test” is a pretty bland word for what would be considered cruel and unusual punishment in most states. Basically, we threw all the traffic on the Internet — and then some — at these bit-blasters in hopes of cutting through the white noise of vendor white papers. At every step of the way we were ably aided and abetted by our partners in crime: Network Test Inc. of Hoboken, N.J., a benchmarking and network design consultancy; and Spirent Communications of Calabasas, Calif., a test equipment supplier.

Here’s what we found:

Juniper’s M160 is the best of breed. It beat out Cisco’s product in three out of four overall areas: IP, MPLS, and OC192 (10 Gbit/s). Cisco and Juniper shared the honors in the fourth category: OC48 (2.5 Gbit/s) performance.

In some areas, the M160 is in a class by itself: It holds more BGP routes and more MPLS label-switched paths than any other box. It deals with network instability far better. And it exhibits much lower average latency and latency variation.

Specifically, the M160 outpaced the Cisco 12416 in seven out of the 16 individual tests offered, and tied for first with Cisco in five events (see Table 1). Where does that leave Cisco? With an impressive product that pulled ahead of Juniper in the four remaining tests.

Table 1: Results Summary
Charlotte's Networks Cisco Systems Foundry Networks Juniper Networks Winner
IP baseline tests: OC48 3* 1 4 2 Cisco
MPLS baseline tests: OC48 N/A 1 N/A 1 Cisco, Juniper
IP baseline tests: OC192 N/A 1 N/A 1 Cisco, Juniper
MPLS baseline tests: OC192 N/A 1 N/A 1 Cisco, Juniper
Longest-match lookup: OC48 N/A 1 2 2 Cisco
Longest-match lookup: OC192 N/A 1 N/A 2 Cisco
BGP table capacity N/A 2 3 1 Juniper
MPLS LSP capacity N/A 2 N/A 1 Juniper
Route flapping: OC48 N/A 1 3 1 Cisco, Juniper
Route flapping: OC192 N/A 2 N/A 1 Juniper
Convergence: OC48 N/A 2 3 1 Juniper
Convergence: OC192 N/A 2 3 1 Juniper
Filtering: OC48 N/A N/A 2 1 Juniper
Filtering: OC192 N/A N/A N/A 1 Juniper
Class of service: OC48 N/A 1 N/A 1 Cisco, Juniper
Class of service: OC192 N/A 1 N/A 2 Cisco
* Numbers represent relative ranking
N/A = Not applicable
OC48 = 2.5 Gbit/s
OC192 = 10 Gbit/s
BGP = Border gateway protocol
LSP = Label-switched path
MPLS = Multiprotocol label switching

For a yet more detailed breakdown of the test results, click here. There’s no doubt that the 12416, with its OC192c interfaces and 320-Gbit/s switching fabric, is a vast improvement over its GSR predecessor. Keep in mind that Cisco’s product is new — and thus less seasoned than Juniper’s M160, which has been shipping since spring 2000. In fact, Cisco’s new offering is just a memory upgrade and a couple of features away from being a serious threat to Juniper.

Now things get interesting.

There’s only one way for Cisco Systems Inc. (Nasdaq: CSCO) to take its second-place finish: personally. Over the past few years it’s watched Juniper Networks Inc. (Nasdaq: JNPR) walk away with market share (some would suggest that “run away” is a more apt description). Its stock, which once seemed to deny the laws of gravity, is in freefall. Cisco didn’t just want a win; it needed one. But the test results prove it’s not about to walk away from the core market.

So if Cisco and Juniper have at each other, does that mean other core router vendors stand to benefit from the bloodshed? Not in this market. True, there are a bunch of startups out there that claim they can deliver something the market leaders can’t. Unfortunately, what most can’t deliver is a core router. We issued our call for products to 11 vendors. There were only two other takers besides Cisco and Juniper: Charlotte’s Networks Ltd. and Foundry Networks Inc. (Nasdaq: FDRY).

How did they do? Let’s just say we were underwhelmed by the test results. Maybe Foundry got the message. After we’d finished testing its product, it announced that it was bailing out of the core router business. Smart move (see Foundry Retreats from the Core). This market belongs to our two top finishers for the foreseeable future.

Then again, both Foundry and Charlotte’s Networks deserve credit for having the guts to show up. Avici Systems Inc. (Nasdaq: AVCI; Frankfurt: BVC7), which is the number three core router player in terms of market share, had agreed to take part but, in the end, thought better of it — they didn't show.

Links to the individual sections:

The Core of the Problem
Building a Better Testbed
OC48 Throughput and Forwarding
OC192 Throughput
MPLS Throughput
Looking at Latency
Packet Ordering
Longest-Match Lookups
BGP Table Capacity
MPLS Tunnel Capacity
Route Flapping
Convergence Testing
Focus on Filtering
Quality of Service

Test Methodology
1 of 15
Next Page
<<   <   Page 44 / 44
jerry33 12/4/2012 | 7:54:03 PM
re: Internet Core Router Test Skeptic wrote:
"There is a difference between walking through
a lab and working with a lab that you are
missing. The lab is full of equipment yes,
but every bit of it has an owner who is
constantly fighting for more (or to keep theirs
from being taken away) and corporate people
who are constantly either trying to reduce
capital spending or trying to divert equipment
from the lab to make a customer sale."

Again, you do'nt want us to belive that, do you?
We don't talk here about a small company, and we don't talk about a new product. We are talking of an interface that Juniper is producing regularly, and if Juniper knew long enough about the test like Mr. David Newman said, Juniper could produce some extra interfaces for the test and after the test is finished she could use the interfaces for selling! isn't it?
Unless Juniper had other reasons not to do so!
That is something I would expect you to be skeptic about. Unless you have other reasons not to do so!!!

jerry33 12/4/2012 | 7:54:02 PM
re: Internet Core Router Test Skeptic wrote:
"And from what I've heard, the people who are
looking at it are not necessarly looking at
it as a core router."

Well how would you call a router that runs
OC-192 and OC-48 packets?

I would call it a core router!
Like Sisco and Juniper does!

And those people whom you are talking about, probably they suffer from visual or cognitive problem! LOL

jacobi 12/4/2012 | 7:52:08 PM
re: Internet Core Router Test Dear Testers,
I'm interested in some reliability assesment regarding the core router test. Have you noted/counted any BER (of Cell Err Rate) for each of the DUTs, except the testing for out-of-order packets ?
That is, can you distinguish between intentional packet dropping and accidental one ?
Moreover, have you tried to stress the router (i.e. pull out redundant cards etc.) so it'll have to use it's redundant data path / schedulers ?

Another question is, have you checked whether a conjested router backpressures or just drops? is there a better way of handling conjestion, and if so, did you measure the difference between the router tested ?

Shimshon Jacobi
balakrig 12/4/2012 | 7:46:29 PM
re: Internet Core Router Test Hi,
We are in the process of writing an MPLS forwarding benchmark for the network processor forum.
My question is :
How many labels were used during the MPLS testing?
dnewman 12/4/2012 | 7:45:21 PM
re: Internet Core Router Test "How many labels were used during the MPLS testing? "

Hi Ganesh,

Thanks for your inquiry. In the capacity tests, we used up to 10,000 LSPs. As noted in the article, the capacity of the devices under test is probably much higher when refresh reduction is used.

The number of labels in the baseline tests was a function of the number of interfaces on the testbed. In the OC48 testbed, each of three interfaces on each of four routers (3 * 4 = 12) forwarded traffic to each of three interfaces on three other routers (3 * 3 = 9). Ergo, we used 12 * 9 or 108 labels. In the OC48 testbed, there were 48 total edge interfaces and 36 possible destinations, so we used 48 * 36 = 1,728 labels.

Hope this helps.

David Newman
Network Test
balakrig 12/4/2012 | 7:44:18 PM
re: Internet Core Router Test Thanks a lot David.I really appreciate it.
Jeongho 12/4/2012 | 7:34:52 PM
re: Internet Core Router Test I am confusing of the facts that the differences between the test result in Charlotte's home and that in Lightreading's home comes.

Which one is latest ? The two are representing the big difference.
dnewman 12/4/2012 | 7:31:33 PM
re: Internet Core Router Test
There is exactly one set of Light Reading results; they are posted here.

Since publication of the Light Reading test, Charlotte's Web Networks has commissioned multiple tests of its own. These are vendor-sponsored projects; as with any vendor-sponsored tests, these are neither verified nor endorsed by Light Reading or Network Test.

David Newman
Network Test
<<   <   Page 44 / 44
Sign In