Juniper Wins Monster Router Test

Juniper Networks Inc. (Nasdaq: JNPR) has defeated its number one rival, Cisco Systems Inc. (Nasdaq: CSCO), in the first multivendor test of Internet core routers.

The independent evaluation, which was commissioned by Light Reading and took six months to complete, proves that Juniper’s M160 platform is currently superior to Cisco's latest 12416 product in three key performance areas: IP (Internet protocol), MPLS (multiprotocol label switching), and OC192 (10 Gbit/s). The vendors’ products are evenly matched in the performance of their OC48 (2.5 Gbit/s) interfaces. (Click here to view the complete report).

The tests were performed for Light Reading by Network Test Inc., an independent benchmarking and network design consultancy. “In some areas Juniper’s M160 is in a class by itself,” says David Newman, president of Network Test.

His report on the test results concludes that:

“[The M160] holds more BGP (border gateway protocol) routes and more MPLS label-switched paths than any other box. It deals with network instability far better. And it exhibits much lower average latency -– the amount of delay a router introduces -– and latency variation.”

Despite losing to Juniper in three out of four overall areas, the test results also contained good news for Cisco. For example, the data demonstrates that its OC192 interfaces not only exist -– but can also process a torrent of data at line rate. Indeed, Cisco’s 12416 turned in the highest single data rate achieved in the entire test: more than 271 million packets per second.

With improvements, Cisco’s router could represent serious competition for Juniper. “Cisco has served notice that it’s no longer the easy target that allowed Juniper to gain 30 percent share in just a few years,” says Network Test’s Newman.

In contrast, the test results turned in by the other two vendors tested -- Charlotte’s Networks Inc. and Foundry Networks Inc. (Nasdaq: FDRY) -- were all cloud, no silver lining. Charlotte’s Networks’ Aranea-1 fumbled packets at every level of loading offered – including one percent. Foundry’s Netiron pretty much gave up the ghost in the flapping and convergence test (which might explain why, since the tests, the company has announced that it will withdraw from the core router market).

Results such as these will do little to encourage the belief that other vendors will be able to do anything to weaken Cisco and Juniper’s stranglehold on the market for Internet core routers for the foreseeable future. (Avici, which places a distant third to the two vendors in terms of market share, failed to show up for the test.)

In an interview last year, Scott Kriens, CEO of Juniper, made the following statement to Light Reading: “Service providers already have two credible sources for high-speed routers: Cisco and Juniper. The market has yet to demonstrate that it wants a third.” On the basis of the Light Reading test results, service providers couldn’t have a third source even if they wanted it.

The diagnostic equipment used in the test was manufactured by Spirent Communications. The equipment, worth $2.6 million, was used to evaluate routers from the four vendors worth a combined total of $29 million .

This is the first time that the networking industry has known for a fact which company had the better product. Until now, service providers and other customers have largely had to rely on vendor-sponsored tests, marketing materials, and hearsay when analyzing core router products.

Light Reading’s test represents a number of other significant firsts:

  • The first multivendor test of core routers
  • The first test of 10-Gbit/s OC192 router interfaces
  • The first time that Cisco agreed to let any of its gear be evaluated in an independent public test

    All of the test results are being published on Light Reading's new Web site, Light Testing (www.lightreading.com/testing), which is being launched today. Light Testing will host the results of a string of tests being planned by Light Reading on leading-edge optical networking equipment and services

    -- Stephen Saunders, U.S. editor, Light Reading http://www.lightreading.com
  • <<   <   Page 2 / 19   >   >>
    dnewman 12/4/2012 | 8:46:34 PM
    re: Juniper Wins Monster Router Test ahoerle,

    We conducted only 1 of 9 sets of tests on the Charlotte's Networks gear before running out of time. For all the rest, the entry simply means "not applicable."

    Unfortunately Charlotte's Network was not able to come back to the lab for retesting in any of the several time slots we offered. However, the vendor says it is commssioning a private retest in the same lab using the same equipment.

    We are *not* saying Aranea-1 doesn't work. Our statement was that there is no level at which the router doesn't forward packets without loss. As Figure 2 shows, the device does forward traffic fairly close to line rate, albeit with loss of packets and some entries in its routing table. This is discussed in the text.

    optical_1 12/4/2012 | 8:46:33 PM
    re: Juniper Wins Monster Router Test why avici bailed out ?????
    Are there any problems with their routers in the
    MPLS/IP/oc-192 area???

    dnewman 12/4/2012 | 8:46:32 PM
    re: Juniper Wins Monster Router Test Thanks for your excellent feedback.

    I'll be the first to admit I'm far from an expert in analyzing the significance of alterations caused by reordering, especially with regard to its impact on TCP.

    In the article we note merely that the impact we observed was neither as serious a problem as Cisco claims (90 percent reordering observed at MAE-East) nor the complete nonissue Juniper claims (0 percent impact). Our measurements showed between 0.41 percent and 7.77 percent of packets out of order.

    IGP measurements are very important indeed. This is an area we're already working on for the next test.

    With regard to OC192 density, note that even for Cisco these are very early days for availability! For several vendors we had significant portions of their entire inventory, and it was hard enough just to get the 12 OC192 interfaces we requested.

    On page 3, we do note in the features table the maximum number of OCXX POS interfaces supported by each vendor.

    David Newman
    dnewman 12/4/2012 | 8:46:32 PM
    re: Juniper Wins Monster Router Test
    Thanks for your inquiry. Avici voiced concerns about supporting the project, nothing more. Prior to the last-minute withdrawl Avici was quite enthusiastic about its OC192 and MPLS capabilities, and the vendor's engineering staff made many constructive suggestions about the test methodology.

    My interpretation is that this was a support issue. I wouldn't read any more into it than that.

    David Newman
    ipguy 12/4/2012 | 8:46:30 PM
    re: Juniper Wins Monster Router Test I would be interested in seeing a test of the 12008 vs. the M40. Kind of like the lone router test, but actually having some real world competition. Many companies will not be ready for 160's and 12416's for a while, so this test would validate today's requirements.
    kevchen 12/4/2012 | 8:46:27 PM
    re: Juniper Wins Monster Router Test David, as for AVCI, in terms of journalistic correctness, rather than stating AVCI "failed to show," (which portrays AVCI as some sort of a "slacker" company), wouldn't it have been better to state what you did in your recent post below:

    "Avici voiced concerns about supporting the project, nothing more. Prior to the last-minute withdrawl Avici was quite enthusiastic about its OC192 and MPLS capabilities, and the vendor's engineering staff made many constructive suggestions about the test methodology. "

    At least the foregoing text would have given AVCI fair coverage, especially considering the fact that they "made many constructive suggestions about the test methodology" which must have been helpful to Lightreading.com.

    Hindsight now tells us that at the time the Lightreading tests were initiated, AVCI was in the midst of setting up KEY DEALS with AT&T, Qwest, France Telecom.

    Perhaps AVCI wanted to devote all of their resources to capturing these key deals and the lightreading test had to take a back seat.

    Subsequent press releases highlighting those AVCI deals with the above customers provide further confirmation that AVCI's support staff was, in fact, very busy closing those deals.
    RouteThis 12/4/2012 | 8:46:27 PM
    re: Juniper Wins Monster Router Test David,

    Thanks for the article. The section on your traffic model denotes that 56% of IP traffic across 'Net core routers is 40 byte traffic...what types of traffic is this comprised of?

    Have you or Lightreading ever done an in depth article on this type of information?? It would be very helpful/enlightening to see a ranking of what applications/IP ports run across Core and Edge internet routers (and on enterprise routers for that matter). This would allow end customers, SPs and vendors know how to best optimize their gear.

    It would also be interesting to note how much traffic is actual data and how much is made up of TCP/IP, UDP, etc. messages transporting the data.

    Here is another site with some info...different byte-size traffic breakdown than your article.

    rtfm 12/4/2012 | 8:46:27 PM
    re: Juniper Wins Monster Router Test I think LR and David Newman have done a good job with the test, and laying most things out in the open. From experience, a lot of people must have spent a lot of effort on this.

    To me, the tests indicate that Juniper and Cisco are reasonably competitive, and that users should understand the differences and trade-offs between the platforms (especially as configured).

    I also agree that testing the various alternatives (smaller "core routers" --but nothing is terabit yet!--) to see if they have similar functionality would be very useful. In addition, testing "edge" routers might be a useful (though much more complicated) exercise.
    irish_heart 12/4/2012 | 8:46:26 PM
    re: Juniper Wins Monster Router Test It's clear that you've received some email from Cisco's finest. I wish they could just look at the facts and see that they need to do some work to their product before it will be truly on par with Juniper. Instead, they whine about stuff that doesn't really matter. Packet re-ordering is only an issue if it happens under normal or high volume network use. We tested this at my company, and IT DOES NOT matter. It is real, but it is a laboratory corner case. Congrats on your test. It is both a useful and interesting data point.
    Did you happen to get into what features you cannot run on Cisco's latest line cards? Did they reveal this to you? You see, the new Engine 4 line cards force you to make a choice between features or performance. Interesting to note, but all of Cisco's line cards force you to make choices about what features you are willing to sacrifice.
    nadie 12/4/2012 | 8:46:26 PM
    re: Juniper Wins Monster Router Test Hi,

    in the article it is stated that:

    "A network comprising 50 routers from Vendor X, each dropping 1 percent of traffic, will experience loss of at least 50 percent."

    My maths might be rusty, but I think a 1% loss
    is a 99% pass probability, that is, for 50 routers, there is a (0.99)^50 pass probability.

    In numbers, 60% pass probability or 40% loss
    probability, of coures I might have forgotten
    "probabilities and statistics 101".


    <<   <   Page 2 / 19   >   >>
    Sign In