Cisco Twists Test Results

Desperation seems to be setting in at Cisco Systems Inc. (Nasdaq: CSCO), judging by a press release it issued last night, claiming that Cisco, rather than Juniper Networks Inc. (Nasdaq: JNPR), had come out on top in Light Reading's test of Internet core routers (see Cisco Claims Router Test Success).

The press release not only misrepresents the overall conclusions of the test but also makes inaccurate statements about the results. In addition, it fails to mention that Light Reading commissioned the tests, which were carried out by Network Test Inc., and that the full results are published on this site (see Internet Core Router Test).

"I'm very disappointed in this misrepresentation," says David Newman, president of Network Test. "Cisco's 12416 put up some very, very good numbers in this test, so the company had no need to spin it the way they did."

At press time, Cisco had not responded to Light Reading's requests that it should issue a second press release correcting these errors and omissions. So, for the record, here’s a replay of parts of Cisco’s press release, annotated with our comments:

Cisco’s Headline: Cisco Reconfirms Leadership In Test of High-End Routers

Truth: Juniper did better overall.

Cisco’s Headline: Only Vendor to Demonstrate 100% Line Rate IP and MPLS Performance For 2.5Gbps OC-48 and 10Gbps OC-192 Throughput Truth: Cisco did not achieve line-rate throughput in MPLS and IP OC192. In fact, in one of these tests throughput was just 52 percent.

Cisco:The Cisco 12416 Internet router has outperformed all other networking vendors in nine out of twelve tests.

Truth: Cisco outperformed Juniper's M160 in only four of the 16 tests.

Cisco:Cisco won every category where the new 1-port OC-192 and 4-port OC-48 line cards and 10-Gbps switch fabric were tested.

Truth: Juniper tied or beat Cisco outright in multiple OC192 tests

Cisco:"Cisco was the only vendor to deliver line rate performance for all of the IP and MPLS forwarding tests for both OC-48 and OC-192," said Robert Redford, senior director of marketing, IP POP systems business unit at Cisco.

Truth: Robert Redford may have said this -- Paul Newman may have said it, too -- but it just ain't true. Cisco did not deliver line-rate performance in all tests. In particular, the 40-byte IP throughput over OC192 was just 52 percent, as already noted.

Larry Lang, Cisco’s vice president of service provider marketing says that Cisco was “pretty careful” about checking its facts in the release. Cisco will issue a correction if needs be, he adds.

The press release “doesn’t misrepresent the test. It differs from your editorial conclusions, and that’s okay,” says Lang. He adds that Cisco doesn’t dispute the test methodology. “It’s a fine test. Our difference is in the conclusions.”

Lang bobbed and weaved when asked why Cisco hadn’t acknowledged that the test had been commissioned by Light Reading and hadn’t referenced the full results on Light Reading's site. “I guess I was confused,” he says. “There’s a difference over whose test it is and who did the test.” -- Peter Heywood, international editor, Light Reading http://www.lightreading.com
<<   <   Page 2 / 16   >   >>
optiray 12/4/2012 | 8:44:36 PM
re: Cisco Twists Test Results

Wow! This is cool. Poor LR, how does it
feel to be on the receiving end? After all,
unlike you, Cisco has got its facts correct!

Ha Ha Ha Ha !!!

Sparky 12/4/2012 | 8:44:35 PM
re: Cisco Twists Test Results "Still, it is an error to argue in front of your data. You find yourself insensibly twisting them around to fit your theories." Sherlock Holmes in The Adventure of Wisteria Lodge.

These articles are ridiculous and the authors and testing suspect with at best mediocre results. Evaluations should be based on quantifiable and qualitative results vs. my box is better than yours, Ha-ha-ha!

To say that the M160 is BoB based on out-performing the GSR 12416 in a few tests, keeping in mind that the composition of tests was incomplete, is myopic to say the least.

It is safe to say that those who wished to see the M160 succeed engineered this test. The proof is in the test itself and the editorial comments by the author. With so much invested in equipment and time why not perform a truly comprehensive test, on core routing features such as OIR, fully loaded box, QoS latency, capacity and redundancy. In my humble opinion, Cisco has always been a winner. There is no news or ratings in reading about Cisco Systems winning again. LR creates contention to get people to read their hype.
flanker 12/4/2012 | 8:44:35 PM
re: Cisco Twists Test Results There would only be irony if Cisco had actually won the test. Did they? It's ok to vent, though. You guys got the living crap kicked out of you on the board's yesteday.


By the way, I have nothing against CSCO. Maybe they did win, but all I see on the boards are sour grapes.
optical illusion 12/4/2012 | 8:44:35 PM
re: Cisco Twists Test Results Apparently CSCO withdrew the release. Can anyone find it on the site?


tink 12/4/2012 | 8:44:34 PM
re: Cisco Twists Test Results Sparky,

You state, "It is safe to say that those who wished to see the M160 succeed engineered this test."

If this were the case, why would they engineer a test where they failed to deliver line-rate, a claim they've been making for almost a year now? Your assertion/argument fails.

Sparky 12/4/2012 | 8:44:34 PM
re: Cisco Twists Test Results Tink,

Why do you think my assertion is incorrect? Think about it. JNPR surely thought they would be able to perform at line rate or at a minimum beat CSCO. They thought wrong. What is important to understand is why were important features to core routing not tested?

xerxes98 12/4/2012 | 8:44:33 PM
re: Cisco Twists Test Results It's interesting to note that minus the ACL tests, Cisco actually scored 18 first-places to juniper's 17 ... Juniper got 36 of its first-place results from ACL tests alone, an area that LR knew Cisco didn't support in the first place (nor did the other two vendors for the most part) ... so what we have here is a win on ACL by default ... is that really so impressive ?

I will be interested to see if LR would repeat this test once Cisco starts ACL support ...
probably not because then they would have to actually admit that Cisco has a better box ...

I still have not seen a reply as to why there were no OIR or fully-loaded box tests done ...
dnewman 12/4/2012 | 8:44:33 PM
re: Cisco Twists Test Results
"It is safe to say that those who wished to see the M160 succeed engineered this test. "

Careful: Cisco's engineers were among those who made constructive suggestions about, and contributions to, our test methodology. In fact, during the test Cisco's engineering staff praised the methodology for its rigor and thoroughness.

I am very sorry to see this war of words escalate. Cisco's 12416 achieved fine results in this test. It is regrettable that Cisco chose to misrepresent the results rather than letting the 12416's excellent numbers speak for themselves.

David Newman
Network Test
Dr. Dre 12/4/2012 | 8:44:33 PM
re: Cisco Twists Test Results I am offering a potential solution to this apparent deadlock.

Would Lightreading be willing to take a poll amongst the readers to see who they believe won the tests based on the presented results.
That is pick one of:

1) Cisco Won
2) Juniper Won
3)Results are Inconclusive

The poll might give us a more clear idea what the general consensus is.

Yes some votes will be biased, but hopefully the majority wont.

In my own opinion, the tests seem reasonable. But I am not so sure a true winner can really be determined.

Any comments....

--Dr. Dre
GonePostal 12/4/2012 | 8:44:32 PM
re: Cisco Twists Test Results This testing tit for tat stuff is boring and irrelevent. The real question is whether Larry Lang still cuts his hair with a Flowbie machine.
<<   <   Page 2 / 16   >   >>
Sign In