Cisco Twists Test Results

Desperation seems to be setting in at Cisco Systems Inc. (Nasdaq: CSCO), judging by a press release it issued last night, claiming that Cisco, rather than Juniper Networks Inc. (Nasdaq: JNPR), had come out on top in Light Reading's test of Internet core routers (see Cisco Claims Router Test Success).

The press release not only misrepresents the overall conclusions of the test but also makes inaccurate statements about the results. In addition, it fails to mention that Light Reading commissioned the tests, which were carried out by Network Test Inc., and that the full results are published on this site (see Internet Core Router Test).

"I'm very disappointed in this misrepresentation," says David Newman, president of Network Test. "Cisco's 12416 put up some very, very good numbers in this test, so the company had no need to spin it the way they did."

At press time, Cisco had not responded to Light Reading's requests that it should issue a second press release correcting these errors and omissions. So, for the record, here’s a replay of parts of Cisco’s press release, annotated with our comments:

Cisco’s Headline: Cisco Reconfirms Leadership In Test of High-End Routers

Truth: Juniper did better overall.

Cisco’s Headline: Only Vendor to Demonstrate 100% Line Rate IP and MPLS Performance For 2.5Gbps OC-48 and 10Gbps OC-192 Throughput Truth: Cisco did not achieve line-rate throughput in MPLS and IP OC192. In fact, in one of these tests throughput was just 52 percent.

Cisco:The Cisco 12416 Internet router has outperformed all other networking vendors in nine out of twelve tests.

Truth: Cisco outperformed Juniper's M160 in only four of the 16 tests.

Cisco:Cisco won every category where the new 1-port OC-192 and 4-port OC-48 line cards and 10-Gbps switch fabric were tested.

Truth: Juniper tied or beat Cisco outright in multiple OC192 tests

Cisco:"Cisco was the only vendor to deliver line rate performance for all of the IP and MPLS forwarding tests for both OC-48 and OC-192," said Robert Redford, senior director of marketing, IP POP systems business unit at Cisco.

Truth: Robert Redford may have said this -- Paul Newman may have said it, too -- but it just ain't true. Cisco did not deliver line-rate performance in all tests. In particular, the 40-byte IP throughput over OC192 was just 52 percent, as already noted.

Larry Lang, Cisco’s vice president of service provider marketing says that Cisco was “pretty careful” about checking its facts in the release. Cisco will issue a correction if needs be, he adds.

The press release “doesn’t misrepresent the test. It differs from your editorial conclusions, and that’s okay,” says Lang. He adds that Cisco doesn’t dispute the test methodology. “It’s a fine test. Our difference is in the conclusions.”

Lang bobbed and weaved when asked why Cisco hadn’t acknowledged that the test had been commissioned by Light Reading and hadn’t referenced the full results on Light Reading's site. “I guess I was confused,” he says. “There’s a difference over whose test it is and who did the test.” -- Peter Heywood, international editor, Light Reading http://www.lightreading.com
Page 1 / 16   >   >>
chuckjackson 12/4/2012 | 8:44:39 PM
re: Cisco Twists Test Results I think the key to the conflicting conclusions between LR and Cisco is the 40-byte versus internet mix. If you consider only internet mix, almost all of Cisco's claims are correct. In general, I would think that the internet mix carries more weight. It is a more realistic test case. How often do service providers see 100% 40-byte traffic? The 40-byte traffic is more of a corner case. It's more important for ego rather than a realistic fair comparison. Perhaps, Cisco should have clarified their conclusions by adding in a statement that their conclusions were based on the internet mix traffic. In the same token, LR should have clarified their conclusions by adding a statement that their conclusions were based on the 100% 40-byte only traffic.

IMO, cisco's press release is accurate and justifiable.
light_on_dude 12/4/2012 | 8:44:39 PM
re: Cisco Twists Test Results Hey Peter, how does it feel to have YOUR words twisted? sounds like Cisco did to Lightreading, what Lightreading usually does to vendors...hows it feel to be twisted?? LOL!! (please don't say i work for cisco...I still hate them as much as ever) It's just damn funny to see this turnabout!!!
photonic 12/4/2012 | 8:44:39 PM
re: Cisco Twists Test Results
After accusing all us readers of being cry babies, it is really funny to see Light Reading editors turn into one themselves.

I love it.

dnewman 12/4/2012 | 8:44:38 PM
re: Cisco Twists Test Results Please read the disclosure statement on the first page of the test before making unfounded allegations:


The test results are what they are. No test, in and of itself, favors any particular vendor.

Let's stick with facts, shall we?

David Newman
Network Test
dnewman 12/4/2012 | 8:44:38 PM
re: Cisco Twists Test Results Sorry, but even considering Imix alone Cisco's release still misrepresents the facts.

For example, Cisco did not beat all other comers in nine out of 12 events. There were 16 events in which Cisco participated, and of these 16 Cisco won only four events outright.

By the way, I think considering the test without 40-byte, or without filtering, or without whatever, is an excellent idea -- if that meets your networking requirements.

Note here that I say networking requirements; spinning revisionist history to shore up your stock portfolio or reconfirm your prejudices doesn't change the test result.

David Newman
Network Test
iptwister 12/4/2012 | 8:44:38 PM
re: Cisco Twists Test Results Benjamin Disraeli is quoted as saying "... there are lies, damned lies, and statistics".

The tests executed seem to give an advantage to Juniper.

I wonder if the Light Reading editorial staff hols stock in JNPR.

(who happens to be shorting JNPR)

tpomerhn 12/4/2012 | 8:44:37 PM
re: Cisco Twists Test Results Yeah... I especially like the "Paul Newmann" comment. Sounds like LR's whining quite a bit, now. :)

I agree with Chuck - Cisco's press release is accurate when you use Imix, and maybe that should've been spelled out. I thought the Imix was more of a applicable test, too.

I, for one, would've liked to see tests on hot-swapping and failover - I'd like to see how these monsters responded to a few "pulls on the ol' OC-192 card :)"
optical illusion 12/4/2012 | 8:44:37 PM
re: Cisco Twists Test Results A NOTE TO ALL VENDORS:

A vendor could lose a customer pretty quick if they read this site and get the impression they might be dealing with used car salesmen.

Not to mention that Juniper will play up the disputed press release for all it's worth.
csco-grunt 12/4/2012 | 8:44:36 PM
re: Cisco Twists Test Results Got to endorse the ha-ha-ha here, it's called
"hoist by your own petard."

Stop complaining, and tell the intern who writes
your editorials for you to take a class in logic.

liteguy 12/4/2012 | 8:44:36 PM
re: Cisco Twists Test Results Does the Light reading staff love juniper or what? They should stop writing articles on them. To make it easier, they should move all juniper praise / cisco bashing to a tab on the side of the home page, this way we dont have to see it, and all the juniper employees can read it and have fun with it. By the way, I dont work for cisco, and I dont own stock in either.
Page 1 / 16   >   >>
Sign In