Cisco Twists Test Results
The press release not only misrepresents the overall conclusions of the test but also makes inaccurate statements about the results. In addition, it fails to mention that Light Reading commissioned the tests, which were carried out by Network Test Inc., and that the full results are published on this site (see Internet Core Router Test).
"I'm very disappointed in this misrepresentation," says David Newman, president of Network Test. "Cisco's 12416 put up some very, very good numbers in this test, so the company had no need to spin it the way they did."
At press time, Cisco had not responded to Light Reading's requests that it should issue a second press release correcting these errors and omissions. So, for the record, here’s a replay of parts of Cisco’s press release, annotated with our comments:
Cisco’s Headline: Cisco Reconfirms Leadership In Test of High-End Routers
Truth: Juniper did better overall.
Cisco’s Headline: Only Vendor to Demonstrate 100% Line Rate IP and MPLS Performance For 2.5Gbps OC-48 and 10Gbps OC-192 Throughput Truth: Cisco did not achieve line-rate throughput in MPLS and IP OC192. In fact, in one of these tests throughput was just 52 percent.
Cisco:The Cisco 12416 Internet router has outperformed all other networking vendors in nine out of twelve tests.
Truth: Cisco outperformed Juniper's M160 in only four of the 16 tests.
Cisco:Cisco won every category where the new 1-port OC-192 and 4-port OC-48 line cards and 10-Gbps switch fabric were tested.
Truth: Juniper tied or beat Cisco outright in multiple OC192 tests
Cisco:"Cisco was the only vendor to deliver line rate performance for all of the IP and MPLS forwarding tests for both OC-48 and OC-192," said Robert Redford, senior director of marketing, IP POP systems business unit at Cisco.
Truth: Robert Redford may have said this -- Paul Newman may have said it, too -- but it just ain't true. Cisco did not deliver line-rate performance in all tests. In particular, the 40-byte IP throughput over OC192 was just 52 percent, as already noted.
Larry Lang, Cisco’s vice president of service provider marketing says that Cisco was “pretty careful” about checking its facts in the release. Cisco will issue a correction if needs be, he adds.
The press release “doesn’t misrepresent the test. It differs from your editorial conclusions, and that’s okay,” says Lang. He adds that Cisco doesn’t dispute the test methodology. “It’s a fine test. Our difference is in the conclusions.”
Lang bobbed and weaved when asked why Cisco hadn’t acknowledged that the test had been commissioned by Light Reading and hadn’t referenced the full results on Light Reading's site. “I guess I was confused,” he says. “There’s a difference over whose test it is and who did the test.” -- Peter Heywood, international editor, Light Reading http://www.lightreading.com