x
Optical/IP

Cisco Twists Test Results

Desperation seems to be setting in at Cisco Systems Inc. (Nasdaq: CSCO), judging by a press release it issued last night, claiming that Cisco, rather than Juniper Networks Inc. (Nasdaq: JNPR), had come out on top in Light Reading's test of Internet core routers (see Cisco Claims Router Test Success).

The press release not only misrepresents the overall conclusions of the test but also makes inaccurate statements about the results. In addition, it fails to mention that Light Reading commissioned the tests, which were carried out by Network Test Inc., and that the full results are published on this site (see Internet Core Router Test).

"I'm very disappointed in this misrepresentation," says David Newman, president of Network Test. "Cisco's 12416 put up some very, very good numbers in this test, so the company had no need to spin it the way they did."

At press time, Cisco had not responded to Light Reading's requests that it should issue a second press release correcting these errors and omissions. So, for the record, here’s a replay of parts of Cisco’s press release, annotated with our comments:

Cisco’s Headline: Cisco Reconfirms Leadership In Test of High-End Routers

Truth: Juniper did better overall.

Cisco’s Headline: Only Vendor to Demonstrate 100% Line Rate IP and MPLS Performance For 2.5Gbps OC-48 and 10Gbps OC-192 Throughput Truth: Cisco did not achieve line-rate throughput in MPLS and IP OC192. In fact, in one of these tests throughput was just 52 percent.

Cisco:The Cisco 12416 Internet router has outperformed all other networking vendors in nine out of twelve tests.

Truth: Cisco outperformed Juniper's M160 in only four of the 16 tests.

Cisco:Cisco won every category where the new 1-port OC-192 and 4-port OC-48 line cards and 10-Gbps switch fabric were tested.

Truth: Juniper tied or beat Cisco outright in multiple OC192 tests

Cisco:"Cisco was the only vendor to deliver line rate performance for all of the IP and MPLS forwarding tests for both OC-48 and OC-192," said Robert Redford, senior director of marketing, IP POP systems business unit at Cisco.

Truth: Robert Redford may have said this -- Paul Newman may have said it, too -- but it just ain't true. Cisco did not deliver line-rate performance in all tests. In particular, the 40-byte IP throughput over OC192 was just 52 percent, as already noted.

Larry Lang, Cisco’s vice president of service provider marketing says that Cisco was “pretty careful” about checking its facts in the release. Cisco will issue a correction if needs be, he adds.

The press release “doesn’t misrepresent the test. It differs from your editorial conclusions, and that’s okay,” says Lang. He adds that Cisco doesn’t dispute the test methodology. “It’s a fine test. Our difference is in the conclusions.”

Lang bobbed and weaved when asked why Cisco hadn’t acknowledged that the test had been commissioned by Light Reading and hadn’t referenced the full results on Light Reading's site. “I guess I was confused,” he says. “There’s a difference over whose test it is and who did the test.” -- Peter Heywood, international editor, Light Reading http://www.lightreading.com
<<   <   Page 16 / 16
Sparky 12/4/2012 | 8:43:19 PM
re: Cisco Twists Test Results First, let me start by agreeing with the others on this board that you are clueless. Nothing personal but you come off as being arrogant and obstinate.

Rajah:
So?? What does that have to do with twisting the results? Did you ever write an exam in school? One of the basic principles is to answer the question and stick to the subject.

Sparky:
You are the one not sticking to the subject of this particular thread. I should know I started it. You are in your own world.

Rajah:
That's just your opinion. Can you name the "many people" other than Cisco proponents who thought the tests weren't meaningful?

Sparky:
It is not just my opinion, if you have been reading these threads you would know that many other have been saying the same thing. You deny that OIR, capacity, redundancy and QoS latency tests would not have been vital to a comprehensive test of core routers?

Sparky
melao 12/4/2012 | 8:43:17 PM
re: Cisco Twists Test Results http://newsroom.cisco.com/dlls...

this is the site.
it has wuotes of david newman. Well, something is very odd in this story.
perry1961 12/4/2012 | 8:43:16 PM
re: Cisco Twists Test Results Start-up Charlotte's Networks Joins Cisco and Juniper as Light Reading's Top Three Leading Core Router Providers
Charlotte's to Commission Spirent Communications and The Tolly Group to Authenticate Ranking in Light Reading's First Test of Core Routers
Andover, Mass.--(BUSINESS WIRE)--March 16, 2001-- Charlotte's Networks, a leading developer of carrier-class routers for the emerging IP infrastructure, today said Light Reading and Network Test have validated Charlotte's Networks as the third contender in the core router market.

Charlotte's was pleased to participate in the first test of core routers by Light Reading, the premier information resource for the optical networking industry. The test was performed by David Newman, President of Network Test, the respected pioneer of network testing.

Charlotte's, the new kid on the block, started development of its core router 2 1/2 years after Juniper and 15 years after Cisco, and is quickly growing-up -- out of the toddler years. Charlotte's was the youngest out of the four participants of the test and the only start-up with the confidence to participate one-on-one with Cisco and Juniper.

Eli Stein, vice president of engineering for Charlotte's, commented: ``Cisco and Juniper are the established market leaders. We are pleased with Light Reading's validation of Charlotte's as the third contender. We feel confident in Charlotte's position in this potential $15 billion market, especially since our box has double the speed and port density compared to any other vendor.''

Stein continued, ``Prior customer commitments prevented us from completing all aspects of the Light Reading test within the deadlines for publication. However, we feel so confident in the Aranea's ability to perform, Charlotte's is engaging the Tolly Group and Spirent Communications to reproduce Light Reading's first test of core routers. We look forward to the final results, which will establish Charlotte's as the only viable alternative to Cisco and Juniper.''

The test will be conducted by Spirent Communications, a worldwide leader in performance analysis solutions. The test will be performed at Spirent's lab using the exact equipment and identical test methodology used in Light Reading's first test of core routers. The Tolly Group, long recognized as a leading independent networking test benchmarking firm, will ensure that the test will be identical in every manner to the Light Reading test. Charlotte's and Light Reading have agreed to publish the results.

About Charlotte's Networks

Charlotte's Networks is a leading developer of carrier-class routing solutions for the emerging IP infrastructure. Charlotte's Networks' innovative solutions provide the high performance, scalability, Quality of Service and reliability required by carriers, large Internet Service Providers (ISPs) and Competitive Local Exchange Carriers (CLECs). Charlotte's Networks has headquarters in Andover, with R&D facilities in Israel. For more information, contact Charlotte's Networks toll-free at 866/738-4675, or visit the company online at http://www.cwnt.com/.

Charlotte's Networks is a technology investment partner of MRV Communications Inc. (Nasdaq:MRVC - news). For more information, visit http://www.mrv.com/.

Belzebutt 12/4/2012 | 8:43:15 PM
re: Cisco Twists Test Results They added the words "The test was sponsored by Light Reading". But they still won't link directly to the test results.
skeptic 12/4/2012 | 8:43:15 PM
re: Cisco Twists Test Results I guess Cisco's pride was hurt.
JUST TELL THE TRUTH and GET TO WORK ON A BETTER PRODUCT!
---------------------------------

"Better product"

1. Put virtual memory in the system so you can generate unrealistic benchmarks. After all, performance and stability doesn't matter. All that matters is a big number.

2. Break your routing implementation so it continues to incorrectly accept BGP routes even after the forwarding system has filled up with routes. Customers just don't care about this sort of thing and the testers will not catch on to the implications of it.

3. Next time you have to do a line card, forget unimportant "geek" features like preserving packet ordering. What matters is getting the product to customers fast. And when the customers report the bug, first pretend it isn't happening. Then have your marketing people work their friends outside the company to convince everyone that the bug isn't important.

4. Become really friendly with the testers. Have your CEO give an interview to light reading. Light reading will do anything for an important interview. Just ask Caspian networks.

stilespj 12/4/2012 | 8:43:11 PM
re: Cisco Twists Test Results I guess Cisco defends their spin by saying "it all depends what the meaning of is is"

Sheesh, Cisco, trying to spin your close second place finish into an outright win is extending the elastic properties of the truth beyond their breaking points!

Paul
Rajah 12/4/2012 | 8:43:06 PM
re: Cisco Twists Test Results
Sparky:
First, let me start by agreeing with the others on this board that you are clueless. Nothing personal but you come off as being arrogant and obstinate.

Rajah:
Other than net-exprt who has been getting his own rave reviews, can you tell me who the "others" are?

Sparky:
You are the one not sticking to the subject of this particular thread. I should know I started it. You are in your own world.

Rajah:
Uh....Are you in another planet? What's the subject name assigned to this Article Talk forum? The last I checked, it was "Cisco twists Test Results". That is the subject. Like I said, did you ever write an exam in school?

Sparky:
It is not just my opinion, if you have been reading these threads you would know that many other have been saying the same thing.

Rajah:
And many others say exactly the opposite.

Sparky:
You deny that OIR, capacity, redundancy and QoS latency tests would not have been vital to a comprehensive test of core routers?

Rajah:
You still don't get it do you? The issue is not of the type or nature of tests that would make for a "more" meaningful or comprehensive test. Its WHETHER and WHY did Cisco twist the results of the tests that were performed, regardless of the types of tests. Regarding your more "comprehensive" tests, are you implying that the tests performed were not at all comprehensive? If you seem to think so, it then brings up the question of how and why Cisco entered the tests in the first place. And please, don't make yourself look like more of an idiot with your theories of Cisco having absolutely no say in the test plans. I see you still have not replied to the other thread showing your "basis" for your argument. Looks like there never was one, was there?







net_exprt 12/4/2012 | 8:42:49 PM
re: Cisco Twists Test Results By the way, LightSwitcher, is that what you do at work all day long? Are you proud of your ability to arrange words together to come up with the following gem:
"JUST TELL THE TRUTH and GET TO WORK ON A BETTER PRODUCT"...

Tell us more about yourself, "lightSwitcher"...I hear there is a tremendous market out there for lightswitchers...make sure your resume is polished up and ready to go, oh lightswitcher...
net_exprt 12/4/2012 | 8:42:49 PM
re: Cisco Twists Test Results raja, raja, raja...

There are those that read the "monster win" article and understand it for what it is...an article based on sensationalism and poor journalism...and then there are folks like you, who are either naive enough to be mesmorized by the headline, or are Juniper employees... As I have stated before, there are alot of factors associated with such testings.
As far as "rave reviews" as you put it so elloquently, all I saw where idiotic comments coming out of you by taking sentences that were out of context and responding only to those.
<<   <   Page 16 / 16
HOME
Sign In
SEARCH
CLOSE
MORE
CLOSE