Charlotte's Web: Who's Best Now?Charlotte's Web: Who's Best Now?

Charlotte’s Web Networks Ltd. plans to announce the results of tests on its core router, the Aranea-1, tomorrow, in another effort to prove that it can outperform the big guns in this field -- the 12410 from Cisco Systems Inc. (Nasdaq: CSCO) and the M160 from Juniper Networks Inc. (Nasdaq: JNPR).

The results indicate that the Aranea-1, when equipped with 32 OC48 (2.5 Mbit/s) ports, has excellent performance. It achieves 100 percent of the maximum theoretical data rate when handling three different sizes of packets (40, 512, and 1,500 bytes), according to Mier Communications Inc. (Miercom), the company that conducted the tests.

The Aranea-1 does a good job of policing congested traffic without compromising performance, latency, or jitter, adds Miercom. It also learns 200,000 BGP (border gateway protocol) routes in 45 seconds while forwarding traffic at line rate, it says.

“There’s more to a router company than hardware, but Charlotte’s got exceptional hardware,” concludes Robert Smithers, Miercom’s president.

In its Lab Testing Summary Report, Miercom makes some flattering comparisons between the results of its tests on Charlotte’s Aranea-1 and the results of tests that it performed a few months ago, comparing the Cisco 12410 and Juniper M160.

However, all of this needs to be taken with a pinch of salt, and doesn’t necessarily mean that the Aranea-1 is the better router.

For a kickoff, Charlotte paid Miercom to conduct the tests, and thus was in a position to specify what aspects of its router should be tested and what ought to be glossed over. It was also able to say what results should be published or kept confidential.

Charlotte asked Miercom to repeat the methodology it had used in its earlier Cisco/Juniper comparison and then add in a couple of extra tests “to up the ante” as Smithers puts it.

That, however, muddies the waters, because Cisco also paid Miercom to conduct the comparison with Juniper. Cisco is known to have kept some results confidential, possibly because they didn’t support the conclusions it trumpeted in its press release (see Cisco Touts Router Test Results). Not surprisingly, Juniper declined to participate in the tests, so it’s also possible that its M160 wasn’t configured as well as it might have been.

In the Charlotte test, Miercom repeated “every test [in the Cisco/Juniper comparison] that was publicly available” according to Smithers. All the same, comparing the results of this comparison with tests conducted on Charlotte’s Aranea-1 a couple of months later is dodgy. “The results don’t directly compare,” acknowledges Smithers.

When considering what might be the highest performance router, it's also worth noting that Cisco and Juniper routers support OC192 (10 Mbit/s) ports, and Charlotte's Aranea doesn't.Light Reading published the results of its Internet Core Router Test, conducted by Network Test Inc. and Spirent Communications.

In this independent test (Light Reading paid for it), Juniper’s M160 was the overall winner but Cisco’s 12410 came a close second, although Cisco “interpreted” the results differently and claimed it had won (see Cisco Claims Router Test Success, Light Reading Disputes Cisco's Claims , and Cisco Reasserts Router Test Success).

The other two participants, Charlotte and Foundry Networks Inc. (Nasdaq: FDRY) did badly. In Charlotte’s case, a bug was found in its software that prevented it participating in many of the tests, and it wasn’t able to rectify the problem in time. (Its press release about the results -- see Charlotte's Net: 'We're Number Three!' -- was even funnier than Cisco’s.)

Once Charlotte had debugged its software, it commissioned another lab, The Tolly Group to test the Aranea-1 using the same methodology as the one used by Light Reading. This appeared to show that Charlotte’s router could now deliver excellent performance, but the fact that it was a private test, paid for by Charlotte, meant there was no way of verifying the results (see Charlotte's Networks, Take 2).

The real proof of Charlotte’s performance advantage is likely to come from its customers. “Multiple” beta trials are already under way, according to Jeff Graham, director of marketing at MRV Communications Inc. (Nasdaq: MRVC), Charlotte’s parent company. However, the company has yet to announce an actual order.

— Peter Heywood, Founding Editor, Light Reading
http://www.lightreading.com