Charlotte's Networks, Take 2
The results appear to demonstrate that the Aranea-1 is now ready for commercial use, having overcome problems in an early version of its software that resulted in it doing badly in a ground-breaking test of core routers sponsored by Light Reading, the results of which were published in March (seeInternet Core Router Test).
The Aranea-1 did the equivalent of falling at the first fence in Light Reading's tests, which were conducted by Network Test Inc. and Spirent Communications. It performed poorly in a small subset of benchmarks before the problem was discovered, and Charlotte's Networks wasn't able to fix its software and submit the Aranea-1 for restesting in the time available (see Charlotte's Networks).
In restrospect, Charlotte's Networks was probably a little foolhardy to get involved in the test when its product wasn't quite ready. But it deserves a lot of credit for the way in which it's dealt with the aftermath.
Once Charlotte's Networks had fixed its software problems, it commissioned another lab, The Tolly Group, to test the Aranea-1 using exactly the same testbed, methodology, and benchmarks that were used in the Light Reading tests. It also got Spirent to validate this.
"The people involved with the Charlotte's Networks-sponsored tests have told me they did reproduce exactly what we did for Light Reading," says David Newman, president of Network Test.
A summary of results from Charlotte's Networks is shown on Page 2 of this article.
The results appear to indicate that the Aranea-1 now boasts throughput at or near line rates, with extremely low latency. Its routing table appears to accommodate close to 1.5 million BGP (border gateway protocol) entries -- 15 times the total number in the world at present -- and its flapping and convergence performance appears to hold up under extreme loads.
"Overall, the retest results appear to be a huge improvement over the numbers we obtained," says Newman. "Note that I say 'appears' here. The standard caveat applies -- that the results were vendor-sponsored and thus not independently verifiable".
It's also worth pointing out that Light Reading effectively conducted two tests originally -- one on equipment with OC48 (2.5 Gbit/s) interfaces and one on equipment with OC192 (10 Gbit/s) interfaces. The tests conducted by The Tolly Group for Charlotte's Networks only covered equipment with OC48 interfaces.
It's tempting to compare the detailed results on Page 2 with the ones from Light Reading's original test -- notably the ones from Juniper Networks Inc. (Nasdaq: JNPR) and Cisco Systems Inc. (Nasdaq: CSCO), both of which notched up impressive performance figures.
However, this would be comparing apples with oranges because several months separate the two tests. It's likely that the other participants in the original test have improved the performance of their products in the interim.
"I'm quite sure Cisco and Juniper have improved their routers since the Light Reading results were published," says Newman.
The other participant in the original test, Foundry Networks Inc. (Nasdaq: FDRY), also claims "dramatic improvements" in performance since its Netiron core router was given a grilling by Newman.
Like Charlotte's Networks, Foundry did badly in the original test and blamed it on "software bugs". Marshall Eisenberg, Foundry's director of product marketing, says Foundry fixed the bugs and retested its router in its own labs, using Light Reading's methodology. The improvement in performance was "like night and day" according to Eisenberg -- although its results, like Charlotte's Networks', have not been independently verified.
Want more detail? Charlotte's Networks will be presenting the results of its tests in a Light Reading Webinar at 11am PST/2pm EST on June 26. Click here to register.
Next Page: Charlotte's Networks' Test Results
— Peter Heywood, Founding Editor, Light Reading
http://www.lightreading.com For more information on Supercomm 2001, please visit the Light Reading Supercomm 2001 Preview Site.