The results demonstrate that at least two edge routers -- the ST200 from Laurel Networks Inc. and the SmartEdge 800 from Redback Networks Inc. (Nasdaq: RBAK) -- have got what it takes.
Both boxes handle huge numbers of Layer 2 and Layer 3 MPLS VPNs. In fact, they handle more Layer 2 VPN tunnels than we were able to measure, and our test bed was no toy. It broke all records in terms of pushing edge routers to the limit. The tests themselves -- conducted by Network Test using equipment and specially designed test software from Spirent Communications -- took nine months to complete and involved 40 staff.
Apart from demonstrating the ability to handle massive numbers of VPNs, the tests also measured: throughput and latency when handling different packet sizes; the time needed to restore 500,000 routes after a network failure; enforcement of quality-of-service requirements; and capacity in handling routing protocols.
In the report giving the full results of the tests (see Metro Edge Router Test), David Newman, president of Network Test, highlights three particular findings:
- Using Layer 3 MPLS VPNs, both boxes emulated more than 2,400 virtual routers and handled hundreds of routes per customer.
- Layer 2 MPLS VPNs scaled even higher, with both products forwarding traffic through nearly 40,000 tunnels. As far as we’re aware, both boxes set new records for public tests of MPLS VPN scaleability.
- Routing capacity tests for both boxes produced absurdly high numbers, typically one or even two orders of magnitude beyond today’s levels.
"Laurel’s product fared better in the baseline throughput and failover tests, and it scaled much higher in the Layer 3 MPLS VPN tests. For its part, Redback’s SmartEdge router did better in some baseline latency and routing scaleability tests (though not by meaningful margins on the latter, in our view)."
The most disappointing aspect of the test was that other players declined to participate. More than 20 were invited.
The absence of Cisco Systems Inc. (Nasdaq: CSCO) is understandable, according to Newman. Cisco is thought to have a new edge router in the pipeline, and its existing products probably wouldn't have stacked up well against Laurel's and Redback's.
The same cannot be said of the other market leader, Juniper Networks Inc. (Nasdaq: JNPR), according to Newman. Juniper had two strong candidates -- its own M40e and the ERX for which it bought Unisphere -- and yet it pulled out of the tests, having originally agreed to take part (prior to the Unisphere acquisition). Juniper cites lack of resources for its decision, but other reasons are suspected. "Clearly it was politics, not products, that prevented Juniper from putting its best foot forward," writes Newman in his report.
The focus of the test on high-end routers might explain why some vendors declined to participate, according to Giles Heron, principal network architect at PacketExchange Ltd., a service provider deploying Layer 2 MPLS VPNs in Europe. "The big cutoff point was the use of OC48 (2.5 Gbit/s) versus OC12 (622 Mbit/s) interfaces," writes Newman. "This did preclude some of the smaller folk like CoSine Communications Inc. (Nasdaq: COSN), but none of the bigger fish like Cisco, Juniper, Alcatel SA (NYSE: ALA; Paris: CGEP:PA), or Riverstone Networks Inc. (Nasdaq: RSTN)," he says. "None of the major players in this space have a legitimate beef on that point."
As noted, these tests were totally independent -- they were paid for by Light Reading. The importance of this was underscored in Testing Testing, a poll of readers taken earlier this year.
— Peter Heywood, Founding Editor, Light Reading