Light Reading, along with its testing partners – Network Test of Westlake Village, Calif., a benchmarking and network design consultancy; and Spirent Communications of Calabasas, Calif., a test equipment supplier – has just completed a massive edge router trial. We pounded the Laurel Networks Inc. ST200 and Redback Networks Inc. (Nasdaq: RBAK) SmartEdge 800 in the most rigorous routing test we’ve ever conducted.
Make that ten tests: Our methodology covered everything from basic throughput to resiliency testing to MPLS Layer 2 VPNs. This sizeable undertaking was more than nine months in the making, and involved a staff of more than 40 engineers and project managers (see Thanks).
The good news? Both vendors passed with flying colors.
Among the high points:
- Using Layer 3 MPLS VPNs, both boxes emulated more than 2,400 virtual routers and handled hundreds of routes per customer.
- Layer 2 MPLS VPNs scaled even higher, with both products forwarding traffic through nearly 40,000 tunnels. As far as we’re aware, both boxes set new records for public tests of MPLS VPN scaleability.
- Routing capacity tests for both boxes produced absurdly high numbers, typically one or even two orders of magnitude beyond today’s levels.
We weren’t too surprised about Cisco: Judging from data sheets and anecdotal evidence, its current products wouldn’t have fared well at all with parts of our methodology. We’ve also heard there are better products in the pipeline.
Juniper had no such excuse: In fact, the data sheet for the vendor’s M40e (e as in edge) served as one of our guides in putting together a test methodology. Even more promising, Juniper bought Unisphere Networks, the leading edge router maker, while we were preparing for this project. We looked forward to testing one and possibly two entries from Junisphere.
Juniper actually did agree to take part, but the deluge began soon after it swallowed Unisphere (or was it the other way around?). First, we heard multiple reports of internal strife. Then we got word that Juniper was officially withdrawing from the test (accompanied, of course, by reassurances that all was well internally). Then followed another cycle of reconsideration, but ultimately Juniper sat this one out. Too bad: Clearly it was politics, not products, that prevented Juniper from putting its best foot forward.
We also gathered an impressive collection of excuses from the more than 20 other vendors that opted out (please see No Shows before asking why this vendor or that one isn’t represented here). Some of these vendors simply didn’t have appropriate product; our requirement for OC48 (2.5 Gbit/s) interfaces ruled out several potential players. Others had reasons of their own.
Whatever shortcomings we uncovered in the Laurel and Redback products (and there were very few), both vendors deserve major credit for their willingness to submit their products to public testing.
If we had to choose between the Laurel and Redback routers, we’d give the edge to Laurel’s ST200, while noting that good arguments can be made for either router. Laurel’s product fared better in the baseline throughput and failover tests, and it scaled much higher in the Layer 3 MPLS VPN tests. For its part, Redback’s SmartEdge router did better in some baseline latency and routing scaleability tests (though not by meaningful margins on the latter, in our view). Both boxes turned in excellent results overall.
A summary of all the tests and results are provided in the table below.
Table 1: Results in a Nutshell
|Laurel ST200||Redback SmartEdge 800|
|Percentage of practical maximum rate forwarded without loss, 40-byte IP packets||100.00%||98.47%|
|Percentage of practical maximum rate forwarded without loss, Internet mix||99.99%||99.12%|
|Average delay in microseconds for varying loads, 40-byte IP packets||28-31||18-32|
|Average delay in microseconds for varying loads, Internet mix||48-1,759||27-69|
|Maximum delay in microseconds for varying loads, 40-byte IP packets||74-99||153-2,435|
|Maximum delay in microseconds for varying loads, Internet mix||637-8,716||554-1,452|
|Failover time for 500,000 routes, in milliseconds||9.85||57.19|
|Layer 3 (RFC 2547bis) MPLS VPNs: maximum number of virtual routing and forwarding (VRF) instances||2,420*||2,420*|
|Layer 3 (RFC 2547 bis) MPLS VPNs: maximum number of table entries per customer when supporting 2,420 virtual routers||900||300|
|Layer 2 (Martini draft) MPLS VPNs: Maximum number of tunnels||39,232*||39,232*|
|Quality of Service|
|BGP Routing Information Base (RIB) table capacity: maximum number of routes learned||5,414,062||4,000,000|
|BGP Forwarding Information Base (FIB) table capacity: maximum number of routes advertised||849,990||1,350,000|
|BGP peering capacity: maximum number of concurrent sessions||1,012||1,408|
|OSPF capacity: maximum number of link state advertisements (LSAs) supported||2,000,064||2,600,070|
|IS-IS capacity: maximum number of label switched paths (LSPs)||3,394,050*||3,394,050*|
|* Exceeds test-bed capacity|
Detailed results follow in subsequent pages:
- The Metro Edge Router
- First Things Fast
- Get Stuffed
- Dealing With Delay
- Failover & Resiliency
- MPLS: Scaling the Heights
- Martini Time
- QOS Matters
- BGP Basics
- BGP RIB Capacity
- A Big FIB
- BGP Peering
- OSPF Capacity
- IS-IS Capacity
- First Things Fast