Optical/IP Networks

Metro Edge Router Test

Who says the Internet boom is over? Carriers may still be caught in a capex crunch, and Wall Street may still be nursing a nuclear hangover, but in networking the pace of innovation never slows down. A new breed of box, the metro edge router, gives providers the technology they’ll need to roll out new services and scale those services to new levels. That’s not just marketing hype: These routers actually work pretty much as promised in delivering services like MPLS VPNs, QOS enforcement, and routing scaleability.

Light Reading, along with its testing partners – Network Test of Westlake Village, Calif., a benchmarking and network design consultancy; and Spirent Communications of Calabasas, Calif., a test equipment supplier – has just completed a massive edge router trial. We pounded the Laurel Networks Inc. ST200 and Redback Networks Inc. (Nasdaq: RBAK) SmartEdge 800 in the most rigorous routing test we’ve ever conducted.

Make that ten tests: Our methodology covered everything from basic throughput to resiliency testing to MPLS Layer 2 VPNs. This sizeable undertaking was more than nine months in the making, and involved a staff of more than 40 engineers and project managers (see Thanks).

The good news? Both vendors passed with flying colors.

Among the high points:
  • Using Layer 3 MPLS VPNs, both boxes emulated more than 2,400 virtual routers and handled hundreds of routes per customer.

  • Layer 2 MPLS VPNs scaled even higher, with both products forwarding traffic through nearly 40,000 tunnels. As far as we’re aware, both boxes set new records for public tests of MPLS VPN scaleability.

  • Routing capacity tests for both boxes produced absurdly high numbers, typically one or even two orders of magnitude beyond today’s levels.
If there’s any bad news to be had, it’s that market leaders Cisco Systems Inc. (Nasdaq: CSCO) and Juniper Networks Inc. (Nasdaq: JNPR) didn’t put their products to the test.

We weren’t too surprised about Cisco: Judging from data sheets and anecdotal evidence, its current products wouldn’t have fared well at all with parts of our methodology. We’ve also heard there are better products in the pipeline.

Juniper had no such excuse: In fact, the data sheet for the vendor’s M40e (e as in edge) served as one of our guides in putting together a test methodology. Even more promising, Juniper bought Unisphere Networks, the leading edge router maker, while we were preparing for this project. We looked forward to testing one and possibly two entries from Junisphere.

Juniper actually did agree to take part, but the deluge began soon after it swallowed Unisphere (or was it the other way around?). First, we heard multiple reports of internal strife. Then we got word that Juniper was officially withdrawing from the test (accompanied, of course, by reassurances that all was well internally). Then followed another cycle of reconsideration, but ultimately Juniper sat this one out. Too bad: Clearly it was politics, not products, that prevented Juniper from putting its best foot forward.

We also gathered an impressive collection of excuses from the more than 20 other vendors that opted out (please see No Shows before asking why this vendor or that one isn’t represented here). Some of these vendors simply didn’t have appropriate product; our requirement for OC48 (2.5 Gbit/s) interfaces ruled out several potential players. Others had reasons of their own.

Whatever shortcomings we uncovered in the Laurel and Redback products (and there were very few), both vendors deserve major credit for their willingness to submit their products to public testing.

If we had to choose between the Laurel and Redback routers, we’d give the edge to Laurel’s ST200, while noting that good arguments can be made for either router. Laurel’s product fared better in the baseline throughput and failover tests, and it scaled much higher in the Layer 3 MPLS VPN tests. For its part, Redback’s SmartEdge router did better in some baseline latency and routing scaleability tests (though not by meaningful margins on the latter, in our view). Both boxes turned in excellent results overall.

A summary of all the tests and results are provided in the table below.

Table 1: Results in a Nutshell
Laurel ST200 Redback SmartEdge 800
Percentage of practical maximum rate forwarded without loss, 40-byte IP packets 100.00% 98.47%
Percentage of practical maximum rate forwarded without loss, Internet mix 99.99% 99.12%
Average delay in microseconds for varying loads, 40-byte IP packets 28-31 18-32
Average delay in microseconds for varying loads, Internet mix 48-1,759 27-69
Maximum delay in microseconds for varying loads, 40-byte IP packets 74-99 153-2,435
Maximum delay in microseconds for varying loads, Internet mix 637-8,716 554-1,452
Failover time for 500,000 routes, in milliseconds 9.85 57.19
VPN Scaleability
Layer 3 (RFC 2547bis) MPLS VPNs: maximum number of virtual routing and forwarding (VRF) instances 2,420* 2,420*
Layer 3 (RFC 2547 bis) MPLS VPNs: maximum number of table entries per customer when supporting 2,420 virtual routers 900 300
Layer 2 (Martini draft) MPLS VPNs: Maximum number of tunnels 39,232* 39,232*
Quality of Service
Rate enforcement Passed Passed
Rate shaping Passed Passed
Routing Performance
BGP Routing Information Base (RIB) table capacity: maximum number of routes learned 5,414,062 4,000,000
BGP Forwarding Information Base (FIB) table capacity: maximum number of routes advertised 849,990 1,350,000
BGP peering capacity: maximum number of concurrent sessions 1,012 1,408
OSPF capacity: maximum number of link state advertisements (LSAs) supported 2,000,064 2,600,070
IS-IS capacity: maximum number of label switched paths (LSPs) 3,394,050* 3,394,050*
* Exceeds test-bed capacity

Detailed results follow in subsequent pages:

— David Newman is president of Network Test, an independent benchmarking and network design consultancy based in Westlake Village, Calif. He likes surf guitar and bicycle racing, and his turnoffs include mean people and anonymous troll posts on Light Reading’s message boards. Newman can be reached at [email protected]. Next page: The Metro Edge Router

1 of 15
Next Page
Page 1 / 4   >   >>
dnewman 12/5/2012 | 12:54:56 AM
re: Metro Edge Router Test
A reader emailed me privately with a good question about Light Reading's metro edge router test. I've pasted the reader's question, regarding the overhead introduced by bit and byte-stuffing, below and I will paste my response in a separate posting.

David Newman
Network Test


I'm hoping you can clarify something about your worst-
case packet rate test - see:


There were two paragraphs that have me a little puzzled:

"In the tests with 40-byte packets, LaurelG«÷s ST200 had the highest
throughput G«Ű 42.4 million packets per second (pps), or about 98.13 percent
of line rate G«Ű compared with 41.8 million pps (about 96.6 percent of line
rate) for RedbackG«÷s SmartEdge 800 (see Figure 2)."

"A bit here and a byte there doesnG«÷t sound like much, but it adds up. In our
40-byte IP packet tests we found that bit- and byte-stuffing reduced the
practical maximum throughput by nearly 2 percent. The actual maximum depends
on packet contents; for example, a router will bit-stuff like crazy when it
sees a pattern of all 1s."

The reason I was confused was because of the 2% of overhead.
For HDLC/PPP I would expect there to be at least 2 bytes of
overhead for each and every packet - these being one flag
byte separating each frame, and 1 protocol byte per frame
to describe the encapsulated packet. Even just 2 bytes is
a 5% overhead on the line for 40 byte packets.

I'll refer to the RFC for PPP in HDLC framing (rfc 1662) and
PPP (rfc 1661), as well as PPP LCP Extensions (rfc 1570):


The only way to get down to 2 bytes per 40 byte packet is by
negotiating a PPP link with the following options enabled
(see rfc 1570 above):

- protocol field compression
- address and control field compression
- frame check sequence (fcs) alternatives

Can you verify that this was the scenario tested?

The only other possibility I can think of is using another
PPP option called compound frames, which packs multiple
packets into each PPP frame. You still need a single protocol
byte to precede each of the multiple packets within the PPP
frame, but the overhead will tend toward 1 byte in 40, which
is still 2.5%. Any bit stuffing overhead would be in addition
to this 2.5%.

Would it be possible to publish a more detailed breakdown
of the PPP parameters used during the test?

dnewman 12/5/2012 | 12:54:55 AM
re: Metro Edge Router Test
Here is my reply to a reader's inquiry about the amount of overhead added by bit- and byte-stuffing in Light Reading's metro edge router test.

The fields that are always present in PPP frames, like start/stop delimiters and protocol fields, are not escaped. After all, PPP wouldn't work very well if these fields were
canceled out by escaping.

Escaping only occurs when streams intended as control sequences -- such as the 7E start/stop delimiter -- occur *elsewhere* in the frame. Examples of these other places include IP destination addresses, IP checksums, and test sequence numbers.

However, and I think this is the crux of the issue, it is not valid to assume that the IP checksum will *always* be escaped. In fact,
there is only a 1 in 256 probability of byte stuffing for the start/stop delimiter. Thus the overhead added by escaping is much lower than 5

This is the important part: Overhead increases when, and only when, a control sequence is actually escaped.

On OC-48 POS, we calculated theoretical maximum throughput using 9 bytes of overhead per 40-byte IP packet:

4 bytes = preframe (PPP header)
4 bytes = postframe (32-bit CRC)
1 byte = start/stop delimiter

Sonet overhead reduces the bandwidth available to frames to 2,396,160,000 bit/s. With 49-byte frames at that rate, the theoretical maximum throughput on OC-48 POS with 40-byte IP packets and a 32-bit CRC is 6,112,653.06 fps.

But that's only the *theoretical* maximum. As we discuss in the article, the *practical* maximum throughput may be lower because of byte-stuffing. The amount of extra overhead is not deterministic. As I said before, there is a 1 in 256 (0.390625 percent) chance of a random byte being a 7E. But that's only on randomly occurring bytes. In our POS packet, only a few fields are subject to randomness:

2 bytes = IP header checksum
1 byte = IP source address
1 byte = IP destination address
16 bytes = test sequence number
4 bytes = PPP CRC

So, out of 49 bytes, only 24 bytes were subject to potential byte stuffing. (Note that I'm only using 1 byte for the IP addresses because it was only in the first octet that byte stuffing would have occurred; we didn't use the 7E pattern anywhere else in our addresses.) Although the probability of all 24 bytes being stuffed simultaneously is remote, it could add very substantial overhead.

But this never did happen because we never drove the OC-48 circuits at line rate. Had we done so, the aggregate offered rate for all 4 core-side (OC-48) interfaces would have exceeded that of the customer-side (DS-3, gigE) interfaces by 11.57 percent. Since we do *not* want to congest the DUT during a baseline test, we reduced the offered load to the OC-48s to match the aggregate rate we offered the customer side interfaces.

Therefore, stuffing wasn't a significant factor with OC-48. It was a significant factor on the DS-3 PPP circuits, since these we drove these as close to maximum as possible. There, our calculations show that bit-stuffing (not byte-stuffing) added overhead of ~1.6 percent *below* the theoretical line rate of 117,578.97 fps.

David Newman
Network Test

Disclosure: At the time of testing, neither Network Test nor I held any financial interest in any of the router makers discussed in this article.
Doris D 12/5/2012 | 12:54:35 AM
re: Metro Edge Router Test So how does this relate to "The Internet Core Routing Test: Complete Results - 6/29/2001 "?


andrwsc 12/5/2012 | 12:54:03 AM
re: Metro Edge Router Test David, you wrote:
> In fact, there is only a 1 in 256 probability
> of byte stuffing for the start/stop delimiter.

Of course, there is also a 1 in 256 probability that a 7D byte value in the frame (== HDLC escape octet) will itself need to be escaped.

Therefore, there is double the amount of extra overhead that you imply in your message (still a small value, however).
skeptic 12/4/2012 | 9:13:01 PM
re: Metro Edge Router Test When the MPLS VPN route capacity was measured,
did you verify that you could forward on all
the routes being advertised?

The most extreme test was 2420*900 = 2178000
routes. I have some doubt that all those prefixes
got down to the line card, but its possible and
I would like to know if you tested that case.
fiber_r_us 12/4/2012 | 9:13:01 PM
re: Metro Edge Router Test Kudos for Laurel and Redback for participating and shining in the tests.

It seems *very lame* that Juniper, Nortel, Riverstone, and Vivace were "resource constrained"! If Laurel and Redback can do it with their limited resources, these others are clearly simply not interested in competing. What a joke.
fiber_r_us 12/4/2012 | 9:13:00 PM
re: Metro Edge Router Test LR behavior after the core routing tests clearly favored Juniper... Yet they didn't participate.

It seems that this test was run relatively well with a lot of the ideas resulting from the core routing tests incorporated.

At least Cisco honestly said their competing product wasn't ready yet.
skeptic 12/4/2012 | 9:13:00 PM
re: Metro Edge Router Test It seems *very lame* that Juniper, Nortel, Riverstone, and Vivace were "resource constrained"! If Laurel and Redback can do it with their limited resources, these others are clearly simply not interested in competing.

Or alternatively they don't have much confidence
in light reading's behavior considering some
of the stuff that went on after the core router
tmc1 12/4/2012 | 9:12:47 PM
re: Metro Edge Router Test skeptic,

depending on how many linecards i do not doubt it. it would be fairly possible to have 512-768k FIBs per linecard. if they spread it across 4 or 5 linecards then i don't doubt they could do this. 2547 makes it easy due to distributed nature of routes compared to unique BGP4 routes from the internet as steve from laurel already commented on.

The most extreme test was 2420*900 = 2178000
routes. I have some doubt that all those prefixes
got down to the line card, but its possible and
I would like to know if you tested that case.
tmc1 12/4/2012 | 9:12:47 PM
re: Metro Edge Router Test no ISP that i know of would ever do this... every router may run BGP4 but you almost never would redistribute EGP routes into an IGP. a big "no-no" i think.


We asked vendors to configure their routers to redistribute all 500,000 routes into OSPF, as an ISP typically would do. Then, we set up the Adtech analyzer on the G«£customerG«• side of the test bed to offer traffic to all 500,000 routes at a rate of 1 million packets per second. At that rate, each lost packet would equate to 1 microsecond of downtime.

Page 1 / 4   >   >>
Sign In