Optical/IP Networks

Endless Ethernet?

As regular Light Readers know, for some time now I’ve been saying that Ethernet technology is set to become a big success on public networks (see Metro Optical Ethernet and Top 10 Trends).

In my opinion the interesting question now is not if Ethernet is going to make it. After all, Ethernet is inexpensive and easy to provision and scale. It’s also a very attractive interface on all types of telecom and data communications gear, thanks to the inexorable rise of IP networks and services.

Instead, we should be asking which of the approaches to deploying Ethernet is going to win out. Ethernet may have the local area locked down, but how are service providers going to transform it into a wide-area wonder — and make some money in the process?

Currently, Ethernet service providers fall into two camps: First, startup carriers like Cogent Communications Inc., Telseon, and Yipes Communications Inc., that are building their networks from scratch, renting dark fiber from Metromedia Fiber Network Inc. (MFN) (Nasdaq: MFNX) and similar outfits. These so-called EtherLECs aggregate and transport native Ethernet end-to-end, using Layer 3 switches and IP routers.

Second, incumbent providers that are looking to roll out Ethernet across the Sonet networks they have already installed. Among incumbents, the IXCs are in the strongest competitive position because they typically own a piece of the Internet backbone and have tremendous capacity built into their network cores.

On the face of it, the EtherLECs appear to be winning this battle. Their simple, streamlined scheme lets them deliver 10 or even 100 Mbit/s for the price of a typical T1 (1.544 Mbit/s). They also are garnering the lion’s share of the attention from press and analysts. And the fact that they’ve already signed customers only gives them something else to smile about.

But I think it’s too early to count out the incumbents. In fact, I’ll go further and say that I think that their Sonet-based networks could actually be an advantage, not a problem, in their forthcoming battle with the EtherLECs.

No, I’m not blind to Sonet’s traditional shortcomings. It wasn’t originally intended to carry LAN services – and it shows. For instance, Sonet doesn’t offer an easy way to efficiently share bandwidth amongst customers, or deliver granular services. That’s why it’s typically relegated to inflexible point-to-point “private line” offerings.

However, incumbent carriers have found a way around these problems by installing next-generation Sonet multiplexers and using them to deliver “Ethernet private line” services.

Sonet products that support these services are available from Appian Communications; Ciena Corp. (Nasdaq: CIEN), via its acquisition of Cyras; Cisco Systems Inc. (Nasdaq: CSCO), via its acquisition of Cerent; Mayan Networks Inc.; Ocular Networks Inc.; Sycamore Networks Inc. (Nasdaq: SCMR), through its acquisition of Sirocco; and others.

Next-generation Sonet systems are typically installed in the basement of the customer premises. The customer sends Ethernet traffic to the Sonet box over a simple 10- or 100-Mbit/s Ethernet LAN. The mux aggregates multiple Ethernet streams and maps them onto an OC12 (622 Mbit/s) or OC48 (2.488 Mbit/s) for transport over the embedded (legacy) Sonet network.

From what I’ve seen, Appian has the solution to watch. Its product can map individual ports or services to separate, secure Sonet channels with end-to-end committed bit rates. Other vendors’ offerings tend to map an entire Ethernet interface to a Sonet channel, which may compromise security or result in less-than-optimal bandwidth utilization.

The benefits to the incumbent service provider are clear. They can manage the service the same way they manage private lines, while reducing their customers’ cost of ownership considerably (giving them more of the proverbial bang — bandwidth — for the buck).

There’s more good news for the incumbents. Ethernet over superstable Sonet translates into five 9s (99.999%) reliability and the sort of security associated with TDM (time-division multiplexing) links. The best the EtherLECs can do is three or four 9s, max.

Ethernet over Sonet also gives the incumbents a way to start exploring more complex service offerings like Ethernet virtual private lines and Ethernet virtual private networks. The latter should be able to mimic frame relay at much lower cost and far greater scaleability.

How does all this compare to the EtherLECs, with their native Ethernet services? As noted, native Ethernet is catching on with startups because it lets them build lower-cost, data-centric networks that make more efficient use of metro bandwidth. The architectures are simple and elegant; the customer interface, familiar: a port on a LAN switch. A service-optimized Layer 3 gigabit Ethernet switch (like the Alpine product from Extreme Networks Inc. [Nasdaq: EXTR]) sits in the basement of the customer premises, aggregating lower-speed traffic onto a gig Ethernet pipe and adding QOS (quality of service) and service classes. This traffic is shunted over leased dark fiber to a network POP (point of presence), where it’s handed off to a router that peers with long-haul IP backbones from the likes of Genuity Inc. (Nasdaq: GENU), Qwest Communications International Corp. (NYSE:Q), and UUNet.

It may not be readily apparent, but that router represents a critical junction: The EtherLECs lose their hold on customer packets at this point (because their networks only cover metro areas), surrendering them to the best-effort IP backbone. Some EtherLECs, however, may opt for bypassing the routed core network and provision direct links between GigE switches using GigE over DWDM. This solves the control problem, but may come at significant cost of operations. It's a bit early to tell.

What about the incumbents? The ones that support Ethernet private lines can offload customer traffic onto their Sonet optical transport backbone, preserving the integrity of the “circuit” across their long-haul network.

When it comes to resiliency on the metropolitan ring, EtherLECs also have fewer options. They can go with spanning tree, which gets them two-second restoration, or the upcoming fast spanning tree, which cuts restoration to less than a second. For Sonet-like failover and protection they can opt for a vendor that supports resilient packet ring — such as Cisco, Lantern Communications, Luminous Networks Inc., Nortel Networks Corp. (NYSE/Toronto: NT), or Riverstone Networks. That will get restoration down to 50 milliseconds, as well as effectively double the capacity of each metro ring — a big plus for service providers that are leasing their fiber. A final alternative is GigE over DWDM. In this scenario, the GigE's are multipelxed onto wavelengths, which are protected by the DWDM system, not by the switches.

So what’s the best bet? Despite the limitations I’ve outlined, the EtherLEC solution seems to make more sense — at least for now — to data-oriented service providers. The network is flat, with only Ethernet equipment between the customer and the network core. That should make operations more efficient, speed service provisioning, and reduce latency. In many ways Yipes and Telseon are pointing the way to tomorrow’s pure IP networks.

At the same time, the way these service providers manage and monitor their native Ethernet is more complex than it should be. VLANs are tagged and double-tagged for security; SNMP passes for service management; and not much of anything is used for fiber plant management except some vendor-specific proprietary schemes that most incumbents wouldn’t recognize.

And what about the Ethernet private line? It’s a smart start, and it’s worth watching this year. Incumbents, especially the big IXCs, aren’t about to let the EtherLECs waltz away with their customers. The private-line approach keeps them in the game until they’re ready to roll out more sophisticated services. And now that capital markets are getting tight, it’s worth remembering that big carriers have big bankrolls. They realize that Ethernet is about to revolutionize service delivery. The time and money they’ve invested in their Sonet nets have not been wasted. There is definitely no reason to count them out of a race that is just getting started.

Where do you stand on the native Ethernet vs. next-gen Sonet debate? Is the private line approach something cooked up by the big carriers to make the best of a bad situation (Sonet by another name)? Or is this a second lease on life for two transports: Ethernet and Sonet? If so, does it mean an end to the all-packet-is-better philosophy that has ascended during the past few years?

You can sound off and weigh in by posting your opinion to our message board. Use the link below.

— Scott Clavenna is president of PointEast Research LLC and director of research at Light Reading http://www.lightreading.com
Page 1 / 8   >   >>
stuartb 12/4/2012 | 8:58:00 PM
re: Endless Ethernet? Scott or other Light Readers,

Is the Appian OSAP a transport platform, a services platform, or both?. From their positioning it's difficult to tell whether or not they support the functions of a DACS or ADM (integrated) or rely on the installed transmission infrastructure.

Also does their product support SONET Virtual Concatenation or any similar bandwidth optimization/mgmt techniques?

kennyl 12/4/2012 | 8:57:57 PM
re: Endless Ethernet? From the web site, the OSAP is fully SONET-compliant, and can plug into an existing SONET network. BTW, the OSAP also supports Ethernet transport for providers that do not have existing SONET infrastructure. On the service side the OSAP is packet and TDM capable. It allows the provisioning of multiple services, including Ethernet Private Line, Frame Relay access, ATM access, etc. Transport is optimized with the ability to share a single SONET/SDH path across multiple network users with complete QoS control.
stuartb 12/4/2012 | 8:57:51 PM
re: Endless Ethernet? To me, "SONET-compliant" and "can plug into an existing SONET network" means that it is not an optical transport box.

dgindc 12/4/2012 | 8:57:32 PM
re: Endless Ethernet? Another great discussion topic on LR!

Just as insurgent carriers have a cost advantage in delivering IP/VPNs, the EtherLECs (how about ELECs) are in a great cost position vs. the older generation because of the huge capital layout required to purchase SONET equipment. The SONET requirement to mux smaller lines into bigger ones was designed with TDM voice in mind. This forces carriers to purchase Add-Drop Muxes along their rings and superexpensive Wideband and Broadband DXC equipment to haul it out of the LATA. However, the majority of CLEC wireline revenue growth is coming from data. With 100 Megs into the customer prem as a base, there is no need for the (E)(C)(B)(Letter of your choice) LEC to multiplex until they meet the long-haul carrier. Additionally, with GigE ports descending to the $500 range, Ethernet is not just cheap to transport, its cheap to aggregate.

SONET is not and will never be a data protocol, it is a voice relic with voice features. The quality features of 50 ms restoration and 5 9's reliability do not pay for themsleves. Additionally, most customers hitting these networks are using a PC and Windows is neither 5 9's reliable nor restorable in 1/20th of a second. 3 9's and 1 second restoration is just fine for a PC. Ethernet over SONET reminds me of IP over circuit-switching, there is no need for the lower layer.
Ranger 12/4/2012 | 8:57:29 PM
re: Endless Ethernet? So I guess you would want your cellphone conversations to be over Ethernet. Analog voice translated to digital becomes data, Ethernet is a medium for data.

I don't know about you, but SONET works fine for me.
sumtnsumtn 12/4/2012 | 8:57:10 PM
re: Endless Ethernet? Course your cell phone won't be on ethernet... but it will be all IP! Ether radio links aren't all that big yet. But once it gets down... in 3G it will likely be on E!
thenight62 12/4/2012 | 8:56:40 PM
re: Endless Ethernet? Seems like an irrelevant point. Fact is, voice is moving to IP. IP over Ethernet or IP over SONET doesn't change anything. You still have to deal with traffic engineering & QoS. The new Ethernet vendors are building the QoS tools into their boxes to provide comparable QoS to TDM. I agree that SONET adds an unnecessary layer of overhead.

>So I guess you would want your cellphone >conversations to be over Ethernet. Analog voice >translated to digital becomes data, Ethernet is >a medium for data.
>I don't know about you, but SONET works fine for >me.
wimchatta 12/4/2012 | 8:56:26 PM
re: Endless Ethernet? dgindc,
It was a good point there, about Windows traffic not needing five 9's stability.

But if Ethernet were to take over networks outside your LAN, would you send payroll information, or place purchase orders over Ethernet without the reliability of a T1? Even if it is statistically a sound idea, people's levels of risk aversion are not going to change soon.

Any thoughts out there on what kinds of transmission really need five 9's vs. 3 9's reliability?
gea 12/4/2012 | 8:56:25 PM
re: Endless Ethernet? There are some truly odd comments that pop up there every now and then. Data people really seem to have no clue about telecom networks.

First of all, a LAN link running GbE is very different from an OC-48 backbone link. In one case, a single corporation may have part of its network go down, or connectivity with the outside world lost. Depending on the corporation, that may or may not matter. With a backbone OC-48/192 link, you'll have thousands of customers (a whole town or small city) lose its ability to communicate if the link fails. Thus five 9s means something very different in this context.

But in any event, the fact that many applications (or even, arguably, most) do not need "five 9s reliability" is not necessarily the issue. The question is, will a carrier buy MAN/WAN gear that will forever and always prevent them from offering 5 9s of reliability, when tried-and-true technology exists that will provide it? Perhaps this is why native GbE transport over MANs is almost nonexistent (no, not even Cogent is running it native). And the cost of transporting anything over SONET (even packets a la packet-over-SONET) is falling dramatically.
LocalYokel 12/4/2012 | 8:56:24 PM
re: Endless Ethernet? Won't 802.17 solve the five 9's issue for ethernet?
Page 1 / 8   >   >>
Sign In