Why the IEEE Picked 400G Over Terabit

SANTA CLARA, Calif. -- Ethernet Technology Summit -- Anyone hoping for a mad rush towards Terabit Ethernet specifications will have had their expectations dampened here in Santa Clara this week, with the age-old hurdle of economics to blame for the hold-up.

In a Wednesday keynote here, John D'Ambrosia, chief Ethernet evangelist for Dell Inc., updated the status of the 400Gbit/s standards effort and defended the Institute of Electrical and Electronics Engineers Inc. (IEEE) decision not to work on a 1Tbit/s standard yet.

"In some cases, technology has to catch up in advance. I think that's a key point to why terabit isn't happening yet," D'Ambrosia said.

The 400Gbit/s effort starts officially in May, when the study group meets for the first time. (D'Ambrosia is acting chairman but hasn't been officially named chairman; they'll need to actually meet in order for that to happen.)

The IEEE 100Gbit/s standard took four years to ratify, starting from the first study group meeting, so it's reasonable to guess that a 400Gbit/s standard might be finalized in 2017.

Why not chase Terabit Ethernet instead? Some people have suggested that, but the economics just don't work out yet, D'Ambrosia said. "The question, ultimately, is doing it at the right cost so the cost per bit continues to fall."

Cost is also going to dictate some of the details of 400Gbit/s. For example, a lot of investment has gone into interfaces that divide signals into 16 lanes -- the 300-pin transponder and certain optical connectors being examples. Those 16-lane formats are likely to be used for 400Gbit/s, partly because they already exist, and partly because the math divides out to 25Gbit/s per lane, which is the channel speed that electronics are starting to reach.

Terabit Ethernet wouldn't enjoy the same advantage. Even when 50Gbit/s electronics exist, that would require a 20-lane interface -- which, while possible, would add to the cost of developing 1Tbit/s devices.

The International Telecommunication Union, Standardization Sector (ITU-T) also appears to be leaning towards 400Gbit/s rather than 1Tbit/s for the next optical speed, but D'Ambrosia cautioned against confusing that effort with the IEEE's Ethernet standard. They're not the same thing.

The groups are cooperating, though. "We've put a stake in the ground now for the ITU-T to build on. In previous generations it was kind of a moving target," he said.

D'Ambrosia also took a moment to invite silicon photonics vendors to come to the IEEE meetings. "Make your case there" instead of in the press, he said.

Silicon photonics doesn't have to be regarded in the 400Gbit/s standard, since the IEEE doesn't standardize particular implementations of technology, but D'Ambrosia's point was that the silicon photonics camp seems to think it's got something to contribute right now. "For those people who think it should, I extend the invitation to get involved," he said.

For more

— Craig Matsumoto, Managing Editor, Light Reading

sjtrowbridge 4/17/2013 | 6:39:13 PM
re: Why the IEEE Picked 400G Over Terabit

As has been pointed out, Mr. Vissers as an individual
delegate is representing his own (or his companyGÇÖs) opinions and not the views
of ITU-T Study Group 15. This organization has a very broad, diverse membership
with a wide range of opinions that need to be reconciled to reach the consensus
for what provisions to specify in a Recommendation. Study Group 15 makes
available for free download all of its approved, published Recommendations. As
is the practice for many other standards organizations, Study Group 15 does not
normally provide access beyond its membership for work in progress or proposals
which have not reached the level of agreement to be included in an approved

Dr. Stephen J. Trowbridge
Chairman, ITU-T Study Group 15

Landshark 4/9/2013 | 4:09:43 PM
re: Why the IEEE Picked 400G Over Terabit Please keep in mind that Maarten
presented one ITU-T participantGÇÖs views on the work started at Question
11/Study Group 15 (Q11/15).-á Maarten no doubt is a valuable editor in
Q11/15, but his opinions do not represent the views of all those involved.-á
The ITU-T results require a consensus of the participants for technical meeting
agreements and then approval by the countries involved (as the ITU-T is a United
Nations technical committee), before they are published.-á The work on
beyond 100G is still in its early stages in Q11/15, with few agreements reached
so far.-á Those agreements to date include that it is too early now to
standardize a new OTN frame.-á Before that could happen, a better
understand of the options and the implications of those options are
required.-á For that reason, contributions on the topics that Maarten
described are being discussed.-á There also seems to be general consensus in
Q11/15 that the first fully interoperable client side OTN interface (IrDI)
should be defined to carry the anticipated IEEE 400GbE, as Maarten wrote.-á
As is pointed out in the article, that 400GbE standard will likely take years
to complete and thus the ITU-T will not know all that is needed to specify a
400G OTN interface for some time.-á In parallel, discussions will continue
around the line side interfaces.-á If consensus can be reached on decoupling
the line and client side interfaces some level of specification on the line
side interfaces might precede a standard for the client side 400G interface. As
an active participant in Q11/15, I personally will welcome all the comments,
input and suggestions for the creation of standards for the B100G systems.
jdambrosia951 4/9/2013 | 4:55:13 AM
re: Why the IEEE Picked 400G Over Terabit Craig - from your headline i guess i wasn't clear at ETS last week. The "IEEE" didn't chose anything.-á Consensus generated during the IEEE 802.3 HSE Consensus Ad Hoc led to the Call-For-Interest, and then ultimately, it was the individuals who voted to approve the formation of the Study Group that made the decision!-á THis is a key point.-á Decisions are not made by the IEEE - they are made at the IEEE... by those individuals who choose to participate.

In regards to the variable rate suggestion - this is not a new idea.-á That idea was considered by the previous Higher Speed Study Group - and it was agreed then that was not a cost effective idea.-á Further, the idea was never proposed during the HSE Consensus Ad hoc, where consensus building on 400GbE was high.-á

sterlingperrin 4/8/2013 | 1:45:02 PM
re: Why the IEEE Picked 400G Over Terabit Maarten, this is excellent information. Thanks so much for this update!
mvissers 4/5/2013 | 7:46:43 PM
re: Why the IEEE Picked 400G Over Terabit The Beyond-100G extension of the OTN is a main task at the moment in ITU-T SG15, specifically in Q.11/15. The discussion was started in May 2010, under the assumption that the next rate would be 400G ODU5/OTU5. In Dec. 2011 a contribution was presented suggesting that the next rate should be a set of N times a basic rate, with N variable. In Sept. 2012 several contributions were presented to support this N x basic rate proposal, and basic rate values of 25G, 50G, 100G, 200G and 400G were evaluated. Most organisations expressed a preference for-áa basic rate of 100G, but it was too early to make any decision.-áDuring the 4th quarter of 2012 a lot of analysis was performed to gain understanding how a N x 100G extension of the OTN could look like.-áThe result of this work was-ásocialized in the-áFeb. 2013 meeting of Q11/15. There are many common elements in the proposals presented during this meeting and there are some differences.

The majority of organisations support the idea of a N x 100G ODU/OTU frame format, of which the ODU frame has N x 3824 columns by 4 rows, and the OTU frame has N x 4080 columns by 4 rows. [Note that this N x 100G ODU/OTU frame format has a structure that is similar-áto the well known STM-N frame (N x 270 columns by 9 rows).] The overhead proposals are suggesting to define the overhead in a similar manner as the STM-N RS and MS overhead. There is no agreement yet to accept this frame format and overhead-áand further discussions will take place during the SG15 meeting in July 2013.

OTN ASIC designers have indicated that there is a preference to use the 100G ODU4/OTU4 circuits as basic building blocks in the Beyond 100G OTN Framer devices. This is one of the drivers behind the N x 100G preference.

There are four different-áinterfaces in the OTN.
1) single channel, client side interface: the well known single channel-áInter Domain Interface (IrDI) that will interconnect administrative domains
2) multi channel, client side interface: the-ámulti channel IrDI, which is not really used today
3) single channel, line side interface for metro networks: the multi-vendor Intra Domain Interface (IaDI)
4) multi channel, line side interface for metro, core, backbone networks: the single-vendor IaDI.
The N x 100G ODU/OTU is expected to be used on the 4th, 1st and 3rd-áinterfaces.

The 4th interface (single vendor)-áwill carry a number of N x 100G ODU/OTU signals (N >= 2). [There has not yet been a discussion of the maximum value of N, and this will effectively be determined by the available technology. I have suggested that N = 256 (25.6 Tbit/s) could be the maximum; it will fill the complete C-band in a metro network and the C+L-bands in a core/backbone network.]-áA number of proposals suggest that-áthe vendor should determine the values of N. This interface is the most important interface for the OTN XCs and all vendors are working on designs of this interface.

For the case that SG15 standardizes this N x 100G ODU/OTU signal, the line side bit rates up to e.g. 25.6 Tbit/s will be known on the day of approval of this B100G extension. Line side interfaces are then effectively de-coupled from the client side interfaces and we are back to the situation we have in SDH where the line side interfaces are also de-coupled from the client side interfaces.

The 1st interface will carry-áinitialy a-á4 x 100G (400G)-áODU/OTU signal; later on this interface may also support standardized higher rate signals, e.g. 10 x-á100G (1T)-áOTU or 16 x 100G (1.6T) OTU. The 400G OTU IrDI and the new 400GE interface are intended to use the same 400G optical module, similar to today's use of the 100G optical module for both the-á100G OTU4 IrDI and the 100GE interface. This interface will become available after IEEE has finished its 400GE development.
The understanding is that the client side interfaces don't need so much flexibility, and can live with a subset of the line side bit rates.-á

The 3rd interface is the most complex interface as it requires interworking between N x 100G optical transmitters and receivers of different vendors and with the all-optical subnetwork of a third vendor. The N x 100G ODU/OTU frame can be specified, but the optical parameters and therefore the N x 100G OTU FEC require much more study.

The bit rate of such N x 100G ODU/OTU signal is under discussion as well; proposed N x 100G OTU bit rates-áare-áN x 255/DIV x 99.5328 Gbit/s, in which the DIV could be (1) 225, (2)-á225.5, (3) 226 Gbit/s, (4) 227. The latter represents N x the OTU4 bit rate, the other three represent a slightly higher bit rate in order to carry multiple ODU4 signals in a N x 100G ODU/OTU. ODU/OTU signals with the first three bit rates are referred to as "ODUCn and OTUCn" signals; "C" represents the Roman value of 100. The 4th ODU/OTU signal is referred to as "ODU4Cn and OTU4Cn" signals.

All N x 100G ODU/OTU bit rates can support the transport of a 400GE client signal with a bit rate of 412.5 Gbit/s (400G with 64B/66B encoding). Some less efficient 400GE encodings can be supported as well, as long as it will not increase the encoded 400GE bit rate with more than ~1%. For the case IEEE would select a higher rate, either the N x 100G ODU/OTU bit rate must be increased beyond the listed values, or xB/yB to 64B/66B Transcoding should be used (in analogy with the Transcoding applied in the-á1GE to ODU0 and 40GE to ODU3 mappings).

Some proposals suggest that the N x 100G ODU signals should not be passed through an electrical switch fabric, but instead terminate on the line cards (like the STM-N signal is terminated on its line cards). Such design will stop the ODU-in-ODU-in-ODU-in.. hierarchy. Beyond 100G client signals are in those proposals mapped into an ODUflex and the ODUflex is multiplexed into the N x 100G ODU/OTU. Other proposals suggest that the 400GE client signal is mapped into the-á4 x 100G ODU. Further discussions are expected in the July 2013 meeting.

sterlingperrin 4/5/2013 | 2:12:01 PM
re: Why the IEEE Picked 400G Over Terabit Last I had checked in, there was a strong movement on the ITU side to create this flexible line rate, based on standardized building blocks, that could scale up and down but would not be a fixed number. The flexible option 1. effectively de-couples the line side from the client side and 2. eliminates the need to keep re-convening on new line side bit rates. I thought this was a great idea, but I'm wondering if line side momentum has swung back to matching IEEE client rates?

A second thought (and I may bet tomatoes thrown at me for this one): Everyone talks of "exponential IP traffic growth" but the actual growth rate numbers have lowered quite a bit over the past 10+ years. It used to be 100%+ annual growth, then 50%+. In our most recent survey, operators are saying 30-50% annual growth for the most part. I am wondering if these growth rates are being factored in to decisions to favor 400G over 1T?

Craig Matsumoto 4/4/2013 | 3:46:10 PM
re: Why the IEEE Picked 400G Over Terabit "Come get involved" was the main point of D'Ambrosia's talk.-á I get the impression he hears a lot of suggestions (complaints) from people who never come to the IEEE meetings. I didn't ask him about the variable-speed Ethernet suggestion (in the Google link in the story), but I'd imagine the answer would be the same: It's not cost-effective for this go-around, but come to the meetings so you can get the discussion started with the right people.
Sign In