Traffic Managers Update

With carriers and service providers looking to maximize operating profits, a key requirement for new network buildouts will be the management and billing of bandwidth. To meet this need, equipment manufactures are looking for significantly more capable solutions for queuing and scheduling of packets.

This traffic management function has traditionally been handled using ASICs designed in-house. The introduction of third-party traffic managers – at 622-Mbit/s, 2.5-Gbit/s, and 10-Gbit/s rates – from both network processor vendors and specialist traffic-manager startups now gives designers a very real alternative.

Light Reading surveyed this market in an earlier report entitled Traffic Manager Chips. But, like the rest of telecom, this market sector is constantly changing, and already it's time for an update. Vendors have introduced a variety of new products, and the market is consolidating with startups quitting the market or getting acquired (see ZettaCom Finds a Home).

For the latest information on this rapidly-developing market take a look at the following pages, which cover high-performance traffic managers from leading vendors, including:

Here’s a hyperlinked summary of the report:

Webinar

This report was previewed in a Webinar moderated by the author and sponsored by Motorola and Teradiant. It may be viewed free of charge in our Webinar archives by clicking here.

Background Reading

— Simon Stanley is founder and principal consultant of Earlswood Marketing Ltd. He is also the author of several other Light Reading reports on communications chips, including Packet Switch Chips, Traffic Manager Chips, 10-Gig Ethernet Transponders, Network Processors, and Next-Gen Sonet Silicon.

Traffic managers can be used in systems for a number of markets, covering rates from 622 Mbit/s to 10 Gbit/s and above. The main application areas are Access, Metro Edge, and Metro Core, although there is significant overlap among them.

In-Stat/MDR (Feb 2004) is forecasting the market for standard traffic managers to "grow with a Compound Annual Growth Rate of roughly 42% from 2002 to 2007." A major element of this growth is an expected trend away from in-house ASIC designs toward standard traffic managers, which currently account for less than 10 percent of the overall market.

“Companies like Juniper are continuing to develop their own core ASICs, but for functions like SAR and ATM traffic managers they seem to be using third-party vendors,” says Kris Harikrishnan, Product Marketing Manager with Azanda.

Traffic management devices and chipset are currently available from 11 companies. There are several startups, and we can expect some consolidation during 2004. The first acquisition for some time in the area was the purchase of ZettaCom by IDT.

The traffic manager market share for 2002, as reported by In-Stat/MDR in December 2003, is shown in Figure 1. Some significant changes to these market shares are likely for 2004 as systems based on the latest traffic managers enter production.

This report covers traffic managers supporting bandwidths of 622 Mbit/s and above. It also covers some network processors that provide advanced traffic management. These devices – such as the Agere APP550, AMCC nP3700, and Bay Microsystems Montego – can be used for both packet processing and traffic management or as standalone traffic managers. They therefore merit inclusion in this report.

The Agere devices require a different firmware load when used for traffic management and are given a "TM" suffix (e.g., APP550TM). For lower-performance applications (below 622 Mbit/s), communications processors with built-in traffic management functions can be used.

The following interactive table summarizes and compares the key features of traffic manager chips:



Dynamic Table: Traffic Manager Chipsets

Select fields:
Show All Fields
CompanyFull Duplex ChipsetChipset ThroughputPer Flow SchedulingMinimum RateRED CurvesClasses/ GroupsVirtual PortsPhysical PortsProgrammable?ClassificationPolicing/ StatisticsATM SARMulticast RootsMulticast BranchesHost InterfaceNPU/PHY InterfaceSwitch InterfacePacket Buffer Memory Max Packet Buffer Size (Mbytes)Control MemoryFull Duplex PowerFull Duplex # DevicesFull Duplex PriceSample Availability

Shaping and Queuing are the core traffic management functions.

One opinion on the number of queues that are required for the different markets is shown in Figure 2 below from Teradiant.

The number of flows that can be independently scheduled is a good guide to the capabilities of a traffic manager. Within each group in Table 1 on the previous page, the traffic managers are listed in order based on this per-flow scheduling capability.

Shaping

The traffic coming into a traffic manager must be shaped to ensure conformance with rate agreements and to avoid network congestion. An important parameter for traffic managers implementing these functions is the minimum rate. This is the lowest rate at which a traffic manager can forward packets or cells. For IP metro applications, coarse-grained traffic managers supporting rates down to 1 Mbit/s are adequate. For multiservice applications, fine-grained traffic managers supporting rates down to 48 kbit/s and below are required.

To avoid congestion in networks, a switch or router must drop cells or packets in a controlled manner or reduce the traffic arriving at the switch. The main mechanism for congestion avoidance in packet networks is Weighted Random Early Detection (WRED). For each flow or group of flows the traffic manager maintains a number of drop profiles or RED curves. These RED curves determine whether packets should be dropped, as the network becomes congested (see Figure 3). Each curve may have a small or large number of points defined. Different curves can be used depending on whether the traffic meets the service-level agreement or not.

The number of RED curves supported by a traffic manager limits the combination of groups controlled and the number of drop profiles for each group. Azanda's devices have a limit on the total number of points on all the RED curves.

Most traffic managers support WRED and all other relevant congestion avoidance mechanisms.

Queuing

When packets arrive into a traffic manager they are stored in the packet buffer and a descriptor is passed to the control memory. The descriptors are placed in a queue. When the descriptor reaches the front of the queue the packet is retrieved from buffer memory and transmitted. All traffic managers support multiple queues in a hierarchy. This multi-stage queuing hierarchy is shown in Figure 4.

The number of queues supported at each stage varies across traffic managers. At each stage there is a scheduling mechanism. There are a number of standard scheduling mechanisms including "fair queuing" and "round robin." There are also "weighted" versions of these mechanisms. Several vendors have developed their own variations, a further source of differentiation among them.

“The minimum quanta for our smooth deficit round robin is a minimum packet size, whereas for other deficit round robin [implementations], the quanta is a maximum packet,” notes Rob Munoz, product marketing manager with Agere.

“We can offer customers unique scheduling and queuing algorithms,” says Neil Cohen, product marketing manager with Mindspeed. “A good example of that is our modified deficit round robin with low latency queuing.”For further details on shaping and queuing mechanisms, see our previous traffic manager report: Traffic Manager Chips.

As well as the core queuing and shaping functions, traffic managers may perform a number of additional functions, including classification, policing and statistics, ATM SAR, and multicast. These are covered in this page.

Programmable Traffic Managers

Traffic management has traditionally been implemented using hardwired-state machines. The TSP3 traffic management devices from Mindspeed include a processor core allowing the user to program the functionality. This provides significantly more flexibility, especially for the peripheral functions covered here. Similar flexibility can also be achieved by using network processors with strong traffic management features like a dedicated traffic manager. Such devices are available from Agere, AMCC, and Bay Microsystems and are included in Table 1. Agere provides traffic management-specific software for this application.

Classification

Several traffic managers will classify incoming packets, adding a tag as required. This functionality is included in all programmable traffic managers and the Scimitar and Saber devices from Azanda. For applications with limited packet processing requirements these devices can be used without a network processor or packet processing ASIC.

Policing and Statistics

The policing function will ensure that the received packets or cells are within a contract agreed to by the service provider. There are various methods of marking, but a common one is to use three-color marking where the packet is marked as red, yellow, or green. The shaping function will employ the results, with those marked red being most likely to be dropped and those marked green least likely to be dropped.

Statistics are required for billing and diagnostics. These may be stored on a per-flow or per-aggregated-group basis. Some devices support on-chip counters that interrupt the host processor on overflow. Other devices provide a statistics port that requires external hardware to record events. All devices provide some level of statistics.

ATM SAR

For interworking between ATM networks and packet networks, an ATM Segmentation and Reassembly (SAR) function is required. Most traffic managers support the basic SAR functionality, splitting packets up into fixed-size cells. The exceptions are the Freescale Q5 and EZChip QX-1, which are designed to work closely with network processors that already include SAR functionality. Not all traffic managers support the full ATM SAR functionality; those that do not are marked "partial" in Table 1.

Multicast

All networks these days must support IP and therefore must support multicast. Multicast is the transmission of a single packet to multiple destinations and is extensively used by IP. Ethernet has built-in support for multicast; however, other Layer 2 protocols such as ATM do not. For these other protocols and for higher-layer routing, the single packet must be duplicated with a copy for each destination. These multicast copies are then either sent over divergent paths through a switch (spacial multicast) or in-line down a single path (temporal multicast).

All traffic managers, except the EZChip QX-1, support multicast, with the number of source packets (roots) ranging from 1K to 256K and the number of copies of each packet (branches) ranging from 128K to 64K. The traffic manager and the control memory attached may limit the total number of branches from all packets.

Most traffic managers have flow-through architectures – i.e., packets pass through the traffic manager. The packet buffer is connected to the traffic manager. The exception is the Q5 from Freescale, which has a look-aside architecture, with the packet store connected to the network processor and descriptors being passed to the traffic manager.

A typical flow-through traffic manager has five interfaces, as shown in Figure 5.

The primary interfaces are: the NPU/PHY interface connecting the traffic manager through a network processor or direct to the framer; and the switch interface connecting the traffic manager to the switch. All the traffic managers except the EZChip QX-1 have a host interface through which the control plane processor on the linecard can be used to configure and monitor the device.

Most traffic managers also have two memory interfaces; one for the packet store, typically DRAM, and one for the control memory, typically SRAM. Traffic managers may have more than one physical interface to support each of these memory blocks.

NPU/PHY and Switch Interfaces

For multiservice applications at 622 Mbit/s and 2.5 Gbit/s, traffic managers support either UTOPIA or POS-PHY Level 2 or Level 3 (U2/3, PL2/3) for both NPU/PHY and switch interfaces. Traffic managers supporting these interfaces can be connected directly to most standard network processors and framers.

POS-PHY Level 3 has been standardized by the Optical Internetworking Forum (OIF) as SPI-3. SPI-3 can be configured as a single 32-bit bus supporting 3.2 Gbit/s or four 8-bit buses supporting 800 Mbit/s each. The Azanda Scimitar is the only 2.5-Gbit/s device to support the 10-Gbit/s SPI-4.2 interface.

At 10 Gbit/s, there is significantly greater variation among vendors. On the NPU/PHY side, most devices support the SPI-4.2 interface developed by the OIF. The decision by Intel Corp. (Nasdaq: INTC) to use this interface on its IXP2800 has been a major factor in persuading companies to adopt this interface.

On the switch side, most devices support CSIX-L1 or NPSI or SPI-4.2. The latest devices from IDT/ZettaCom support all three interfaces. The Network Processing Forum (NPF) developed CSIX-L1 as a standard interface between network processors and switch fabrics. The Network Processor Streaming Interface (NPSI) maintains many of the features of CSIX-L1 but is based on the OIF SPI-4.2 interface. SPI-4.2 has a 4-bit-wide bus supporting a maximum bandwidth of 12.12 Gbit/s.The EZChip QX-1 and the ZettaCom ZTM202 support CSIX-L1 interfaces on both sides, giving direct connection to their own network processor and switch chipset, respectively. The AMCC nP5710/5720 has proprietary interfaces for direct connection to the company’s network processors and switching chipsets. The Sandburst QE-1000 has SPI-4.2 on the NPU side and LVDS on the switch side connecting directly to the Sandburst switching chipset.

Host Interface

The host interface on a traffic manager is probably a secondary consideration, as most control-plane processors can be bridged to the correct bus. The majority of traffic managers support PCI or a generic 32-bit processor interface. The AMCC nP3700 will support either a PowerPC interface or an Ethernet connection. The Teradiant TN8x50 devices support PCI-X. In the future we will see devices supporting PCI Express for the host interface.

Memory Interface and Buffer Sizes

The packet buffer is used to store packets waiting to be scheduled and forwarded by the traffic manager. The packets are streamed into and out of the memory in blocks from 40 bytes up to 9 kbytes. DRAM memory is ideal for this application, giving good performance when data is accessed on blocks, and is cost effective for large (up to 1 Gbyte) packet stores.

For the packet buffer all traffic managers use Double Data Rate DRAM (DDR DRAM) or a derivative. The derivatives use various techniques to reduce access times and increase transfer rates. Derivatives include Fast Cycle RAM (FCRAM), Reduced Latency RAM (RLDRAM), and Rambus’s RDRAM.

Packet buffers should, ideally, support up to 300ms of packet data. This requires roughly 25 Mbytes at 622 Mbit/s, 100 Mbytes at 2.5 Gbit/s, and 400 Mbytes at 10 Gbit/s.

The control memory is used to store descriptors, linked lists, policy tables, and other control data. These data items tend to be a few bytes each and are required by the traffic manager with minimum latency. SRAM memory is best for this application, giving very low latency access.

Most traffic managers use Double Data Rate SRAM (DDR SRAM) or derivatives. The two derivatives, Quad Data Rate SRAM (QDR SRAM) and Zero Buffer Turnaround SRAM (ZBT SRAM) are designed to increase transfer rates and reduce latency.

This page summarizes the three groups of traffic managers and reviews cost and status. The prices shown in Table 1 are publicly released prices for mid-high volume and should be used only as a very rough guide. In most cases, vendors will supply devices in volume at significantly lower prices.

622-Mbit/s Traffic Managers

There are 622-Mbit/s traffic managers available from three vendors; Agere, Azanda, and Mindspeed. All three devices support 622-Mbit/s full duplex in a single device with a full feature set including some classification and an ATM SAR for interworking. Published prices are around $250.

The APP530TM from Agere is a full-blown network processor supplied with traffic management-specific drivers. The integration of network processor functions increases the flexibility of this device but pushes the power to 7W. The MindSpeed TSP3 and Azanda Saber include much of the network processor functionality but limit the power to 3-4W.

Production devices are available from Agere and Azanda. Azanda is the only specialist traffic manager vendor with 622-Mbit/s and 2.5-Gbit/s devices.

2.5-Gbit/s Traffic Managers

Six vendors have announced 2.5-Gbit/s traffic managers. Using the older Agere RSP and Vitesse Pacemaker3.0 or the Mindspeed TSP3 will require two devices to support 2.5-Gbit/s full duplex. The other devices support 2.5-Gbit/s full duplex in a single device.

For power conscious designers, the only 2.5-Gbit/s traffic manager in production with a power consumption of 5W or below is the Azanda Saber 200, which supports POSPHY Level 3 interfaces (PL3). For those using SPI-4.2 network processors, the Azanda Scimitar comes in at 6W.

Published prices for 2.5-Gbit/s traffic managers range from $1,000 for the Azanda Scimitar down to $235 for the Freescale Q5 scheduled to sample in the first quarter of 2004. The Q5 is an FPGA and is designed to work with the Freescale C5e and C3 network processors. Freescale has no plans to produce this product as a standard device. Based on published prices, the least expensive 2.5-Gbit/s traffic manager with standard interfaces is the Agere APP550TM at $523.

Production devices are available from Agere and Azanda.

10-Gbit/s Traffic Managers

Seven vendors have announced 10-Gbit/s traffic manager chipsets. The number of devices required for a 10-Gbit/s full duplex application range from four for the IDT/Zettacom ZTM202 to one-half for the 20-Gbit/s TN8450 from Teradiant.

The EZChip QX-1 is an FPGA, designed to augment the capabilities of the EZChip NP-1 and NP-1c network processors. The AMCC nP7510/7520 chipset and Sandburst QE-1000 Queuing Engine have proprietary interfaces and are primarily designed to work as part of larger switching chipsets. The other devices are standard products with standard interfaces.

The ZettaCom traffic manager family available from IDT has the largest range. The four-device, 30W ZTM202D/C chipset is being replaced by the three-device, 15W ZTM552A/F chipset. For less demanding applications, two ZTM552A can be used alone, limiting the number of flows to 4K rather than the 1M supported with the ZTM552F.

For edge and access applications, the lowest-cost device is the Teradiant TN8250 at $645. At this price, this device is competitive for 2.5-Gbit/s applications that require SPI-4.2 interfaces. Teradiant is the only specialist traffic manager vendor with 10-Gbit/s devices.