re: Juniper Goes Terabit With the T640 I think they passively identify micro-flows on a per-port basis and then assign the flows to "trunks" that move traffic between the caspian systems. Because they have perfect knowledge of all the flows, they can optimize the traffic patterns within a cluster of their systems and potentially do finer grain QOS. ------------------
The idea sounds very interesting, but I think implementation will be very costly and maybe not feasible.
does micro-flow means per connection session? for typical long, very- bursty TCP sessions, any %50 loaded oc-192 can have live sessions of tens of millions minimum at any given time, are all flow states are kept in switches' memories? for every packets you need look up and modify the flow states, it is very costly to do it for high speed(>10 mil packets/s). there is also lots of overhead in flow matainance and management, probably requires entirely new software.
re: Juniper Goes Terabit With the T640Juniper states on page 17 of their whitepaper that the T640 is non-blocking "As long as each ingress PFE does not forward an excessive number of high-priority cells into the fabric and exceed its fair bandwidth allocation, high-priority traffic is guaranteed a path through the switch fabric."
So, how much is excessive?
Let's assume that high-priority equals multi-service.
It doesn't seem like the T640 was truly designed for multi-service networking even though Juniper is clearly going after the multi-service packet core (page 3 of their solution brief). Seems to me they will have to throw bandwith at the problem in order to guarantee that voice, video, and private line connections are not blocked through their "non-blocking" fabric.
I agree with your assertion. This is more like the "same-old" VOQ architecture that Cisco uses in their GSR platforms. Ironic that after having bashed Cisco's architecture, Juniper's much vaunted next generation architecture for the multi-service core is just a re-hash of Cisco's "now-old and aging" platform. In fact, I would argue that Cisco's is much simpler.
Note that the white paper does not mention VOQ anywhere. Does it mean that they do not have multiple ingress queues per egress PFE per service class? They better have it - if not, they will be subject to serious Head-Of-Line blocking!!
re: Juniper Goes Terabit With the T640The idea sounds very interesting, but I think implementation will be very costly and maybe not feasible.
does micro-flow means per connection session? for typical long, very- bursty TCP sessions, any %50 loaded oc-192 can have live sessions of tens of millions minimum at any given time, are all flow states are kept in switches' memories? ----------- I think the states are kept in the ingress port memory as best I could figure out from their material. It still seems like an almost prohibitive amount of state though.
----------- for every packets you need look up and modify the flow states, it is very costly to do it for high speed(>10 mil packets/s). ----------- I don't think it would be necessary to modify the flow state for each packet, but you have to refresh the "liveness" of the flow on a per-packet basis (and also time out liveness per-flow). There would obviously be a high cost for the initial packet in a flow, but after the first one, you might be able to make it much less expensive.
----------- there is also lots of overhead in flow matainance and management, probably requires entirely new software. ----------- I think it would require something like RSVP running purely inside of the box (something to set up end-to-end reservations), something like OSPF to flood topology information. And probably lots more. To do something like this brings all the complexity of a large network inside the box.
Its also quite likely to require specialized hardware to identify and track the flows.
re: Juniper Goes Terabit With the T640Hi Palycat, I disagree with your statement that real life traffic do not generate 100 Mbps of traffic... Case in point, MPEG2 over IP. Broadcast TV program transported over IP. I think it can go as high as 300 Mbps..
re: Juniper Goes Terabit With the T640 The packet re-ordering is due to their packet processing ASIC which has a throughput of 2.5 Gbps. When they decided to support OC-192 (10Gbps) interfaces on M160 they wanted to use the same 2.5 Gig ASIC for time to market reasons and used an FPGA(?) to load balance the incoming packets across 4 such processors. I think this is the root cause for packets to go out of order. Obviously the FPGA cannot identify packets belonging to the same flow at OC-192 rate, otherwise we wouldnt need the ASIC in the first place. M40 doesnt have OC-192 interfaces, so it doesnt have the reordering problem.
Is it the same problem JNPR is trying to solve by using sequence numbers? Does that mean they are still working on a higher speed packet processing ASIC?
As far as TCP/IP goes, there is no guarantee a core router would NOT see back-to-back packets from the same flow. Although, the probability is very low. On the other hand, packet misordering could be a serious problem for real-time applications. Now that they are talking about everything-over-IP.
re: Juniper Goes Terabit With the T640Obviously the FPGA cannot identify packets belonging to the same flow at OC-192 rate, otherwise we wouldnt need the ASIC in the first place. ---------- No, an FPGA could recognize flows. But the problem is that it can't load balance multiple flows across the available paths. Its easy to have a solution that distributes flows. Its distributing the flows in a load-balanced manner with decent utilization in each component path thats difficult. ------------ Is it the same problem JNPR is trying to solve by using sequence numbers? Does that mean they are still working on a higher speed packet processing ASIC? ------------
I think the new system has a new faster ASIC for packet processing. I don't know if it has sequencing issues. I believe though that in the new system juniper's switching technology makes heavy use of parallel paths and thus has a set of reordering issues to deal with.
re: Juniper Goes Terabit With the T640I disagree with your statement that real life traffic do not generate 100 Mbps of traffic... Case in point, MPEG2 over IP. Broadcast TV program transported over IP. I think it can go as high as 300 Mbps.. ------------------------- 300Mbps for a single video stream? please provide more detail.
DVD quality mpeg2 ~ 6mbps average, ~10 mbps peak burst with a small playback buffer. with 100% packet/link overhead, the peak rate is around 20mbps.
If you talking about multiple video streams, you don't have to maintain packet orders between different streams.
most new stream video servers limits the burst rate of indiviual steams before sending to network.
re: Juniper Goes Terabit With the T640The packet re-ordering is due to their packet processing ASIC which has a throughput of 2.5 Gbps. When they decided to support OC-192 (10Gbps) interfaces on M160 they wanted to use the same 2.5 Gig ASIC for time to market reasons and used an FPGA(?) to load balance the incoming packets across 4 such processors.
----------------- small clarification: Juniper's M40/M160 is kind of unique, it use a centuralized packet processing architecture together with a shared memory switches, their ASICs can handle much more than 10Gbps, but problem is they pool packets from all ports together and processing it with a single sets of ASIC. while most competing products processing the flow on each port/linecard.
re: Juniper Goes Terabit With the T640Take one input, split it over 4 internal paths (because you have to), have a splash in the(shared memmory)hot tub, if this packet didn't arrive out of order (unlikely, unless there was an upstream 160) he will be, as he follows one of 4 paths to the egress.
TTM: Time to Test Mother (in the best Principle Skinner taste). Sometimes if it sounds too good to be true, it is.
Juniper had a choice on the ordering problem. This is not the type of bug that is overlooked. They have some of the best router engineers in the world. I would suggest it was the market-gineers that made the decision.
With respect to the T640 half duplex, I will bet my hat that they don't have reordering issues on OC192 and as the architecture provides two paths per potential OC768 I sincerely doubt there's an issue there.
Null0
Respinning m160 ASIC's this late in the game would be suicidal even if they almost admit the problem "The potential for misordering cells as they are transmitted across parallel switch planes to the egress PFEis eliminated" T640 - System and Packet Forwarding Architecture.pdf
I think they passively identify micro-flows on
a per-port basis and then assign the flows to
"trunks" that move traffic between the caspian
systems. Because they have perfect knowledge
of all the flows, they can optimize the traffic
patterns within a cluster of their systems and
potentially do finer grain QOS.
------------------
The idea sounds very interesting, but I think implementation will be very costly and maybe not feasible.
does micro-flow means per connection session? for typical long, very- bursty TCP sessions, any %50 loaded oc-192 can have live sessions of tens of millions minimum at any given time, are all flow states are kept in switches' memories? for every packets you need look up and modify the flow states, it is very costly to do it for high speed(>10 mil packets/s). there is also lots of overhead in flow matainance and management, probably requires entirely new software.