Light Reading

AT&T Still Has IPTV 'Jitters'

Light Reading
News Analysis
Light Reading
8/11/2006
50%
50%

Industry sources say AT&T Inc. (NYSE: T) is struggling with video packet loss on the eve of the major market launch of its U-verse IPTV service.

Word has it the U-verse network loses roughly two packets of data per minute. “A lost video packet is more than 1,400 bytes of information, and that's going to cost you a half second of video,” one source says. For the viewing public that can mean little annoyances like screen pixelation and jitter -- or, at worst, full screen freezes.

In the U-verse distribution network, video packets hop from AT&T’s video super headend, to regional headends, to the local central offices, to nodes in the neighborhoods. At each "hop," packets can arrive in incorrect order or overload the buffers within the routers and switches, leading to losses.

An AT&T spokesman chose not to comment on the packet loss issue.

Microsoft Corp. (Nasdaq: MSFT), which supplies the IPTV middleware for the U-verse network, is said to be hard at work increasing its product's capacity for dealing with lost packets. Using a software algorithm called Resilient UDP, the set-top box, upon detecting missing or misplaced packets, sends a "resend" command up through the network. The missing packets are then sent down from a server. (See Microsoft Soups Up the Set-Top.)

That works fine if traffic levels are normal, the source says. But if packet loss should occur during the final minutes of the Super Bowl, a million resend requests could pummel the network at the same time, seriously burdening the system.

One source close to the situation says Microsoft has already built in a 15 to 30 second delay to live video streams to allow some time for dealing with packet loss. AT&T, the source says, is uneasy about the scaleability of the setup.

Microsoft TV Edition product manager Jim Baldwin says his company's middleware platform adds roughly a quarter of a second delay for packet error correction and another second of delay for instant channel changing, but that's it. Baldwin says AT&T is perfectly happy with his company’s Resilient UDP approach to packet loss, but says AT&T may decide to use forward error correction as well.

The Scientific-Atlanta Inc. encoders at the headend of the system, Baldwin says, will add a certain amount of latency to the video streams, but AT&T will decide how much. Scientific-Atlanta chose not to comment for this story. (See AT&T, Verizon Tout Telco TV .)

Meanwhile, AT&T may be hedging its bets. Sources close to the situation say AT&T engineers are experimenting with forward error correction (FEC) on one leg of AT&T's video network. Applied near the headend encoders, FEC adds additional video bitstreams that can be used to reconstruct damaged streams on the fly, downstream in the network. Also at the headend, the video packets are tagged sequentially so that the system can detect interruptions in the packet order downstream. The FEC technology monitors the bitstreams at several points in the network, including the set-top box, to detect missing or damaged packets.

Baldwin says FEC works well when the type and size of the packet loss is predictable. But IP networks can lose a whole packet, groups of consecutive packets, or just one bit within a packet, he says.

U-verse was already under criticism, as analysts have noted the initial Project Lightspeed rollout of broadband access won't have the punch to carry high-definition TV, and some question whether Lightspeed will roll out as quickly as AT&T hopes. (See Is Lightspeed Slowing?)

Flickers of trouble?
U-verse debuted in late June in AT&T’s home turf of San Antonio, Texas, and the AT&T spokesman says the service is getting good reviews from users. The few actual U-verse users contacted by Light Reading say the packet loss issue, at least so far, hasn’t been very visible on their TVs. (See AT&T to Launch Lightspeed Next Month.)

“The flicker [pixelation] has only appeared a few times,” says U-verse user Alan Weinkrantz. “But keep in mind that I also have HD Cable from TimeWarner, and there are times when I also get flicker or a nano-of-a-second blur on that.” Weinkrantz, whose day job is as a technology public relations man, has become a minor celebrity through his U-verse user's blog.

“I had some pixelation the very first day after I got service,” writes another U-verse customer, Chad Brantly. “The next day, I got a call from AT&T. They said that they had been having some problems in my area, but they had just done a hardware update and things should be better. Since then, I haven't seen any pixelation. The service has been great,” Brantly writes.

A few unhappy U-verse adopters, however, have shown up on a bulletin board called Uverseusers.com. One of the five discussion threads on the site is called “pixelations and freeze-ups,” wherein three users -– “eapinsatx,” “dilbert” and “nohbdy” -– complain of moderate to serious pixelation and screen freezes.

Heavy Reading analyst Rick Thompson points out the San Antonio debut is probably happening in a very "controlled" network environment. With the technology world watching, AT&T is surely taking steps to make sure the fledgling service makes a good first impression.

But the issue of scaleability looms for AT&T’s engineers, as the carrier plans to roll out U-verse in 15 to 20 markets by the end of 2006. (See AT&T Readies Lightspeed in North Texas.) "It's one thing to get a complex technology like IPTV rolled out to a group of a few thousand users," says independent telecom analyst Kermit Ross. "But it’s quite another thing to kick that up a notch to a few hundred thousand and then yet another thing to kick it up to a few million users."

In other words, once AT&T cranks up the numbers, any failure to tame packet loss will be evident on millions of TV screens.

— Mark Sullivan, Reporter, Light Reading

(30)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View
Page 1 / 3   >   >>
trzwuip
50%
50%
trzwuip,
User Rank: Light Beer
12/5/2012 | 3:44:58 AM
re: AT&T Still Has IPTV 'Jitters'
does this mean the d-servers dont work the way microsoft promised?
metroman
50%
50%
metroman,
User Rank: Light Beer
12/5/2012 | 3:44:57 AM
re: AT&T Still Has IPTV 'Jitters'
There is another possibility.

If the traffic is being delivered using multicast, the network elements will need to replicate the traffic. Depending upon the hardware in use, it may be that they are having issues either with the replication itself or the forwarding mechanisms for multicast. Ingress replication for multicast may create congestion in switch fabrics, while if the stream is being replicated to many egress interfaces, all interfaces need to be available before the frame is forwarded. In this case it only take one congested interface to cause network-wide packet loss and service degradation.

The network elements need to have robust hardware to deal with multicast replication and well thought out prioritisation & buffering mechanisms.

Does anyone know if they have the same issues on VoD (unicast)?

Metroman
paolo.franzoi
50%
50%
paolo.franzoi,
User Rank: Light Beer
12/5/2012 | 3:44:57 AM
re: AT&T Still Has IPTV 'Jitters'

My guess is that the small buffers on the STB to have the short channel switch time is the issue. Pretty much any network imperfections flow through to the STB. Scaling this will be pretty funny, especially that there is no HDTV.

seven
stolsma
50%
50%
stolsma,
User Rank: Light Beer
12/5/2012 | 3:44:57 AM
re: AT&T Still Has IPTV 'Jitters'
I realy don't understand that there are that much packets lost per minute. Packet loss is more or less dependable on congestion and buffer depths, and if the article is true then there is a lot of congestion on the path from Master head end to the Customer. In my opinion a case of bad network planning/packet prioritization (I expect that this video traffic has a higher priority than Internet traffic..) that can be solved fairly easy. Or is there something I don't see ???

stolsma
alcatell1
50%
50%
alcatell1,
User Rank: Light Beer
12/5/2012 | 3:44:56 AM
re: AT&T Still Has IPTV 'Jitters'
or possibly fatal flaws in 7450 multicast throughput
paolo.franzoi
50%
50%
paolo.franzoi,
User Rank: Light Beer
12/5/2012 | 3:44:55 AM
re: AT&T Still Has IPTV 'Jitters'

Couple of things. You are all making an assumption that any packet loss is due to forwarding issues, prioritization, or congestion.

1 - 1 Packet (potentially 1 Bit Error) in 2 minutes of data transmission is a BER of around 10-9 for an SDTV stream (1 error in approximately 240 Mb). Ever checked the BER of a DSL line?

2 - The buffers are very small on MS STBs to achieve the fast channel change. In normal IPTV setups, there are seconds of buffering and channel switches look like Satellite systems. Network jitter must be minimized to keep these STBs fed. This limits abilities of the multicast network to buffer in replication - as such buffering will add jitter as more paths are added to that channel.

seven
gianconstantine
50%
50%
gianconstantine,
User Rank: Light Beer
12/5/2012 | 3:44:55 AM
re: AT&T Still Has IPTV 'Jitters'
I think you may not be considering the nature of multicast vs. unicast in delivering video. The bandwidth requirements on the backplane are not heavy enough to cause problems. Consider 250 channels encoded in VC1. While high, let's consider 3Mbps stream rate. Even if all 250 channels are being subscribed to off all cards in 10 blade chassis (for argument's sake), the switching fabric would be pushing less than 8Gbps.

I ran an IPTV network in Atlanta for all of 2005. I slammed Cisco 6500 Sup720/MSFC3 with multicast and unicast traffic and the switch fabric stats were as clean as a whistle.

Microsoft and Alcatel have some sort of funky setup that's screwing them. This problem is not so simple as an incapable switch or router.

Gian.
cabecar
50%
50%
cabecar,
User Rank: Light Beer
12/5/2012 | 3:44:55 AM
re: AT&T Still Has IPTV 'Jitters'
VoD should not have the same issues as it is not Multicast traffic. The problem with VoD is that each new instance requires a new stream. This means that they need to have Video pump engines close to the edge... if they don't a spike in VoD usage can affect users over the entire network. If they do have pump engines near the edge then a spike will only affect edge users in a particular region. To prevent this they need to implement video CAC for Unicast VoD.
stolsma
50%
50%
stolsma,
User Rank: Light Beer
12/5/2012 | 3:44:54 AM
re: AT&T Still Has IPTV 'Jitters'
Thanks for all the reply's. It's all a little bit clearer to me now..

I know something about the 7450 and I'm almost sure there is nothing wrong with the multicast replication (as sure you can be after 9 months of testing..). And to be honest we didn't see any alarming packet loss (except some bit errors on bad optical lines that resulted in bad packets, resolved by replacing optical cards, can happen any time).

What we did see was packet loss on DSL lines because of impuls noise and bad copper lines (comparable to seven's 10-9), btw some impuls ratio settings and DSL CRC checking can help there. Trade off is the added latency for other traffic like gaming or internet pings. When I wrote my first message I was still under the impression that Lightspeed was FttH but I saw that also FttC (with DSL as last/first mile) was used.

Personally I think that solving this problem with unicast retransmit requests is not the way to go but who am I...

stolsma
stolsma
50%
50%
stolsma,
User Rank: Light Beer
12/5/2012 | 3:44:54 AM
re: AT&T Still Has IPTV 'Jitters'
cabecar wrote: ".... To prevent this they need to implement video CAC for Unicast VoD."

Anyone out there who can clarify if AT&T is doing something like this ??? I know some operators who are trying to implement a function this. Some with great results!!

stolsma
Page 1 / 3   >   >>
Educational Resources
sponsor supplied content
Educational Resources Archive
Flash Poll
From The Founder
Networks of the future will rely on "white box" switches and servers rather than proprietary hardware and that's going to alter the shape of the communications industry. Who says so? John Chambers.
LRTV Custom TV
The Benefits of HyperScale Clouds for NFV

3|27|15   |   01:50   |   (0) comments


Hyperscale cloud has been developed by the Internet giants to support the creation and delivery of software-based services at blistering speeds, and at the lowest possible cost. The original ETSI NFV vision was to adopt hyperscale cloud architecture and practices. This vision has become somewhat obscured along the way, due to misunderstandings about the hyperscale ...
LRTV Huawei Video Resource Center
eLTE Rapid Meets the Need for Speed

3|26|15   |   4:45   |   (0) comments


Designed especially for emergency and dedicated ad hoc local mobile communications coverage, Huawei's eLTE Rapid solution can deliver trunked voice, video and data coverage for multiple users over a 6km range and be set up in just 15 minutes, explains Huawei's Norman Frisch.
LRTV Huawei Video Resource Center
On Videos: Challenges & Opportunities

3|26|15   |   5:56   |   (0) comments


Most everything is now connected. And along with 4K and 4G technologies, everyone could be creating and broadcasting video contents. Users are expecting better video experience with any screen, anywhere and anytime. Operators will meet new challenges, but also see some big opportunities.
LRTV Custom TV
JDSU: Delivering Dynamic Networks for a Personalized Experience

3|26|15   |   5:59   |   (0) comments


Light Reading speaks to JDSU at Mobile World Congress 2015 about new solutions in the areas of HetNets, VoLTE, backhaul, virtualization, big data analytics, and real-time intelligence.
LRTV Custom TV
Smarter Service Chaining & New Ways to Benefit From Qosmos Technology

3|25|15   |   03:11   |   (0) comments


David Le Goff, director of strategic and product marketing at Qosmos, explains how the company has added application awareness to subscriber information to make service chaining more efficient and reduce costs for networking and infrastructure. In addition, Qosmos technology, which has been delivered as C libraries, is now also available as a virtual machine, ...
Between the CEOs
Qosmos CEO: The Changing Face of DPI

3|24|15   |   13:53   |   (0) comments


LR CEO and Founder Steve Saunders sits down with the head of Qosmos to talk about the changing state of the art in deep packet inspection technology, including its role in SDN and NFV architectures. Also, how the comms market is becoming more like the automotive industry.
LRTV Huawei Video Resource Center
FC Schalke Scores With Its Agile Stadium

3|24|15   |   6:23   |   (0) comments


Top German soccer club FC Schalke 04 has deployed a new, agile WiFi network from Huawei in its Veltins-Arena stadium and is reaping the benefits in terms of customer satisfaction and business opportunities, explains marketing chief Alexander Jobst.
LRTV Huawei Video Resource Center
Huawei’s Insights on Mobile Video

3|24|15   |   7:51   |   (0) comments


More people than ever are now watching videos on smartphones. Seventy percent of mobile traffic will be video traffic until 2018. In this video, Huawei's exports give their insights on mobile video in terms of business model, network planning and 4G network construction.
LRTV Documentaries
The Rise of Industry 4.0

3|24|15   |   02:26   |   (9) comments


Are you ready for the fourth industrial revolution? It's a big deal for influential operators such as Deutsche Telekom.
LRTV Huawei Video Resource Center
Getting Connected With eLTE

3|23|15   |   06:04   |   (0) comments


Trunked radio communications have entered the 4G LTE world, and with Huawei's eLTE solution, can now deliver a full range of data and video services as well as push-to-talk voice, explains Huawei's Norman Frisch.
LRTV Huawei Video Resource Center
Funkwerk’s on Track With Huawei

3|19|15   |   3:23   |   (0) comments


GSM-R technology specialist Funkwerk and Huawei have forged a partnership that is benefiting both parties, notes Funkwerk's Gottfried Winter.
LRTV Documentaries
How EANTC Tested Cisco's Virtualization Solutions

3|18|15   |   5:49   |   (0) comments


Carsten Rossenhövel, managing director of independent test lab EANTC, tells Light Reading founder and CEO Steve Saunders about the innovative approach his team had to take when validating Cisco's service provider virtualization and cloud solutions.
Upcoming Live Events
April 14, 2015, The Westin Times Square, New York City, NY
May 5, 2015, Hyatt McCormick Place, Chicago, IL
May 6, 2015, Georgia World Congress, Atlanta, GA
May 12, 2015, Grand Hyatt, Denver, CO
May 13-14, 2015, The Westin Peachtree, Atlanta, GA
June 8, 2015, Chicago, IL
June 9-10, 2015, Chicago, IL
June 9, 2015, Chicago, IL
June 10, 2015, Chicago, IL
September 29-30, 2015, The Westin Grand Müchen, Munich, Germany
All Upcoming Live Events
Infographics
Hot Topics
AT&T Woos SMBs With Small-Scale WiFi
Sarah Thomas, Editorial Operations Director, 3/26/2015
The Rise of Industry 4.0
Ray Le Maistre, Editor-in-chief, 3/24/2015
Google Hires Wall Street's Most Influential Woman as CFO
Dan Jones, Mobile Editor, 3/24/2015
Just Don't Say IBM Is 'Relaunching' Networking Business
Mitch Wagner, West Coast Bureau Chief, Light Reading, 3/26/2015
Average US Broadband Speeds No Great Shakes
Mari Silbey, Independent Technology Editor, 3/25/2015
Like Us on Facebook
Twitter Feed
Webinar Archive
BETWEEN THE CEOs - Executive Interviews
LR CEO and Founder Steve Saunders sits down with the head of Qosmos to talk about the changing state of the art in deep packet inspection technology, including its role in SDN and NFV architectures.
Chattanooga’s EPB publicly owned utility comms company has become a poster child for how to enable a local economy using next-gen networking technology. Steve Saunders, Founder of Light Reading, sits down with Harold DePriest, president and CEO of EPB, to learn how EPB is bringing big time tech to small town America.
Cats with Phones
Interspecies Phone Love Click Here
"No, you hang up."
"No, YOU hang up."
Latest Comment