Video services

AT&T Still Has IPTV 'Jitters'

Industry sources say AT&T Inc. (NYSE: T) is struggling with video packet loss on the eve of the major market launch of its U-verse IPTV service.

Word has it the U-verse network loses roughly two packets of data per minute. “A lost video packet is more than 1,400 bytes of information, and that's going to cost you a half second of video,” one source says. For the viewing public that can mean little annoyances like screen pixelation and jitter -- or, at worst, full screen freezes.

In the U-verse distribution network, video packets hop from AT&T’s video super headend, to regional headends, to the local central offices, to nodes in the neighborhoods. At each "hop," packets can arrive in incorrect order or overload the buffers within the routers and switches, leading to losses.

An AT&T spokesman chose not to comment on the packet loss issue.

Microsoft Corp. (Nasdaq: MSFT), which supplies the IPTV middleware for the U-verse network, is said to be hard at work increasing its product's capacity for dealing with lost packets. Using a software algorithm called Resilient UDP, the set-top box, upon detecting missing or misplaced packets, sends a "resend" command up through the network. The missing packets are then sent down from a server. (See Microsoft Soups Up the Set-Top.)

That works fine if traffic levels are normal, the source says. But if packet loss should occur during the final minutes of the Super Bowl, a million resend requests could pummel the network at the same time, seriously burdening the system.

One source close to the situation says Microsoft has already built in a 15 to 30 second delay to live video streams to allow some time for dealing with packet loss. AT&T, the source says, is uneasy about the scaleability of the setup.

Microsoft TV Edition product manager Jim Baldwin says his company's middleware platform adds roughly a quarter of a second delay for packet error correction and another second of delay for instant channel changing, but that's it. Baldwin says AT&T is perfectly happy with his company’s Resilient UDP approach to packet loss, but says AT&T may decide to use forward error correction as well.

The Scientific-Atlanta Inc. encoders at the headend of the system, Baldwin says, will add a certain amount of latency to the video streams, but AT&T will decide how much. Scientific-Atlanta chose not to comment for this story. (See AT&T, Verizon Tout Telco TV .)

Meanwhile, AT&T may be hedging its bets. Sources close to the situation say AT&T engineers are experimenting with forward error correction (FEC) on one leg of AT&T's video network. Applied near the headend encoders, FEC adds additional video bitstreams that can be used to reconstruct damaged streams on the fly, downstream in the network. Also at the headend, the video packets are tagged sequentially so that the system can detect interruptions in the packet order downstream. The FEC technology monitors the bitstreams at several points in the network, including the set-top box, to detect missing or damaged packets.

Baldwin says FEC works well when the type and size of the packet loss is predictable. But IP networks can lose a whole packet, groups of consecutive packets, or just one bit within a packet, he says.

U-verse was already under criticism, as analysts have noted the initial Project Lightspeed rollout of broadband access won't have the punch to carry high-definition TV, and some question whether Lightspeed will roll out as quickly as AT&T hopes. (See Is Lightspeed Slowing?)

Flickers of trouble?
U-verse debuted in late June in AT&T’s home turf of San Antonio, Texas, and the AT&T spokesman says the service is getting good reviews from users. The few actual U-verse users contacted by Light Reading say the packet loss issue, at least so far, hasn’t been very visible on their TVs. (See AT&T to Launch Lightspeed Next Month.)

“The flicker [pixelation] has only appeared a few times,” says U-verse user Alan Weinkrantz. “But keep in mind that I also have HD Cable from TimeWarner, and there are times when I also get flicker or a nano-of-a-second blur on that.” Weinkrantz, whose day job is as a technology public relations man, has become a minor celebrity through his U-verse user's blog.

“I had some pixelation the very first day after I got service,” writes another U-verse customer, Chad Brantly. “The next day, I got a call from AT&T. They said that they had been having some problems in my area, but they had just done a hardware update and things should be better. Since then, I haven't seen any pixelation. The service has been great,” Brantly writes.

A few unhappy U-verse adopters, however, have shown up on a bulletin board called Uverseusers.com. One of the five discussion threads on the site is called “pixelations and freeze-ups,” wherein three users -– “eapinsatx,” “dilbert” and “nohbdy” -– complain of moderate to serious pixelation and screen freezes.

Heavy Reading analyst Rick Thompson points out the San Antonio debut is probably happening in a very "controlled" network environment. With the technology world watching, AT&T is surely taking steps to make sure the fledgling service makes a good first impression.

But the issue of scaleability looms for AT&T’s engineers, as the carrier plans to roll out U-verse in 15 to 20 markets by the end of 2006. (See AT&T Readies Lightspeed in North Texas.) "It's one thing to get a complex technology like IPTV rolled out to a group of a few thousand users," says independent telecom analyst Kermit Ross. "But it’s quite another thing to kick that up a notch to a few hundred thousand and then yet another thing to kick it up to a few million users."

In other words, once AT&T cranks up the numbers, any failure to tame packet loss will be evident on millions of TV screens.

— Mark Sullivan, Reporter, Light Reading

<<   <   Page 2 / 3   >   >>
metroman 12/5/2012 | 3:44:53 AM
re: AT&T Still Has IPTV 'Jitters' I have heared a lot of rumour about the 7450 and it's multicast performance. Nothing concrete just the same comments coming up about it "having issues". Can anyone shed any light?

And in reply to the other comment about multicast on switch fabrics following my posting:

3 key points -

1. Multicast is not the only traffic I am sure.

2. It's not about fabric capacity, it's about output queue capacity. When doing multicast replication all output queues have to be available before any of them can transmit. If you have one congested output queue you have an issue and possible packet loss.

3. Switches do not have infinate replication capacity, they all have a limit. Also the replication may not be done very well, or at line rate. These architectural issues can combine to have a detrimental affect.

It may not be the case here, just for clarity.


cbrantly 12/5/2012 | 3:44:53 AM
re: AT&T Still Has IPTV 'Jitters' I am a U-verse customer and have talked with logs of users on the UverseUsers.com message board. Most users have found that their packet loss / jitters are a result of using the existing in-home coax. I had the installer run Cat-5 to my gateway and both of my TVs and I have had 0 packet loss.
cc_junk 12/5/2012 | 3:44:52 AM
re: AT&T Still Has IPTV 'Jitters' Is this a Lightreading invented problem? Sheesh, they refer to 3 users complaining about service. Perhaps enough for Lightreading to create a "panic" article, but nothing I would take note of.
cbrantly 12/5/2012 | 3:44:52 AM
re: AT&T Still Has IPTV 'Jitters' The author of this story mentions the problems experienced by one user at UverseUsers.com named dilbert. It is worth mentioning that all of dilberts posts have been made from IP addresses owned by TimeWarner's RoadRunner cable internet service. It is very questionable whether or not dilbert actually has U-verse or if he is just a troll.
stolsma 12/5/2012 | 3:44:52 AM
re: AT&T Still Has IPTV 'Jitters' Indeed one of the biggest problem areas: In home wiring. There is also one other area at home that is important: the wire used from the copper hand-off interfacepoint to the DSL modem. Some customers use simply a cheap non twisted copperpair. And then when a big truck/bus/thunderstorm comes around or a local radio transmitter freak starts transmitting, the TV service doesn't work any more... Very strange.. ;-)

btw there was a firmware update (one of lots of others solving other problems, but what do you expect of equipment/software just 2 years old) for better multicast performance in the 7xxx but you had to have a lot of multicast traffic to experience the problems. In real life we didn't see any problems with multicast.

Mark Sullivan 12/5/2012 | 3:44:51 AM
re: AT&T Still Has IPTV 'Jitters' Aw, you're no fun.

But seriously, read the first half of the article. The UverseUsers comments were thrown in just for laughs.
dilberts 12/5/2012 | 3:44:50 AM
re: AT&T Still Has IPTV 'Jitters' The pictures tear really bad and are very annoying to watch. Other times the screen just totally locks up and it takes 15 minutes to reset. That is a real problem in the middle of a game. Nowhere near the viewing experience of satellite or cable. I don't know how they expect to do HDTV if standard definition is unwatchable. Most of the customers are at&t employees so you really can not believe what they say about how good things are going.

The other big problem is the DVR stops recording if you try and change channels? And no surround sound. What the att reps say is that it will all be fixed by the end of the year, but no one can tell you how?

Another interesting point is the DSL service is not any faster than it was before U-verse. Cable is still quite a bit faster and more reliable.
Frank 12/5/2012 | 3:44:49 AM
re: AT&T Still Has IPTV 'Jitters' I'm willing to bet that there are many MSOs whose networks are suffering similar ills as they continue to pump up their HDTV commitments and provisions, and increase their Internet access speeds and their digital content, possibly more so during peak times than at other times.

Of course, AT&T's measly last mile design, and its insistance to stay with copper, has forced certain constraints that wind up being additive across other network elements in the IPTV platform, but the problem is not due to any of those parameters alone.

From my perspective, one thing is for sure -- and I speak from my own experiences, only -- and that is, the problem is definitely increasing, even on HFC systems, instead of getting better. And this makes eminent sense, when you think abou it, considering the trending taking place marked by continually adding more load to most systems.

In Brooklyn, NY I frequently encounter pixelation on my cable service, which is often followed by a momentary dropout and re-synching on identifiable "digital channel groupings". I.e., the six or ten NTSC 6MHz channels that have been compressed into a single digital channel occupying 6MHz of bandwidth in the digital spectrum of the TW system serving me.

The root causes of the problems that AT&T is encountering - which may be a combination of Microsoft's IPTV setup and attempting to provide triple play services on a bandwidth-restricted platform, among other causes - differs (or only partially overlaps) with the problems that some MSOs are, or very soon will be, facing.

And this points to a pet peeve of mine. I.e., there is no means in place that allows one to obtain a snapshot of how well his or her service is performing, much less assuring the accountability for said performance, that cable tv providers deliver, and this applies to both the MSOs and the Telcos. Just as there is no means for most power utilities to determine who does and who doesn't have power during sporadic power outages.

Ferchrissakes, it's 2006, and the power company still can't tell which of its subscribers is receiving its service, which results in the utility having to walk door to door to find out who does. I want to have access to a dashboard on the Web that I can go to and observe just how much congestion exists on my last mile segment, both for purposes of assessing my Internet access and my tv reception; the number of errored seconds and minutes, selectable per time frame of my choosing, that exists on my cable service for any given channel or group of channels; the status of VoD operation, DVR and other user selectable features; the status of my last reported trouble to the help desk; and more.

This is not only feasible to do, but is being done behind the curtains for the utilities' own consumption! In enterprise communications circles such a window into the carriers' monitoring capabilities for the purpose of assessing SLA performance is known as CNM, or customer network management, which is usually a read-only type of accommodation to let the customer know what is going on.

Getting back to my local cable service, some family members who live about two miles from me, also in Brooklyn, receive Cablevision Communications digital tv service, and they tell me that similar glitches and artifacts show up on their digital channels, as well.

The most entertaining experience I'm having these days occurs when I find it necessary to reboot my STB (take your pick... the one in the living room, the den, or the upstairs study ... but it never happens to all three at once) that seem to lose some of their program elements for VoD features but not for regularly scheduled programming.

Worse, the VoD feature sometimes works for one selection (HBO's The Wire, for example), but not for another (an HBO Boxing event) unless, of course, I remove power to the STB and endure the reboot cycle every time.

Yes, of course I've reported this to TW. But let me ask you this: Do you think that the problems I've described above manifest themselves when the cable guy shows up? Not on your life. And so it goes.
Honestly 12/5/2012 | 3:44:49 AM
re: AT&T Still Has IPTV 'Jitters' Delay in traditional broadcast networks would rock you back a bit. Also, as a Direct TV customer, I get pixelation disrupts randomly.

I hear from users that the ATT service, especially the features are really cool. Instant channel change and a smart guide would make me much happier with my Direct DVR. ATT has this, I do not.

heritagejd 12/5/2012 | 3:44:38 AM
re: AT&T Still Has IPTV 'Jitters' Can't solve everything with software!
<<   <   Page 2 / 3   >   >>
Sign In