AT&T Still Has IPTV 'Jitters'

Industry sources say AT&T Inc. (NYSE: T) is struggling with video packet loss on the eve of the major market launch of its U-verse IPTV service.

Word has it the U-verse network loses roughly two packets of data per minute. “A lost video packet is more than 1,400 bytes of information, and that's going to cost you a half second of video,” one source says. For the viewing public that can mean little annoyances like screen pixelation and jitter -- or, at worst, full screen freezes.

In the U-verse distribution network, video packets hop from AT&T’s video super headend, to regional headends, to the local central offices, to nodes in the neighborhoods. At each "hop," packets can arrive in incorrect order or overload the buffers within the routers and switches, leading to losses.

An AT&T spokesman chose not to comment on the packet loss issue.

Microsoft Corp. (Nasdaq: MSFT), which supplies the IPTV middleware for the U-verse network, is said to be hard at work increasing its product's capacity for dealing with lost packets. Using a software algorithm called Resilient UDP, the set-top box, upon detecting missing or misplaced packets, sends a "resend" command up through the network. The missing packets are then sent down from a server. (See Microsoft Soups Up the Set-Top.)

That works fine if traffic levels are normal, the source says. But if packet loss should occur during the final minutes of the Super Bowl, a million resend requests could pummel the network at the same time, seriously burdening the system.

One source close to the situation says Microsoft has already built in a 15 to 30 second delay to live video streams to allow some time for dealing with packet loss. AT&T, the source says, is uneasy about the scaleability of the setup.

Microsoft TV Edition product manager Jim Baldwin says his company's middleware platform adds roughly a quarter of a second delay for packet error correction and another second of delay for instant channel changing, but that's it. Baldwin says AT&T is perfectly happy with his company’s Resilient UDP approach to packet loss, but says AT&T may decide to use forward error correction as well.

The Scientific-Atlanta Inc. encoders at the headend of the system, Baldwin says, will add a certain amount of latency to the video streams, but AT&T will decide how much. Scientific-Atlanta chose not to comment for this story. (See AT&T, Verizon Tout Telco TV .)

Meanwhile, AT&T may be hedging its bets. Sources close to the situation say AT&T engineers are experimenting with forward error correction (FEC) on one leg of AT&T's video network. Applied near the headend encoders, FEC adds additional video bitstreams that can be used to reconstruct damaged streams on the fly, downstream in the network. Also at the headend, the video packets are tagged sequentially so that the system can detect interruptions in the packet order downstream. The FEC technology monitors the bitstreams at several points in the network, including the set-top box, to detect missing or damaged packets.

Baldwin says FEC works well when the type and size of the packet loss is predictable. But IP networks can lose a whole packet, groups of consecutive packets, or just one bit within a packet, he says.

U-verse was already under criticism, as analysts have noted the initial Project Lightspeed rollout of broadband access won't have the punch to carry high-definition TV, and some question whether Lightspeed will roll out as quickly as AT&T hopes. (See Is Lightspeed Slowing?)

Flickers of trouble?
U-verse debuted in late June in AT&T’s home turf of San Antonio, Texas, and the AT&T spokesman says the service is getting good reviews from users. The few actual U-verse users contacted by Light Reading say the packet loss issue, at least so far, hasn’t been very visible on their TVs. (See AT&T to Launch Lightspeed Next Month.)

“The flicker [pixelation] has only appeared a few times,” says U-verse user Alan Weinkrantz. “But keep in mind that I also have HD Cable from TimeWarner, and there are times when I also get flicker or a nano-of-a-second blur on that.” Weinkrantz, whose day job is as a technology public relations man, has become a minor celebrity through his U-verse user's blog.

“I had some pixelation the very first day after I got service,” writes another U-verse customer, Chad Brantly. “The next day, I got a call from AT&T. They said that they had been having some problems in my area, but they had just done a hardware update and things should be better. Since then, I haven't seen any pixelation. The service has been great,” Brantly writes.

A few unhappy U-verse adopters, however, have shown up on a bulletin board called Uverseusers.com. One of the five discussion threads on the site is called “pixelations and freeze-ups,” wherein three users -– “eapinsatx,” “dilbert” and “nohbdy” -– complain of moderate to serious pixelation and screen freezes.

Heavy Reading analyst Rick Thompson points out the San Antonio debut is probably happening in a very "controlled" network environment. With the technology world watching, AT&T is surely taking steps to make sure the fledgling service makes a good first impression.

But the issue of scaleability looms for AT&T’s engineers, as the carrier plans to roll out U-verse in 15 to 20 markets by the end of 2006. (See AT&T Readies Lightspeed in North Texas.) "It's one thing to get a complex technology like IPTV rolled out to a group of a few thousand users," says independent telecom analyst Kermit Ross. "But it’s quite another thing to kick that up a notch to a few hundred thousand and then yet another thing to kick it up to a few million users."

In other words, once AT&T cranks up the numbers, any failure to tame packet loss will be evident on millions of TV screens.

— Mark Sullivan, Reporter, Light Reading