Content delivery network (CDN)

Net Neutrality: Level 3 Sees Peering Progress Soon

Despite the growing controversy over network interconnection points, a senior Level 3 Communications executive believes the peering disputes among content providers, middle-mile network operators, and broadband providers may be well on their way to getting resolved.

Speaking at Light Reading's recent Big Telecom Event (BTE)in Chicago, Bill Wohnoutka, vice president on the solutions architecture team at Level 3 Communications Inc. (NYSE: LVLT), said he believes the peering disputes among content providers, middle-mile network operators, and broadband providers will be much closer to resolution within a year. When asked on what will have changed in the market by June 2015, Wohnoutka said, "We will have progressed significantly on some of the peering capacity issues."

Wohnoutka's optimism comes even as public peering fights, particularly between Netflix Inc. (Nasdaq: NFLX) and ISPs, are heating up. Recently, Netflix agreed to pay both Comcast Corp. (Nasdaq: CMCSA, CMCSK) and Verizon Communications Inc. (NYSE: VZ) for direct interconnection links to improve the quality of video streaming on their broadband networks. However, the online video giant isn't happy with the situation, and it has publicly scolded ISPs over ongoing video streaming quality issues. (See Verizon Threatens to Sue Netflix.)

Meanwhile, as a middle-mile network provider, Level 3 gets involved in the issue because it helps to carry Netflix traffic to ISPs in some regions. In a blog post in May, Mark Taylor, vice president of content and media for Level 3, laid out in detail how ISPs have allowed some of their own peering points with Level 3 to degrade. According to Level 3, the tactic is a way of putting pressure on Netflix to cut deals directly with last-mile providers… leaving middlemen like Level 3 and Cogent Communications Holdings Inc. (Nasdaq: CCOI) out of the equation.

The Federal Communications Commission (FCC) has tried to stay out of peering disputes, claiming that interconnection deals aren't a network neutrality issue. However, Chairman Tom Wheeler agreed to look at the situation more closely this month after sustained public outcry. (See Net Neutrality Redux? FCC Probes Peering Problems.)

On the BTE panel -- which focused on the topic of multimedia distribution -- Wohnoutka said the FCC must consider peering agreements as a factor in net neutrality. Although Level 3 isn't necessarily seeking new regulations, Wohnoutka said he is hoping the FCC will "bring a rational dialogue to the table," and even mandate collaboration of some sort between middle-mile networks and last-mile providers.

As to why peering points have become a major issue now, Ted Middleton, vice president of product design for Verizon's Digital Media Services business, noted that, "Naturally as service providers bring CDNs in house … that changes the way that they're going to want to manage traffic profiles." In other words, because ISPs such as Comcast and Verizon are creating their own content delivery networks to manage IP traffic, there is a natural tendency to want to monetize those investments. Middleton, who was the panel with Wohnoutka, added that Verizon would like to see the marketplace remain open and encouraging of a fair exchange of traffic between networks.

Wohnoutka explained the peering situation another way. "We haven't evolved our attitudes, and video blew up in the meantime," he said.

According to Wohnoutka, Level 3 is now promoting a two-pronged approach to the peering problem. First, he said, there needs to be an expansion of the number of peering points in the US. There are only six to nine major peering points today, and that leads to serious bottlenecks when network traffic is heavy.

EdgeConneX Inc. Chief Architect Phill Lawson-Shanks, who was also on the BTE panel, agrees wholeheartedly with this point. His startup company is rapidly building out data centers to serve as peering points in dozens of US markets where connectivity is lacking.

Second, middle-mile and last-mile providers need to come up with collaborative caching models that help to spread costs more evenly between the two types of networks. Level 3, for example, has proposed a concept called bit-mile peering, which would calculate the costs involved in traffic exchange based on the combined variables of volume and distance traveled. Such a strategy would create incentives for more local caching because shorter distances would directly reduce costs.

Interestingly, Wohnoutka hinted that Level 3 may be making progress in furthering some of its collaborative ideas. "We're trying to find creative solutions with companies like Verizon and Comcast," he said.

Given Wohnoutka's optimism about peering progress over the next year, it sounds like, at the very least, his company hasn't hit a brick wall. That's good news for everyone -- network operators, content companies, and consumers alike.

— Mari Silbey, special to Light Reading

Infostack 7/9/2014 | 9:49:50 PM
Re: Self-serving Binge watching is changing people's habits.  They are time shifting consumption.  The trends are supported by Ooyala's data.  And by the way, most 4G networks can't handle more than a few minutes of video before they crap out; if you even have coverage.  At least that's my observation.  So much of it is wifi-offload and a few minutes here and there on 4G.

Again, when you get it.  My recent experience trying to watch just 2 WC games on ESPNGo this past week was pathetic.  In one place only 3G coverage; thank goodness for outdoor cable wifi.  The other was on a train where I had 2-4 4G bars constantly, but could only get 10 decent minutes of video out of 60.

Ya still haven't addressed my points about interconnection and where the WAN/MAN demarc should be if the markets were competitive.  It has enormous implications going forward.
Mitch Wagner 7/9/2014 | 7:20:13 PM
Re: Self-serving Mark Twain did a piece way back in the 1800s about Mississippi riverboat pilots forming what amounted to a union. At first, he said, only the least desirable pilots joined, and the union was a joke. Later, more are more pilots joined, until finally riverboat owners couldn't find a pilot unless that pilot was a member of the union. 

The same model applies today. If broadcasters' and cable provders' domination of video content is broken, it'll start with sports that barely anybody wants to see -- professional eating contests and badminton. At first. 

People watch longform video in pieces? Intersting. I confess to being old-fashioned enough that I resist that -- I want to watch a movie or episode of a TV drama in its entirety, or at most in two pieces. That's kept me away from the TRUE GRIT remake and HOUSE OF CARDS, because I haven't found the time to sit down to a whole episode. 
Infostack 7/3/2014 | 3:24:41 PM
Re: Self-serving @Fake

Wish you had commented on the interconnection issues, as that was my key point about the article and your comment.  ;-)

But as for the demise of the linearTV model, it could well be the disaffected outlier channels and smaller sports that collectively move "off-net" along with their viewership.  Demand is highly marginal and variable; it is not average.  The current service provider models and pricing bundles would have you think so; but it is not.

I would not want to bet on a biz-model that assumes a major sports network or cable channel breaks away because they also have too much to lose.

Another factor supporting my assertion is that long-form video is increasingly viewed in small bits on the mobile device and that mobile BB connections will exceed wired BB connections 5:1 in 5-10 years.  That is a relatively new trend as well and therefore not factoring into people's expectations at present.
Mitch Wagner 7/3/2014 | 1:37:10 PM
Re: Self-serving Infostack - " Just imagine they can get 20-30 cable channels to break away from the cable cabal (the key reason for Comcast/TW and AT&T/Direct is leverage over content) and form an "on-demand" package that retails for $20/month?  How quickly do you think 30% of the current pay TV base would cut the cord or trash the dish?"

Depends on which channels and what content. 

Live sports and marquee events like THE GAME OF THRONES will drive the breakup of the cable monopoloy, when it happens. 
Infostack 7/2/2014 | 5:09:31 AM
Re: Self-serving @Fake to put it all at the feet of Level3 is too one-sided and short-sighted.  There are several issues to consider.  

One is Netflix' flat rate all-you-can-eat pricing model (AYCE).  They have to continue to drive their per gigabyte consumed (stock) or mbs transmitted (flow) costs down.  In some ways they are a victim of their own success and probably need to evolve their pricing models; especially if they are going to get into premium and live.

That requires them to continue to drive WAN/core economies to the edge; particularly when 4K comes out and they want to cache that content out where demand really is.  The competitive CDN's and transit providers may not be in a position to do this; as Level3 points out.  But remember competitive forces are loathe to admit strategic or tactical weaknesses (even if they are created by exogenous forces externally) publicly because of the impact from the capital markets and competitors.

But the second point is that Netflix is very disruptive to those last mile access monopolies' linearTV models.  Just imagine they can get 20-30 cable channels to break away from the cable cabal (the key reason for Comcast/TW and AT&T/Direct is leverage over content) and form an "on-demand" package that retails for $20/month?  How quickly do you think 30% of the current pay TV base would cut the cord or trash the dish?  Especially as local signals can be picked up freely with a low-cost antennae and anyone can have their own Aereo.  It won't be easy, but 4K will help the process along.  And it could happen relatively quickly at any point in the next 2-5 years.

So looking at it another way, Netflix benefited from everyone else's relative scale as it rode out to the edge on layer 3-4 CDN's and layer 2 transport providers.  Now, those competitive (public) providers lose out on the benefit of Netflix' scale just as it can be the engine that pulls everyone out to the edge.  That's because the last mile edge access providers (incorrectly termed ISPs) are holding the WAN/MAN demarc closer to the core and forcing Netflix off the public net onto their private net.  We saw this move from public to private last in the enteprise market 50-60 years ago in the form of PBXs.

And all of this has big implications for future trends like 4K VoD, 2-way HD video collaboration, seamless mobile BB, and IoT; all of which require cloud intelligence moving to the edge to reduce capacity costs (particularly upstream) and latency, and improve QoS and security, and provide redundancy.

The last mile providers have 3 ways to extend their layer 1-2 monopolies:

-vertically integrate up the stack (which TCP/IP disrupts)

-virtually integrate up the stack by bundling and pricing (but now that is overpriced)

-pushing towards or holding the layer 1-2 demarc closer to the core (which is what they are doing with Netflix).

They've been exercising all of these in one fashion or another for 100 years, starting with the 50 mile exclusion zone in Kingsbury.  Another example was the extended flat-rate dial-up of the 1980s which inadvertently gave rise to the internet in the 1990s.

So, by rationalizing away Netflix' competitive choice as a mere "squeezing out" of Level3 and other competitive transit providers you are missing some of the key points the markets and policy makers should be discussing. 

Mitch Wagner 7/1/2014 | 5:34:52 PM
Self-serving Level3 seems self-serving here. If content providers like Netflix connect directly to service providers, that squeezes Level3 out. 

A deal is in the offing -- but will it preserve innovation, one of the key goals of net neutrality advocates?
msilbey 7/1/2014 | 8:13:35 AM
Re: Optimism in Necessity? Love the description/analogy. And yes, that about sums it up - acknowledging that we're dealing with a much less limited resource than oil, and one that isn't tied to location in the same way.

One of the many interesting conversations I heard at BTE was the ongoing debate about federating networks. Google talks about the need to federate with SDN development, but others point out that CDNs have been talking about network federation for years, with little result. As you say, wouldn't it be great for us all to work together? But everyone wants that grand alliance on slightly different terms. 
fanfare 7/1/2014 | 12:45:33 AM
Re: Optimism in Necessity? I'd say you are on the right track here.  Companies like VZ have other issues to consider when it comes to addressing peering, and net neutrality.  Comcast, on the other hand, has a somewhat smaller array of issues, and is more apt to push harder on something like we saw with Netflix.  While both Comcast and VZ want to continue to own local/residential access, VZ also needs to consider LH costs and all that implies.


LVLT knows that sooner or later things will have to smooth out.  With the push toward using it's IP transport powerhouse to focus on enterprise, the company is positioning itself to take advantage of the regulatory environment no matter which way it goes.  LVLT would ultimately like to open up the public trough for bandwidth, but if the last mile milking entities want to continue to push the other way, LVLT is going to be prepared to pull advantage on that end as well.  As many have said all along, we will go whichever way you want, but in the end, your going to either pay us.... or pay more to build your own transit services.

Succinctly, elite bandwidth assets, in LH and metro have moved too far into the spotlight and the days of LH/ULH being dismissed as over capacitied commodities are soon ending.  The value of being able to provide secure low latency transit on an end to end solution is fastly being recognized by the masses, and the other players see the writing on the wall.  LVLT will have the ability to flex no matter which way they push, and if they push too far in one direction, well they will end up putting themselves at a disadvantage from which LVLT will capitalize.  The only question for the other players is: do they want to go "with" us into the future... or do they want to be left behind "by us" in the future.
sam masud 6/30/2014 | 4:30:49 PM
Let's see Can't wait on the Solomonic (at least I hope it is Solomonic) agreement on this one. Still, get the feeling that whatever it is, it will not be the last word on it because it affects players at every level (content, transport, and ISPs).
smkinoshita 6/30/2014 | 3:06:09 PM
Optimism in Necessity? I wonder if the optimism comes in the form of necessity?  Net Neutrality is a hot issue (especially on sites like Reddit) and I wonder if it's one of those situations where what's best for everyone ultimately is what's best for everyone?

By that, I mean I wonder if his optimism is sort of similar to the oil cabal?  All the major players know they have a hot resource, but they know that in the long run there's little difference between the product when it works.  So while the major players would all like to maximize their profits, they know that if one pulls too much, another can take it all away by simply offering more for less.  

Unlike oil, where buyers can often switch easily between suppliers -- at least on the consumer level -- there's usually a little more inconvenience with Internet.  So winning that market share is more beneficial, thus all the players have to be 'nice' less the others take advantage of their greed (in the name of greed).

Am I on the right track here?
Sign In