Template Tools

We've moved

Everything since June 1 is at  fastnet.news. This is just the archive from before June, 2015

For the Record: Dave on Comcast Level 3 Issues
Wednesday, 01 December 2010 20:25
Jason Livingood pointed out a significant error in my posting to Dave Farber's Interesting People list, although I believe the analysis holds. First is my initial comments, than my back and forth with Jason. Painfully geeky. I'll try to edit it to a clear article as soon as I can.


Comcast says Level 3 should pay because they send more traffic to Level 3 than the other direction. There's an enormous amount of disinformation coming from all sides, that "the Internet will die" to "this is just an obscure peering problem. These are not the Droids you are looking for. Move along." Some thoughts: 

1-    Most important: Overall download to upload on broadband in now 3:1 to 5:1 and becoming less symmetric.
Every backbone provider to every broadband telco/cableco is asymmetric and getting more so. (That even applies to AT&T and Verizon despite their broadband customers, because the imbalance is strong.) (Note - see below for what may be an exception. Every is possibly too strong a word.)

So this is about charging everyone, not any particular issue with Level 3. Level 3 is by far the largest backbone, by Renesys figures three times the size of AT&T or Verizon. Every backbone will be in a range of upload:download similar to the consumer ratio and have to pay.

If Comcast, Verizon, and the other Big 8 impose this rule, most of the traffic to 80% of U.S. homes will be charged.

2- Comcast and the other carriers charging could be a good thing if much of the benefit were passed along to consumers in turn. Economists have a "two-sided market" model that with strong competition much of the benefit would be passed on. Cisco and other lobbyists have been ignorantly applying the theory where competition is weak and it doesn't apply.

With the local telco/cableco controlling 95% of U.S. broadband customers, I believe competition is weak and most of the benefit will go to company profits. Costs of delivering broadband service continue to go down every year while prices generally go up. With strong competition, prices would likely move down with costs.

Cable broadband according to John Hodulik of UBS and Craig Moffett of Bernstein has margins of about 90%. I'd include more of the network overhead but still find margins of over 70%. I do not believe increasing those margins important. I do believe keeping Internet costs down is a good thing.

3- Bandwidth is cheap but not free. Usage is increasing. If bandwidth costs were increasing significantly, I'd be much more open to ways to charge more. Fortunately, Moore's Law has been bringing the cost of bandwidth down very rapidly as well. Volume is up while cost per bit is down, both 25-40%/year. They've been about equal every year for almost a decade.

Comcast, Cisco, and AT&T are all noting a slight drop in the rate of traffic increase. Congestion according to Comcast is now significantly reduced, with "far fewer than 1% of customers affected by traffic management." Even then, Comcast says they reliably provide 7 megabits down even to the affected customers.

4- The total cost of bandwidth is far too low to explain substantial bandwidth charges, less than $1/month/customer at large carriers like Comcast. That's 1% to 3% of the $40/month charged. Again, if the charges Comcast wanted were in line with the related costs, I'd be much more sympathetic. I wrote "
Comcast's Fair 250 Gig Bandwidth Cap " because at that level usage is costly and a charge is fair. But this isn't about cost of service; it's about converting market power into income.

Ed Whitacre said "They're not going to use my pipes." His successor Randall Stevenson told Wall Street the same thing. I thought they would become the key test, but Comcast instead is the stalking horse. Level 3 picking up Netflix inspired them to look at how they might charge. The company's purpose is to make more money, so if they see an opportunity they take it.

The question is whether this is good for the Internet and the country. The FCC Chair owes us a close look and the courage to act if this is as costly for consumers as I think it is.



Jason is correct and I missed the change in Comcast's traffic. I've changed what I'm writing on this from "every" to "virtually all." I've asked Comcast's official spokesman for details about any large backbone that is close to an equal traffic ratio and haven't heard back yet.

Typical U.S. broadband providers are seeing a ratio around 4:1 downstream to upstream, increasing. So it's pretty certain few if any backbones will be close to symmetric.

Implication, which is as true for AT&T and Verizon as it is for Comcast: If they insist on being paid when traffic isn't symmetric, they effectively are demanding to be paid for most of the bits they receive. That would be an historic change, adding a level of charging to the Internet. (Prior moves to charge Akamai, etc. may have been a move in this direction and set a bad precedent.)

This could be a perfectly fine thing, if the market were highly competitive. Economists looking at "two sided markets" in that case would conclude that some of the charge would be passed on to the consumer as benefits.

I believe that isn't the case in the U.S., unfortunately, much as most of us would prefer solving problems through more competition. 95+% of U.S. broadband is through the local telcos or cablecos. That duopoly is strong market power and I believe yields what an economist would consider "weak competition." Wireless will play some role, but the majority of U.S. wireless - even Verizon's extraordinary new LTE network - has a cap that a 60+% under today's average U.S. broadband usage of about 15 gigabytes/month. So wireless is only a "partial substitution."

Under weak competition, I believe most of the benefit of this charge will go to the carriers' bottom line. With Wall Street using cableco financial reports to estimate a 90% margin on cable broadband, I don't think they need higher margins. (I would include more overhead and figure margins of 70-75%, still twice the corporate norm.) 

So as I "follow the money," the result of actions like Comcast's would be to add costs to the overall Internet system that are not related to the actual costs of providing the service. The additional income would mostly go to carrier profits, not consumer benefits.

That's especially true because high Internet charges handicap the important potential competition of video over the web, indirectly raising the cost of video service.

To estimate the impact of this charge, we need to know how much it is, ideally on a per gigabyte basis. All data welcome, on or off the record. My best guess is that the amount Comcast wants from Level 3 will not drastically change Internet economics in the next year or two. Because AT&T and Comcast each control close to 1/4th of the U.S. customers, they have the potential to ask for much more. That's why I think that so important.


p.s. thanks to Jason for catching my error. Correcting mistakes is the greatest favor you can do for a reporter.

p.p.s. Jason also is accurate, I believe that Internet traffic growth remains high. That's a good thing because it means people are benefiting more from the Internet. Moore's law is bringing costs per bit down, so I'm almost certain the cost per month per customer of bandwidth has been about flat. One large European carrier CEO tells the NY Times the cost is 55 cents/month but I believe he is particularly efficient. My reporting suggests a figure closer to $1/month for a typical large, wired carrier, 2-3% of the $30-60 typically charged the customer. By all accounts, the cost per customer is little changed over about 8 years. I'd love more accurate data from AT&T, Verizon, Comcast or others. Wireless is different, as are carriers too small to connect with fiber directly to peering points.
The original
Comcast's Joe Waz argued on the Comcast blog and Dave Farber's list that Comcast should charge Level 3 because the they were downloading from Level 3 more than they were uploading. The issue is that today essentially all U.S. broadband providers upload far less than they download. That would mean charging everyone, something I disagree with. So I posted
A question.
Are you contending that Comcast need not peer with carriers with substantial asymmetry, say within 100% of the national average?
I ask because the current ratio of downstream to upstream traffic on the U.S. Internet is about 4:1, which means essentially everyone except the carriers will have significantly asymmetric traffic with any large carrier such as Comcast. Comcast will always receive far more traffic than it sends with any backbone provider and the proportion is increasing.
Which implies that Comcast could charge for just about all the traffic coming in. If I'm reading you wrong, please get back to me.
I haven't heard from Joe with an answer.
Ben Popken of the Consumerist called as well, and quoted me saying
"Industry analyst Dave Burstein told Consumerist, "The Justice Department should step in with antitrust."

"As far as I know, no primary backbone provider like Level 3 has ever been required to pay to deliver traffic to another major carrier. Payments have only been required from smaller carriers who are not truly "peers."

"In particular, Comcast's claim their charge to a backbone provider is ordinary "paid peering" is bs. The details of Comcast's claims would mean they collect a fee from every bit, especially video over the Internet unless it's sent by AT&T or Verizon. AT&T and Verizon would of course gladly collect fees for video to their customers as well."

Nate Anderson of Ars Technica (a great site) included me in his story How Comcast became a toll-collecting, nuke-wielding hydra