The Battle for the Bottleneck

Since Peter and Henry each independently forwarded this Wall Street Journal article on cell phones to me, I thought I should respond here. First off, it is a truly excellent article, and I agree completely with Walter Mossberg, the author. He does a fine job laying out the key issues in an easy-to-understand manner. With respect to the upcoming wireless spectrum auction there, the odds of open handset rule going through went up this week when Verizon bailed on its attempt to sue the Commission over it.

Since Mossberg did such a good job discussing the state of the market, I thought I might discuss why I think the market is like this. The short version is this: while the Internet revolution has created a profusion of telecommunications that a short time ago would have been unimaginable, at the same time it has threatened to make telecom providers more irrelevant than they have been since their inception and so they fight it tooth and nail. And now for the (really) long version:

The problem with the Internet viewed from the perspective of the telecoms is that it is an end-to-end network, meaning everything interesting happens at the endpoints and they're stuck in the middle. The stuff in the middle is designed to be transparent and fungible (particularly in the case that some of it is vaporized by a nuclear bomb). This is very different from what we had before the Internet. Previously any goods or services traveling down the phone line into someone's home or office were controlled by the phone company, be it phone service, long distance service, voice mail, video, advanced data services, etc. Even if you just wanted to communicate data between you two office sites, the phone company would set this up then charge you out the wazoo. It was a good business, and they made a profit for each additional application they could develop and provide to consumers over their lines.

But when the Internet comes along, even though the number of applications now skyrockets and the benefits people gain from their telecommunications links grow immeasurably, the telecom companies are no longer in control, and are therefore no longer profiting directly from the provision of services. Whatever the telecom companies were charging people an arm and a leg for can now be done on the Internet for peanuts. So to the extent that telecom providers did provide unique services, now anyone who can pipe in an Internet connection over any medium can eat their lunch. You don't have to rely on the phone company for voice communications or cable for video anymore. And many applications the phone companies hadn't implemented yet or hadn't even dreamed of are now out there being heavily used for free. It seems like it should be their shining hour, but it's really more of a let-down. Sure they are still selling and making money on the basic connectivity that underlies all of the Internet, and in record volume, but they'll be selling that in any case. It's the add-ons, the profits coming from provision of actual commercial services to end-users they want. So while openness is what has made the Internet great for us users, there has been a very consistent effort on the part of telecom companies to maintain control where they haven't yet lost it, or to regain control where it has been lost.

At the start of this thing the big telecom companies didn't even want to deal with home consumers; for them data services were things to be sold to corporate customers at outrageous prices that no home user would contemplate paying. So when PC ownership took off and it became apparent that there was a market for online services, other companies got involved. The initial generations of modems were built to work over the phone lines by converting their digital signals into analog beeping and squawking (converted back to a digital signal by the modem at the other end of the line) so that they only needed a normal voice line to operate. Phone company participation was not required. The initial big corporate providers of dial-up online services (Microsoft, America Online, Prodigy) ignored the Internet and each built their own walled garden, each providing their own proprietary services to customers. In this sense they adopted the same model as the telecoms. Everything that passed down that line was to be provided by them, and they'd get paid for it. Of course, the fact that dial-up modems require no significant physical infrastructure (all you need is an office with a bunch of voice lines and some computers) meant that the barrier to entry to the online services business was very low. So small shops opened all over the place. They, of course, didn't have the infrastructure to create a bunch of proprietary content and services the way the big guys did, but there was this Internet thing they could hook people up to for pretty low costs that had a lot of content on it. It didn't take consumers long to figure out that there was a whole hell of a lot more interesting stuff outside the walled gardens than inside them (in large part because the users went out and created it themselves). So the walled garden model was soundly defeated (although it took AOL the better part of a decade to admit it).

So we all got onto the Internet and the phone companies had lost control over content and services and so had the ISP's. Everything was free and open and we went crazy with entrepreneurship and had the dot-com bubble. But while that was happening, we were also developing technologically. The networks were trying to keep up with our demand for faster connections to get all the cool new stuff on the Internet (which is to say mostly online gaming, pirated MP3's, and porn). So we moved from dial-up to DSL and cable modems. This is an important transition, because, unlike dial-up modems, DSL and cable modems require physical infrastructure in the telecom networks. There needs to be actual equipment sitting in the local office of the phone or cable company for them to work. All of the sudden the phone companies had a way to take back control over everything. But wait, luckily for us Congress was out ahead of the game and had made arrangements for this in the 1996 Telecom Act. That sounds sarcastic, but isn't; Congress had achieved a remarkably balanced and well-designed compromise. The Act required, among other things, that the regional phone companies play nice with competitors and rent out the equipment needed to make a DSL network function, so that competitors could get at the same customers the regional phone companies served. Problem solved, competitive pressure will prevent the phone companies from reestablishing control over the Internet and abusing their market power.

Sadly it didn't work out that way. The phone companies didn't like this part of the '96 Act (big shock), so they launched an intense campaign of lobbying, foot-dragging, and lawsuits that lasted until a few years ago when Bush's FCC appointees decided to use the Telecom Act for toilet paper and the Supreme Court cheered them on. Internet access is no longer considered a common carrier service and is not subject to open access requirements. Now instead of thousands of ISP's in direct competition with each other, Verizon and AT&T and Comcast and Time Warner basically own the consumer side of the Internet in the US. And, to varying degrees, they may be able to start rebuilding their walled gardens and capitalizing off of all the wonderful services people enjoy on the Internet, and turning our wonderful network into something a little less wonderful. This is what the fight over net neutrality is about.

One of the interesting battles in net neutrality is just how far it reaches into a provider's infrastructure. The case for net neutrality is most clear when it comes of an ISP's actual Internet service, where they're blocking a service or web site they don't want you to access. But there are other ways for an ISP to go about this. Consider cable. The cable companies set up their networks so that only a small part of their pipe is allocated for Internet service. The vast majority of it is carrying video. If they wanted they could scrap a few cable TV channels and expand the datastream to their cable modem subscribers. Now let's say that on-demand TV becomes a very popular service. Comcast has two ways to go about this. They could provide on-demand video pretty easily over the Internet, except they probably don't have enough bandwidth on their Internet service to handle it. So they would need to eliminate some video channels and expand their bandwidth first. But if they did that, anyone could sell on-demand video to their customers over the Internet, and Comcast would get nothing for it. They would have to compete for every customer, and the wouldn't have any real competitive advantage over other video providers. Alternatively, Comcast could keep their Internet service throttled down, and develop a proprietary on-Demand system that would occupy the same bandwidth but run independently of their Internet service. Now they have no competitors and get to tariff users for every use. Oh, happy day for Comcast.

All of this brings us, finally, to the mobile wireless market. There are three key points to notice. First, for technological reasons data services took quite some time to arrive in the mobile wireless market. So they got to sit by and watch all this other stuff happen. Second, like DSL and cable, wireless Internet service requires special equipment, meaning the wireless service provider is necessarily also the Internet service provider and has total control over the network. Third, the cast of the characters is fairly similar (AT&T and Verizon, again, plus a couple other big national companies). So what you have are companies that have learned that what they want, more than anything, is control and the ability to tariff users for every bloody thing they do on the network, and who technically have the capability in this market to make it happen. They have fought bitterly to maintain this control and will continue to. These are concentrated markets, so the threat of competition isn't that great, and clearly the fear of loss of control their users is much greater than their fear of competitive harm. And the big wireless companies are all moving in lock-step on this anyway, so there is no real competitive impetus to stop fucking with customers and actually give us what we want. This is the sort of thing that happens in concentrated markets…

On some level they know they'll eventually have to let their users onto the Internet for real (the iPhone is a concession to that reality, although AT&T saw to it that they get their pound of flesh from iPhone users anyway). But the longer they can keep customers off the Internet, or only let them on some borked up version of the Internet, the better. I mean if people can download ring-tones for free, who's going to pay their provider $3 a pop for them? If you can buy MP3's for your phone from iTunes for a buck, why would you pay Verizon twice as much for their proprietary music-to-phone system? Now they can sell you GPS-based mapping, sports scores, online games, SMS, emails with pictures from the camera phone, and every other damned thing you'd get for free on the Internet. If they provided good Internet service or allowed phones with WiFi capability onto the network, this would all come crashing down.

In the end, the simple reality is that the communications market is deeply dysfunctional. It's highly concentrated, has many components that are or were natural monopolies, and is littered with regulatory detritus from past battles with abusive monopolists (too many of which didn't come out the right way for the good guys). These are markets that desperately need oversight and consumer protections. Frankly, a lot of the physical infrastructure would probably be better off in the public sector. The Internet is an incredible economic engine and allows for so much innovation and competition, it's sad to let these crotchety bastards be the gatekeepers. The analogy to superhighways isn't that far off. It doesn't have to be like this. In other countries (generally where regulators have forced open access on the phone companies) people regularly pay about the same we do for Internet service that is ten times faster, they have mobile phones that can do WiFi and are interchangeable among providers, and they have no real issue over net neutrality because Internet providers don't have enough market power to abuse it. I'll be very curious to see what happens if the Democrats take over at the FCC in a couple years...