(Don't) Be Evil

To follow up my post from a week and a half back, it turns out the reports on the Google-Verizon deal were more or less accurate. Google and Verizon have thrown in together on a proposal for net neutrality legislation. Don't be fooled into thinking the result is a thoughtful compromise--it is a near total capitulation on the part of Google to Verizon's demands (see here for a good "that was then, this is now" comparison of Google's positions). The proposal has been hashed over quite a bit, and I don't want to repeat what has been said elsewhere (you can see here or here for decent commentary, here for a survey of coverage). At a high level, the critical issues are: a) it completely exempts wireless from any rules; b) it creates an incredibly broad exception for any sort of managed services (allowing the creation of tariffed fast lanes); c) the non-discrimination language is so vague that it's hard to tell whether Comcast's interference with BitTorrent (which the FCC previously ruled against) would even be covered; d) the FCC is prohibited from engaging in any prospective rulemaking; e) the FCC's enforcement power is capped at a $2m fine (a pittance for the big carriers); and, f) the FCC is required to grant considerable deference to industry organizations in interpreting the rules. It is, as I stated in my last post, a devastating defeat for net neutrality.

And the deal leaves the FCC in an awful situation. Already the FCC spent so long dithering on this topic that they allowed the carriers to wage a massive and surprisingly successful campaign to lobby Congress against net neutrality (see here for an example of just how much clout they have). It's not a big stretch to think that all of those congressmen will jump on board with the Google-Verizon plan. And given how well respected Google has been as a net neutrality champion, it would not be surprising to see a lot of other more net neutrality-friendly legislators hoodwinked into thinking this is a legitimate compromise that they can get behind. Come November the political situation will only get worse. If the FCC was too timid to act before, the chances of them taking bold action now is about nil. The only real hope is that Genachowski feels so backed into a corner that his only choice is to fight back hard (I'm not terribly optimistic on this).

While I noted before that Google has real conflicts of interest on the net neutrality question, on reflection it is still pretty stunning that they did this. Google is increasingly stepping into fraught policy questions, from net neutrality, to international trade and human rights, to antitrust, to copyright (also this), to privacy issues, and others. In all of these areas, Google has long benefited from its pro-technology, pro-openness, "Don't Be Evil" image. It's not that Google could do no wrong, so much as that techies would always give them the benefit of the doubt. That makes a big difference. But no longer. Google irrevocably shattered that image in a single blow (though the constant drip of all these other issues over the past several years surely didn't help). They will now be viewed as just another mercenary big company feeding its bottom line. The betrayal on net neutrality will cost them on many fronts, and unless there is some quid pro quo from Verizon that we don't yet know about, it doesn't seem like Google got much out of this deal. It's really difficult to fathom why they did what they just did.

Update (8/16): As if on cue to prove my point that things will only get worse, the Tea Partiers just launched a crusade against net neutrality.

Can't Win For Losing

I hate to be all telecom all the time, but this is kind of big news. The FCC has dithered, vacillated, and backslid on net neutrality long enough that Google decided to take matters into their own hands and cut a deal with Verizon. Both parties now appear to be denying that anything happened. But it was extensively reported and it's hard to believe there is nothing to it. And the deal, if there was one, appears to have demolished the FCC's quixotic efforts to reach a solution on net neutrality that left everyone happy. It's not exactly clear what the deal was, but the basic outline of it seems to be that Verizon would have free rein to do whatever the hell they want with wireless service, would mostly be non-discriminatory with wireline service, but would have some sort of for-fee prioritization setup. In other words, it's a pretty devastating loss for the net neutrality supporters.

I think there is a lot of sentiment that we've been stabbed in the back by Google, but Google was never an ideal champion for this cause. They're in some sense the best we've got because they have money and prestige and that seems to be the only real currency in policy-making circles these days. They also have that nice motto about not doing evil. But they have pretty conflicted interests on this. Allowing the service providers to create fast lanes does allow them extort a cut of the revenue from online content and service providers like Google, and will almost certainly hurt Google's profit margins. But Google has a lot of revenue and a lot of money. And the other big impact of it would be to increase the barriers to entry for disruptive new entrants in the online service and content business. It would cost more to get started and increase the companies' cash burn rate, making it harder for businesses to start small and organically build their businesses as they refine their service offerings with the early adopters (which was the path followed by the likes of Napster, YouTube, MySpace, Facebook, Twitter, not to mention Google itself). Basically, having climbed to the top of the heap, Google can, by throwing in with the service providers, attempt to pull the ladder up behind themselves. Google will have significant leverage with service providers. New businesses won't.

So it's great that Google has been (and if its press release it to be believed, still is) a strong advocate for net neutrality, but it's worth noting that their interests don't align completely with the interests of other net neutrality advocates.

But I think the big story here is what a disaster the FCC approach to net neutrality has been. Harold Feld has a really phenomenal post on this, and I don't want to rehash everything he says. So go read it (and note that it was written before this Google mess). And I agree with basically all of that. And I agree with the commenter to that post who notes that this is symptomatic of the way the Obama Administration appears to work (which is more or less what I said back in this post). Virtually every worthwhile policy initiative has an entrenched interest that will oppose it, and the Administration is so cautious and conflict-averse they appear to be willing to let everything be derailed by endless deliberation. They just need to pick some fights, take action, let the incumbents howl, defend themselves to the public as best they can, and then let the chips fall where they may. What they're doing now is completely ineffective (if not counterproductive) and, as Feld notes, utterly demoralizing to anyone trying to serve the public interest. Hopefully this thing with Google, whatever the truth of it really is, will serve as the swift kick in the ass that the FCC needs.

The Optimal Solution

Here's another telecom post (I've got some non-telecom stuff lined up for the near future). As I noted in my post last Wednesday, while net neutrality is currently a necessity, it is a far from optimal solution. It is exceedingly difficult to scope and define. For one, it typically relies on a nebulous concept of "reasonable network management practices" which are considered to be acceptable forms of discrimination. Also, it is difficult to figure out how to make it address service providers segmenting their network connection to deliver managed services (primarily video and voice) over bandwidth kept separate from the Internet segment of the connection without becoming increasingly invasive and intrusive into service provider business practices.

Indeed, as I concluded that post, it would be far better to negate the need for net neutrality by forcing carriers to provider open access to the local loop to their competitors. This solution, however, has its own problems. The biggest one is a pricing problem. When you force open access, you also have to set rates. Set them too high, and you've accomplished nothing because the competitors will be unable to compete with the incumbent and no competition materializes. Set them too low and the incumbent gets choked for revenue and cannot profitably maintain its infrastructure. Accurate, objective pricing data will be difficult to come by, and both sides will have strong motives to skew the numbers in their favor. It will be a constant back-and-forth battle, rife with lobbying, and with regulators ultimately picking winners and losers (either intentionally or accidentally).

No, the optimal solution is this solution. One government-funded fiber network to rule them all. For a one-time investment of about $38b US dollars, Australia will provide an essentially permanent solution to the broadband needs of all of its citizens. The network will be open access, allowing multiple service providers to compete to provide Internet access. It eliminates the natural monopoly of the last mile by making it a publicly-owned utility asset. And by wiring every home with fiber, it eliminates the need to ever perform a major network upgrade again. From here on out it will just be a matter of maintaining existing plant, and occasionally upgrading the optics at each end of the pipe.

Obviously, this would cost a lot more in the US. Looking at just in terms of the population ratio you'd figure that it would cost something like $400-500 billion to do the same thing here. Given the current fiscal situation, that's pretty unlikely to happen. But any municipal or county government with access to some cash really should be doing this.

Some Old News (Net Neutrality)

In an effort to continue to add some new blog content, I'm going to cheat and recycle a blog-worthy (I hope) email I sent a few months ago in response to inquiries from a couple of folks about my opinion on a) this article by Tim B. Lee and b) the D.C. Circuit's opinion in Comcast v. FCC. My reply follows:

Tim B. Lee's (not to be confused with Internet pioneer Tim Berners-Lee) paper has a lot of good content. Where it falls apart is on its competition analysis. Lee's fundamental premise is that interfering with network openness decreases the value of the network, service providers have an interest in maximizing the value of the product they're selling, and, therefore, service providers will not want to interfere with the openness of the network. On p. 23, Lee says this will hold true even in the case of a monopoly. This analysis, I think, ignores a critical factor: the elasticity of the market. Lee's argument holds true only in an elastic market, where consumers will be highly responsive to changes in the value of the good. I would argue that broadband Internet access is a very inelastic market. If you look at usage statistics for people under the age of 65 (and even more dramatically for those under 50), virtually everyone uses the Internet. It plays an integral role in virtually every facet of people's lives--personal communication, social and family relationships, entertainment, education, work and employment-seeking, access to government services, etc. It is not something people will do without based on incremental changes in the value of the product. So if you are hypothesizing a monopoly market, consumers have very little leverage with which to discipline service providers. Service providers would have to do tremendous damage to the product value before they saw a significant change in consumer behavior. Consequently, service providers are free to consider ways in which to make the network more lucrative for themselves (charging content providers for access to exclusive fast lanes and the like), even if it moderately decreases the value of the product they are offering consumers.

To take that one step farther, if service providers can get away with decreasing the current value of the network, they have absolute freedom with respect to controlling the emergence of future services that might add value to consumers. They already have a captive market based on the present value of the offering, so they would have nothing to lose by steering future developments in ways that benefit their bottom line, even if in doing so they diminish the overall value of these developments to their consumers. This is the most concerning aspect. It would be bad to diminish the availability or value of existing Internet-based services, but it would be tragic to undermine the Internet as a source of creativity and innovation.

The crux of this dispute, at present, is video services. Based on the history of the Internet, I think we have to assume that there will continue to be unforeseen developments and emerging new services, and some of these will probably require very high bandwidth connections. But at present, the only applications that really require 30+ Mbps connections are video services (and even then this is mostly for high def, and, in the relatively near future, 3d video). Lee touches on the fact that cable and fiber-based service providers already have a walled-garden for video, but acts as if this stands separate from the net neutrality discussion. It doesn't. I have participated in discussions on net neutrality with the Assistant Secretary for NTIA (outside of the 5 FCC commissioners, probably the most influential government official on this matter), and these walled-off video services were very much a part of the discussion. And the service providers are hardly content to rest on the status of their present walled-gardens for video. Look at the pricing for FiOS or for the emerging DOCSIS 3.0 cable services. Low speed Internet connections (5-20 Mbps) are priced similarly to the prices from other Internet service providers, but the high speed connections (50+ Mbps), which could potentially threaten their video services, are always set at such a high price point that the Internet-only price exceeds the price of getting a lower-speed video connection bundled with video service. It's a pricing structure designed to protect the video services. Meanwhile, Time Warner and Comcast are openly attempting to lock down online distribution rights for cable TV content. They have stated that they will make online access to this content contingent on the user having a subscription to their video service. AT&T and Verizon have not publicly disclosed any parallel efforts, but I would be shocked if they were not working behind the scenes to lock down content of their own. And Comcast is now in the midst of a high profile effort to acquire NBC Universal, bringing in a huge library of content and programming to add their exclusive access system. Acquiring NBC will also give Comcast partial control over Hulu, which will be one of its primary online TV competitors.

All told, this is a broad and comprehensive effort to lock down a high-value, high-bandwidth data service. And the general unavailability of high speed connections combined with market uncertainty over licensing and content access issues has successfully prevented the development of any true innovative competitors in the IPTV space. Moreover, because one element of the service provider strategy is to price high speed connections out of the market, this has had the collateral effect of slowing or preventing the development of other potential high speed services and applications (this is why Google is now proposing to create a city-wide testbed for a 1 Gbps fiber-to-the-home network). Letting Internet service providers into the online content and services market completely upends our expectations, per Lee, of how they should behave to improve the value of their Internet service product. Net neutrality advocates need not restrict themselves to worrying about hypothetical future harms, the real thing is happening right in front of our faces.

In discussing the potential threat of walled-gardens, Lee raises the example of AOL. AOL famously began as a walled-garden with exclusive content, then eventually, grudgingly, gave their users access to the Internet, and finally the exclusive content was dropped entirely. According to Lee, this illustrates that walled-gardens are not profitable. The key point that Lee omits is that AOL was not a monopoly (or even a duopoly). AOL's actions were driven by intense competition, mostly with mom-and-pop Internet service providers, at a time when all it took to be an ISP was to park a computer at the end of a copper phone line. The current service providers are generally monopolists or duopolists. The cost of entry to the market is prohibitively expensive (and economically inefficient). And the incumbents are so large that their collective actions can shape the development of content and services on the Internet in a way that AOL never could. The fact that walled-gardens failed in the 1990's tells us nothing about whether they could be economically viable for Comcast, Verizon, AT&T, and TWC today.

This brings us to the D.C. Circuit's decision last week. In 1996, Congress passed a Telecommunications Act that set up a regime for competitor access to the phone companies' last mile networks, allowing them to compete with the phone companies to provide phone and Internet service over their own infrastructure. At the time there were no commercial cable Internet service providers. As the cable companies became a larger and larger part of the broadband Internet service market, the disparity between phone companies (who were forced to share their plant with competitors) and cable companies (who weren't) became pronounced. Lawsuits by competitive Internet service providers attempting to gain access to the cable companies' plant forced the FCC to make a decision as to how to reestablish parity between the phone and cable companies. By this time (2002), Bush II was in office, and the default Republican position of helping the big guys prevailed. The FCC decided to designate Internet service as an "information service". The Communications Act gives the FCC a powerful tool box of regulatory controls over "telecommunications service" (generally referred to as Title II), but very little authority over "information services". The FCC had to shamefully torture the statutory definitions for telecommunications service and information service in order to make this determination, but that is neither here nor there. Several years later, after the case of a small phone company (Madison River Communications) blocking VoIP services that competed with its voice services, the Commission realized that they might want to regulate the behavior of Internet service providers in some cases. Having excluded Internet service from Title II, they didn't really have the statutory authority to do so, but decided to release a list 4 of non-binding network openness principles. A couple years after that, various members of the public were able to definitively prove that Comcast was interfering with BitTorrent traffic on its network, and they filed complaints with the FCC. After a long proceeding, in 2008, the Commission found the Comcast had violated the 4 principles, and required them to stop. Comcast challenged in the D.C. Circuit, asserting, among other things, that the FCC lacked statutory authority to enforce the 4 principles. They were right. Immediately following the FCC's Comcast decision, I did the first draft of a memo on the decision for a client. I predicted that Comcast would win the case in exactly the manner they did. So I can't say that I disagree with the D.C. Circuit's opinion.

At this point I think there is little question that the FCC will continue to enforce net neutrality principles. It was really the only big ticket item in Barack Obama's 2008 platform that related the FCC. Obama has reiterated his commitment to net neutrality on multiple occasions since taking office. It has to be at the top of the Democratic FCC commissioners' priority lists. So, to me, the question is not if, but how. Already there is a lot of talk on Capitol Hill about creating new legislation to give the FCC authority to enforce net neutrality. However, given the state of the senate, and the other major legislative efforts under way, it seems unlikely that anything like this could become law for a year or two, if at all. One quick way for the FCC to fix the problem by themselves would be to reclassify Internet service as a Title II telecommunications service. Two of the Democratic commissioners (Copps and Clyburn) appear to be inclined to doing this. The chairman, however, has indicated reluctance to do so. Reclassification would not only allow the Commission to enforce net neutrality, but would expose service providers (now including cable providers) to a whole suite of requirements under Title II. This would be a major change in regulatory approach, and the service providers will spend hundreds of millions of dollars over the next year or two to lobby against it. And even once the Commission acts, the service providers will then tie it up in lawsuits for years (as they did to the 1996 Act, from the time it was passed until the Bush-led Commission threw in the towel). It wouldn't be a pretty picture. But I nonetheless think it would be the right thing to do.

Moving Internet services back into Title II would allow the Commission to enforce net neutrality, but more importantly it would give them the tools to force the service providers to open their network to competitors. If done well, that could negate the need for net neutrality. Net neutrality is a regulatory framework designed for non-competitive markets. In a competitive market, consumers can respond to changes in the value of goods being offered, and Lee's thesis above would actually apply. Service providers that cripple their networks in the interest of building walled-gardens will see their subscribers go elsewhere, and the market will provide sufficient discipline to ensure good behavior. The only net neutrality regulation that would be required would be a network management disclosure requirement to ensure that consumers were making informed decisions. I think net neutrality is necessary given the current state of the market, but it is not an optimal solution. Competition would be far preferable. And, of course, for the service providers, competition is a far scarier prospect that net neutrality. Consequently, what we are likely to see is Chairman Genachowski publicly wavering on whether to join with his fellow Democratic commissions in favor of classifying Internet service as a Title II service until the service providers cry uncle and agree to some sort of non-Title II regulatory framework in which the FCC will be able to enforce net neutrality. Again, not my preferred solution, but it's what I view as the most politically feasible approach for Genachowski, and it has a distinctly Obama-esque feel to it.

I'll Have Another Hit of Hope and Change, Please

So it's been, hoo boy, over a year since I posted anything here. I'm once again going to try to get in the habit, and I figure I should start out with something strong. So here's an amazing presentation by Larry Lessig that provides a useful primer on the topics of broadband policy, cybersecurity, and copyright, then bends all three of these topics into a larger point about America's political process. It is very much worth the time to watch.



It's almost an hour long, but I think it stops 20 minutes too soon. He never answers the question of, well, what do we do now? I see that he's promoting a website for Fix Congress First. They appear to have a well-thought-out public campaign financing bill. But is that really the answer? Is there any real possibility this can get passed? (My guess: no.) Could it stand up to Supreme Court review? (Probably not, unless one or more of the conservative justices keels over while the Democrats still hold the White House.) If it did become law, would it solve the problem? That's hard to say, but at least it would be a huge improvement.

In any case, this seems like a hard thing to do as an insurgency. Lessig hits the crux of the problem when he notes that people commonly react that of course the powerful business interests run everything--it's always been like that! Certainly there has always been a certain amount of influence-trading in Washington D.C., but I have a hard time believing that it has always been like this. Nonetheless, I think the tendency among the public to believe that this is the way it always has been and always will be, until the end of time, amen, may present an insurmountable barrier for the popular uprising approach to political reform.

What I'm trying to say, in a long-winded way, is that my one source of crushing disappointment with the Obama administration has been its complete unwillingness to confront any powerful business interest. There were many things I admired about candidate Obama, but highest among them was his apparent dedication to changing Washington and improving the political process. I had hoped it would be the administration, rather than Lessig and his merry band of outlaws, spearheading the movement for political reform. I had hoped that he would confront entrenched interests and use his platform to show how badly they had served the public in the past and how directly their current interests conflicted with the public interest. None of this has happened.

Obama prevented a depression, passed landmark health care legislation and financial reform, and has done a lot of other good things. But he has done nothing in the realm of political process reform. Lessig is right that the FCC totally rolled over on the broadband plan. And Greenwald is right the administration only made it past the entrenched interests on health care reform by buying them out. And something not so different just played out on the banking regulations. It is honestly shocking to me how fearful this administration has been of entrenched business interests. Given the populist mood in the country, I should think the White House would relish a good fight with an unpopular industry (like the banks, or the health insurance companies, or Comcast). That, frankly, is the sort of press they need. But instead, when they get even a whiff of a fight like that coming, they turn tail and head for cover.

I don't know what to conclude. The Obama administration having been a letdown on this, I don't see any viable path towards political process reform in the near future. Between that and the new requirement for a super-majority to pass anything in the Senate (fodder for another blog post), the federal government has truly reached a new level of dysfunction. Despite Obama's legislative successes of the past couple years, I am deeply dismayed about the prospects of the federal government competently addressing any of the major challenges that will come its way over the next 10+ years. We are in a bad way to be sure.

Further Developments in the War on IPTV

A few weeks ago I noted that the networks shut down Boxee's efforts to deliver their content to viewers' TVs via Hulu's streaming web service. I speculated there as to why the networks would not want to play ball with Boxee. There is no question, on the other hand, why the cable companies are frightened of pure IPTV services, and it should come as no surprise that they are actively looking for ways to wall off content from them. Comcast and Time Warner are attempting to lock up cable TV content for online distribution (more details here). This content will be transmitted over the Internet in the same manner that a pure IPTV service would transmit it. The catch is that it will only be available to people who also subscribe to the cable companies' TV service.

Many cable TV programmers are likely to jump at this opportunity, as they've never been able to survive on a purely ad-supported basis and rely on cable carriage fees for about half their revenue. And, of course, even on that basis many cable channels could not survive if they were not tied together in cable tiers with other more popular channels (this is what much of the fight over a la carte cable revolves around). In fact, ESPN got out ahead of this game by trying (with considerable success) to strong arm ISPs into paying for its exclusive online content in a system analogous to a cable carriage agreement. What's next, regulatory battles over a la carte Internet? Oh, Kevin Martin, where have you gone?

On the other hand, for any really successful cable channels, my guess is they could do better by going it alone. This move will necessarily limit their online audience (as big as Comcast and Time Warner are, there are a lot of folks on the Internet who are not Comcast/Time Warner subscribers [though Harold Feld suggests that all MVPDs will be in on this game--anticompetitive conspiracy anyone?]). Moreover, they will be stuck in the position of subsidizing the crappy cable channels just as they have been through cable tiering all along. And if the good channels all flee, this may end up to be a pointless endeavor for the cable companies.

Obviously this move creates one more hurdle for IPTV providers to jump before going head-to-head with cable and telco video services. But it also creates potential net neutrality questions. This article suggests that Comcast will not treat this content any differently from other traffic for the purposes for traffic management or bandwidth caps. But if they or any other ISP were to in any respect preference this traffic, I'm calling it right now: instant FCC smack-down. This is exactly the sort of thing that net neutrality is intended to prevent.

In fact, I think the existence of monthly bandwidth caps at all will soon become highly suspect in the FCC's view. The Comcast Order hinted that monthly caps might be an acceptable network management technique. But anyone who routinely uses an Internet connection to view high def video will chew through these caps (I've seen estimates that HD video requires 4-12GB/hr (depending largely on the type of content), meaning a 250 Gb cap will last between 20-60 hours spread across all PCs and TVs in the household). In essence, monthly caps can be utilized to preference a non-IP-based video service over IP-based competitors.

ps. As you might note from my links here and in the previous post, I've been enjoying Silicon Valley Insider's Dan Frommer on this topic. He seems to be the go-to guy for this stuff.

Where Copyright Enforcement Collides With Network Management

ArsTechnica brings us this story about the fascinating next step in the arms race between online filesharing and the content industries. A new P2P client, developed by the University of Washington, integrates BitTorrent into a social-networking-type framework that renders it impractical for any user to determine the ultimate origin of files he downloads.

The way this apparently works is that a user installs the client and links to his buddies (who have likewise linked to their buddies). When he runs a search it will poll the buddies to see if anyone has the desired file. If not, it will search the buddies' buddies, expanding outward until it finds the file. The file is then routed back to the original user through each intermediate buddy and, significantly, the source of the file is anonymized at each step so that each client is aware only of its immediate neighbors. Consequently an MPAA executive searching the network for infringing files would have no idea which users to sue. They would not have IP addresses or any other identifying information for anyone beyond their immediate circle of friends (supposing, for the sake of argument, that MPAA executives have friends).

It's a great system for people who want to avoid being sued for copyright infringement. The flip side, however, is that routing files through a bunch of intermediate steps rather than directly from the ultimate source to the downloader is hugely inefficient. You burn a lot of network resources to achieve anonymity. This is where the interests of the content industries knock heads with the interests of the ISPs. The ISPs have long had a love-hate relationship with filesharing as it is a network management challenge, but also a great value proposition for their customers. This is less true now, but in the early days filesharing was one of the primary drivers for consumer adoption of broadband. Not only would ISPs be better off with the straight BitTorrent model, but BitTorrent could be optimized to prefer nearby nodes, thereby increasing its efficiency and lowering the burden of filesharing on ISPs. This can be done to some extent by software alone, but could be further enhanced through the cooperation between the software developers and the ISPs. Moreover, the FCC's decision in the Comcast case makes clear that the ISPs are on dangerous ground when they attempt to interfere with filesharing software. Optimization of filesharing software, rather than an escalating battle, seems the sounder option for them.

How this conflict gets resolved is unclear. In the near term there is little that the ISPs can do about it. But at least they may start to become involved in the policy battles over copyrights and filesharing, and one could hope that the significant collateral damage inflicted by this fight, combined with its general futility, could lead to more intelligent policy. It is interesting to note that the adverse parties in this conflict (the big ISPs and major content providers) are the same parties that appear to be aligned on the same side of the Hulu-Boxee story I posted earlier. I don't see any interplay between the two issues at this point, but it's something to watch for.

As an aside, the article quotes the creators of the software as stating that its intent is, in part, to create a platform free from the prying eyes of an oppressive government. This is likely BS. While this software would make it considerably more difficult to track a file to its source, it is not impossible. With access to the PCs of the intermediate users (or, more likely, the logs of their ISPs) it is still possible to track a file back to its original source. While this is impractical in the context of a copyright claim, it is not something I would want to stake my life or liberty on where an oppressive regime is concerned.

ps. Public service announcement: This is unbelievably awesome.

The New Battleground for IPTV

The battle over net neutrality has always been, first and foremost, a battle about who gets to provide video services over the Internet. As such, I think that what just happened between Hulu and Boxee could have significant implications for the future of telecommunications in the US.

Spurred on by the massive success of YouTube, online video has made huge strides in recent years. We're now seeing multiple competing sources for online movies and TV. One of the last remaining hurdles for these providers to overcome is to make the move from consumers' computer monitors to their televisions. One potential route for this to happen is through gaming consoles. Another route is through dedicated hardware, basically a set-top box that connects to the Internet and serves videos. Boxee makes one such device. The key to commercial success for any such device is access to content, and Boxee had until recently been planning to get content from, among other sources, streaming video provider Hulu. Hulu carries content from numerous TV networks, including Fox, NBC, Comedy Central, PBS, and Sci Fi. One could imagine Boxee, with support for Hulu, Netflix, YouTube, and various other sources posing a legitimate threat to traditional cable and fiber video services (e.g. Comcast or FiOS).

The emergence of such a free-standing IPTV service would be an important development that could break the video content market wide open and end the stranglehold that the big cable companies and telcos currently have over it. So it is disturbing to see that the powers that control Hulu (Fox and NBC) nixed Hulu's deal with Boxee. I assume that the scenario played out more or less as Marc Hedlund describes in the article. But it's not entirely clear what the motivation of the content providers is here. I think Hedlund is probably right that the advertising contracts probably work differently for online viewing that for traditional TV viewing, and the the networks have some profit incentive to protect their TV advertising. But this should be a temporary limitation--as new contracts are entered or old contracts are renewed there is no reason why fee structures shouldn't be adjusted to reflect the new reality that Internet video may be viewed on TVs as well as computer monitors. And to that extent, one might think that the networks would have some interest in seeing Boxee get off the ground and build some viewership so that the networks would be able to turn around and sell access to those viewers to potential advertisers. Instead they seem inclined to strangle Boxee in the cradle.

It's possible this is just a negotiating tactic and the networks want to squeeze a few dollars out of Boxee before they acquiesce in the Boxee-Hulu deal. But it's also possible that there are greater philosophical differences at work. It could be that the networks are cozy with cable operators and the telcos (or are frightened of them) and don't want to rock the boat. Or that the networks are concerned about their future in a world of IPTV. There's really no such thing as a "channel" on IPTV. And what, exactly, is a television network with no channels? Ultimately I do think there would be an important role for the networks in an IPTV environment, something more akin to what movie studios do: picking potential projects, then financing, producing, and marketing them. But it would be a big change, and if we've learned anything from the struggles of old content industries on the Internet so far it's that big incumbents are extremely resistant to embracing change. Additionally, even though the networks will still be important in an IPTV universe, they will certainly be subject to new competition. It would be far cheaper and easier for independently produced content to gain an audience via IPTV than it is in the traditional video market.

In any case, if the networks are inclined to fight IPTV, they are in position to set back its progress considerably. Without major network support, IPTV is likely doomed to being a supplementary service to traditional video rather than a direct competitor with it. This would be a bad result for consumers and a bad result for both the telecom market and the video content market, but sadly I'm not sure that it is irrational from the standpoint of the networks. I'm also not sure, at this point, that there is much that can be done from a regulatory standpoint to push the networks towards IPTV, but we should be on the lookout for opportunities to do so.

The Brave New Media World

Henry,

You raise a few interesting points. First, I would just note that your major premise is one that we've discussed a few times at Boys' events, although I can't recall whether you've been present for any of those discussions. It was the subject of a short but prescient book by Cass Sunstein titled Republic.com. He argued, precisely as you have, that proliferation in media sources will fuel self-selection of sources that confirm preexisting biases, undermining our cultural common ground and fragmenting our social beliefs. I think the threat is real, but can be overstated (as I'll discuss further below).

Second, I know exactly what you're talking about with regards to the comment sections on news sites. That stuff is purely toxic. It's even worse than you find on the purely partisan blogs and forums (e.g. DailyKos or RedState). News sites and YouTube comment threads and things of that nature appear to draw the absolute dregs for commenters. It's disturbing to read it, but ultimately I don't think that amounts to much. I suspect that it really is a very small percentage of the people who read the news stories or watch videos who post those comments (and probably with a high percentage of repeat players from one site to another). The numbers of people who participate in sites like DailyKos and RedState are more significant. Of course, those people also tend to suffer from groupthink and hyperbole and certainly represent a fragmentation in worldviews. But again, I'm not yet convinced that this is especially problematic. Their influence is ultimately fairly limited, as each party's Presidential primary this year demonstrated. In both cases the netroots favorites (Edwards for the Democrats, anybody-but-McCain (Ron Paul?) for the Republicans) failed to win.

Ultimately I think the democratization of public discourse via the Internet has been a tremendous win for the public. Contrary to your contention that we've seen a disappearance of sincere public discourse, I think we're seeing the emergence of new and important avenues for public discourse. There is a dialogue taking place on a community of blogs that surpasses in intelligence and candidness anything I can recall from the pre-Internet (or early Internet) days. To name just a few: Andrew Sullivan, Obsidian Wings, Matt Yglesias, Nate Silver, Al Giordano, Dan Drezner, Greg Djerejian, Ezra Klein, Tyler Cowen, John Cole, Cunning Realist, Josh Marshall. This short list covers a pretty broad spectrum of opinions (and certainly there are many others out there). These people are intelligent, generally well-educated, have diverse areas of expertise, and are interested in honest and sincere dialogue. Better yet, many of them read one another (and many other bloggers) and actually engage in back-and-forth discussion.

While the readership of the blogging community is still not at the same level as traditional media consumption, I think its influence is considerable. The bigger blogs (Sullivan, Marshall) draw in the neighborhood of a million page views a day, which is not insignificant. Moreover, that readership is disproportionately drawn from politicians (or at least political staffers), policymakers, and the media. This blogging community has the ability to affect campaign narratives. I think this is one of the reasons the McCain campaign flailed so wildly from one narrative to another over the Summer and Fall. The McCain strategists, all Karl Rove proteges, couldn't understand why their Obama narratives were not sticking the way the Al Gore - fibber and John Kerry - aloof liberal narratives stuck. The bloggers ripped their idiotic narratives to shreds and were quick to pounce if the traditional media went too far astray.

The emergence of this community presents a challenge to Sunstein's predictions. Certainly there is plenty of evidence on the Internet of Sunstinian self-selection and reinforcement of biases. But there is also a conscious acknowledgment among others that in order to be relevant as a thinker and a political writer one must be willing to inhabit the same world as other writers and one must be willing to engage differing views on more than a superficial level. Certainly this creates some tension between the self-reinforcing Freepers and Kossacks and the reality-based bloggers. On the conservative side this tension has spilled over into something like open warfare with the Freepers and Redstaters attempting to expel any conservatives who do not fully acquiesce in the Bush/Rove Republican orthodoxy. It's not clear how this is going to play out, although it seems inevitable that it will end badly for the Freepers, who are at this point quite remote from reality.

To conclude, I think, Henry, that you and Sunstein both had useful insights into the impact of hugely diverse media landscape, but I think the danger here is easily overstated. The broad expansion of the media landscape has occurred because the barriers to publication have dropped to almost nil (case in point: the Boys Weekend Journal), and the elimination of these barriers has also had tremendously positive effects. It does require people to be more savvy in their consumption of, and interaction with, the news, but the Internet has also made a great many people far more savvy about news media than the public has previously been. And the ease of entry into the market and the ease with which consumers can move from one product to another has allowed for something much closer to a true meritocracy than has ever existed in the traditional media. Nate Silver could never have gone from obscure bystander to political media powerhouse in less than a year in the old media market. I would go so far as to say we are entering a renaissance of political discourse. But the interplay between substantial number of people who have defied Sunstein's expectations with the still considerable masses who have conformed to his expectations does warrant continued observation. While relations between the two groups may frequently be strained, I think by virtue of pure merit alone the reality-based bloggers will always remain relevant and important.

American Politics

I was just watching a very good MIT panel on Media coverage of the presidential election when I heard a quote (1:56:50) I just had to transcribe and post here. John Carroll says:

I would like to think that actual reasoned conversation and argument might actually have some effect on people's perceptions and how they vote. I mean, that would be a giant step forward I think. But I'm not sure that's actually going to happen because the two sides have hardened so much. I think Tom [Rosenstiel] writes about this in his book, that there used to be a shared set of facts that people would argue from. They would agree on some basic facts and then, they would argue from there. I think people don't agree on basic facts anymore and I don't think that anybody allows for the possibility that someone who disagrees with them can have a reasonable point of view, a legitimate point of view. I think it's all about delegitimizing the opposition. I think that that makes a very hard dynamic for anything positive to happen.

This inspired me to make a post I've been meaning to add for some time. I've been feeling more and more in the past couple of months that the political climate in the US is becoming fatally ossified and polarized. Political discussion seems to be pretty much dead.

I don't own a TV, and I don't feel that either of the major parties comes comes close to representing me, so I haven't been following the election closely. I do however do some occasional reading at the Economist, the Atlantic, and elsewhere. What has gradually given me this creeping, skin crawling, feeling that political discourse has fallen into the abyss, is reading the comments which are ubiquitously attached to online articles. These pain me deeply.

I've been a long time user of the internet. Trolling and flaming on boards, forums, and news sites are nothing new to me. That is, I have a baseline. Yet, of late, I've found the comments on site like those I mentioned above to be particularly discouraging. A very high percentage of comments are incredibly hostile, loaded with vitriol and ad hominem attacks. It really doesn't depend on the article. Left or Right leaning articles tend to draw the predictable attacks from the opposing side (though it's disappointing that these are being phrased less and less in constructive and reasonable ways), but the really scary thing is that the articles that draw the most fire are those that take a neutral or centrist position. That is, the sides have become so polarized that any viewpoint that does not openly align with one camp or the other is fair game for vicious assaults from both.

The above panel helped me to think about why this is happening. The proliferation of the media, the creation of new cable channels and the new media, while adding depth and breadth to the discourse available to the average media consumer, has actually had a tendency to narrow their field of view. There are so many sources of information from any given perspective that a person can read many blogs and news sites, and watch several political TV shows while never actually exposing themselves any alternative viewpoints. While, a few decades ago, there may have only been a handful choices in terms of channels and papers, the very universality of these media sources required that they cater to both sides of the political divide. Today's media sources are so extensively diversified that we can each choose to steep in a carefully selected set of opinions that precisely support the very perspective that we have already chosen to adhere to.

My gut feeling is that this is leading us toward further calamity. I think that this trend toward polarization and the disappearance of sincere public discussion and debate has been underway for some time. I think our current circumstances (the Iraq war and the financial crises, the biggest political and economic disasters in recent memory) are the fruits of this trend. We are reaping disaster, and yet what we are sowing today is far more corrupted and virulent than the seeds that brought us our current sorrows.

I think this relates to what I have described in the past as the anthropomorphization of our problems. That is, we tend to blame the problems we see on the direct action and intention of other people. We implicitly believe that, if we can counteract the actions of these evil people, we can undo the problems of the world. Hence it makes sense to mobilize the base of our party at whatever cost. In our all out war we sacrifice the truth for political expediency. But if the problems are in fact much more complicated than we appreciate, the ends do not justify the means. In fact, the means do not lead us to our ends at all, but rather to folly and destruction.

Lessig on Technology Policy

Here, in just over 15 minutes, Larry Lessig says basically everything I've ever tried to say about telecom policy on the blog (in the context of a critique of John McCain's tech platform). Take a look, it's well worth the time:



Also, for a number of weeks I've been meaning to post a link to this ars technica article on customer-owned fiber. Projects like this, or like Utah's UTOPIA seem to present the best way forward for the domestic telecom market.

Policy Outcomes

It may not always be immediately obvious what the consequences of screwing up telecom policy will be, but here's an illustration. To be honest, I'm not sure that broadband penetration is a terribly meaningful statistic at this point. There are some outliers in difficult to reach locations who still have no access to broadband, but as a general matter, broadband in some form or another is available to those who want it, at least in those in the countries towards the top of these charts. The more meaningful statistic is the average throughput. Japan, Korea, and France have all passed the threshold where IPTV is practical. In fact, just based on those bandwidth numbers I ran a search for IPTV in France and came up with this. Broadband in the United States, by contrast, is five times slower than Korea or France, and ten times slower than Japan. The real kicker is that we pay more for that privilege than consumers in those other countries. We must be paying extra for our wonderful customer service... The other key point is that around the time of the dot.com boom, the United States was the clear global leader in Internet and broadband service. Japan and Korea, in particular, were late to the game and got to where they are now almost entirely as a result of aggressive regulatory policy. Our steady decline is similarly tied to our moronic telecom policies. Rambling on about the power of the free market (as an alternative to regulation) doesn't get you too far when you're dealing with non-competitive markets, and at some point that starts to show.

Resistance is Futile

Tim Wu on the state of the wireless market:

Who would have guessed that Apple—onetime victim of IBM and Microsoft—would today be an agent and symbol of industry consolidation? I don't know that it's fair to say this is Apple's fault. A telephone monopoly has been the norm for most of American telecommunication history, except for what may turn out to have been a brief experimental period from 1984 through 2012 or so. Like the short British experiment with republican government under Oliver Cromwell, it may be that telephone monopolies in America are a national tradition. In this larger story, the iPhone matters just as one of the last nails in the coffin of Bell's would-be competitors.

Brutal, but true. AT&T and Verizon cleaned house in the FCC's recent 700 MHz auction and appear to be scooping up spectrum and independent carriers faster than they can consume them. Sprint has been driven, in its desperation, into a last ditch joint effort with Clearwire, Comcast, Time Warner, Google, and Intel to offer an alternative data network -- a high stakes gamble that faces numerous business and technological obstacles. T-Mobile, the only other nationwide wireless carrier seems content to slowly sink into irrelevance. If the Sprint gambit fails, AT&T and Verizon will stand alone atop the wireless market. And even if it succeeds, the three viable national wireless networks will, by no coincidence, be under the control of the same four entities that have established total dominance over wireline voice, video, and data service in the United States: AT&T, Verizon, Comcast, and Time Warner. Either way, the mass extinction of competition in the American telecom marketplace is all but complete. To make matters worse, on the wireline side Comcast and Time Warner have few (if any) overlapping markets, and the same is true of AT&T and Verizon. To the extent that consumers prefer quadruple play offers (voice, video, data, and wireless), a proposition all the major carriers are betting heavily on, the vast majority of consumers will have only two options, if that. What has befallen the telecom market over the past 10 years is a disaster.

While we can only hope that help will be on the way once Democrats (hopefully) take over the White House and the FCC, it may be too little too late. To undo what has been done would require drastic measures. It would mean breaking up these big four, forcing some open access regime on them, or (best of all) forcing a separation of their physical network assets from their content and service delivery businesses. Any of these options would entail a major battle fought simultaneously on political, legal, and regulatory planes that, in all likelihood, would take the better part of a decade to play out. I doubt even an enlightened Democratic government would have the stomach for that. The only real alternative will be for the FCC to increasingly re-regulate these businesses. When the market cannot enforce discipline on providers of necessary services, regulators have to step in and assume the task. Net neutrality will only be the beginning...

The Battle for the Bottleneck

Since Peter and Henry each independently forwarded this Wall Street Journal article on cell phones to me, I thought I should respond here. First off, it is a truly excellent article, and I agree completely with Walter Mossberg, the author. He does a fine job laying out the key issues in an easy-to-understand manner. With respect to the upcoming wireless spectrum auction there, the odds of open handset rule going through went up this week when Verizon bailed on its attempt to sue the Commission over it.

Since Mossberg did such a good job discussing the state of the market, I thought I might discuss why I think the market is like this. The short version is this: while the Internet revolution has created a profusion of telecommunications that a short time ago would have been unimaginable, at the same time it has threatened to make telecom providers more irrelevant than they have been since their inception and so they fight it tooth and nail. And now for the (really) long version:

The problem with the Internet viewed from the perspective of the telecoms is that it is an end-to-end network, meaning everything interesting happens at the endpoints and they're stuck in the middle. The stuff in the middle is designed to be transparent and fungible (particularly in the case that some of it is vaporized by a nuclear bomb). This is very different from what we had before the Internet. Previously any goods or services traveling down the phone line into someone's home or office were controlled by the phone company, be it phone service, long distance service, voice mail, video, advanced data services, etc. Even if you just wanted to communicate data between you two office sites, the phone company would set this up then charge you out the wazoo. It was a good business, and they made a profit for each additional application they could develop and provide to consumers over their lines.

But when the Internet comes along, even though the number of applications now skyrockets and the benefits people gain from their telecommunications links grow immeasurably, the telecom companies are no longer in control, and are therefore no longer profiting directly from the provision of services. Whatever the telecom companies were charging people an arm and a leg for can now be done on the Internet for peanuts. So to the extent that telecom providers did provide unique services, now anyone who can pipe in an Internet connection over any medium can eat their lunch. You don't have to rely on the phone company for voice communications or cable for video anymore. And many applications the phone companies hadn't implemented yet or hadn't even dreamed of are now out there being heavily used for free. It seems like it should be their shining hour, but it's really more of a let-down. Sure they are still selling and making money on the basic connectivity that underlies all of the Internet, and in record volume, but they'll be selling that in any case. It's the add-ons, the profits coming from provision of actual commercial services to end-users they want. So while openness is what has made the Internet great for us users, there has been a very consistent effort on the part of telecom companies to maintain control where they haven't yet lost it, or to regain control where it has been lost.

At the start of this thing the big telecom companies didn't even want to deal with home consumers; for them data services were things to be sold to corporate customers at outrageous prices that no home user would contemplate paying. So when PC ownership took off and it became apparent that there was a market for online services, other companies got involved. The initial generations of modems were built to work over the phone lines by converting their digital signals into analog beeping and squawking (converted back to a digital signal by the modem at the other end of the line) so that they only needed a normal voice line to operate. Phone company participation was not required. The initial big corporate providers of dial-up online services (Microsoft, America Online, Prodigy) ignored the Internet and each built their own walled garden, each providing their own proprietary services to customers. In this sense they adopted the same model as the telecoms. Everything that passed down that line was to be provided by them, and they'd get paid for it. Of course, the fact that dial-up modems require no significant physical infrastructure (all you need is an office with a bunch of voice lines and some computers) meant that the barrier to entry to the online services business was very low. So small shops opened all over the place. They, of course, didn't have the infrastructure to create a bunch of proprietary content and services the way the big guys did, but there was this Internet thing they could hook people up to for pretty low costs that had a lot of content on it. It didn't take consumers long to figure out that there was a whole hell of a lot more interesting stuff outside the walled gardens than inside them (in large part because the users went out and created it themselves). So the walled garden model was soundly defeated (although it took AOL the better part of a decade to admit it).

So we all got onto the Internet and the phone companies had lost control over content and services and so had the ISP's. Everything was free and open and we went crazy with entrepreneurship and had the dot-com bubble. But while that was happening, we were also developing technologically. The networks were trying to keep up with our demand for faster connections to get all the cool new stuff on the Internet (which is to say mostly online gaming, pirated MP3's, and porn). So we moved from dial-up to DSL and cable modems. This is an important transition, because, unlike dial-up modems, DSL and cable modems require physical infrastructure in the telecom networks. There needs to be actual equipment sitting in the local office of the phone or cable company for them to work. All of the sudden the phone companies had a way to take back control over everything. But wait, luckily for us Congress was out ahead of the game and had made arrangements for this in the 1996 Telecom Act. That sounds sarcastic, but isn't; Congress had achieved a remarkably balanced and well-designed compromise. The Act required, among other things, that the regional phone companies play nice with competitors and rent out the equipment needed to make a DSL network function, so that competitors could get at the same customers the regional phone companies served. Problem solved, competitive pressure will prevent the phone companies from reestablishing control over the Internet and abusing their market power.

Sadly it didn't work out that way. The phone companies didn't like this part of the '96 Act (big shock), so they launched an intense campaign of lobbying, foot-dragging, and lawsuits that lasted until a few years ago when Bush's FCC appointees decided to use the Telecom Act for toilet paper and the Supreme Court cheered them on. Internet access is no longer considered a common carrier service and is not subject to open access requirements. Now instead of thousands of ISP's in direct competition with each other, Verizon and AT&T and Comcast and Time Warner basically own the consumer side of the Internet in the US. And, to varying degrees, they may be able to start rebuilding their walled gardens and capitalizing off of all the wonderful services people enjoy on the Internet, and turning our wonderful network into something a little less wonderful. This is what the fight over net neutrality is about.

One of the interesting battles in net neutrality is just how far it reaches into a provider's infrastructure. The case for net neutrality is most clear when it comes of an ISP's actual Internet service, where they're blocking a service or web site they don't want you to access. But there are other ways for an ISP to go about this. Consider cable. The cable companies set up their networks so that only a small part of their pipe is allocated for Internet service. The vast majority of it is carrying video. If they wanted they could scrap a few cable TV channels and expand the datastream to their cable modem subscribers. Now let's say that on-demand TV becomes a very popular service. Comcast has two ways to go about this. They could provide on-demand video pretty easily over the Internet, except they probably don't have enough bandwidth on their Internet service to handle it. So they would need to eliminate some video channels and expand their bandwidth first. But if they did that, anyone could sell on-demand video to their customers over the Internet, and Comcast would get nothing for it. They would have to compete for every customer, and the wouldn't have any real competitive advantage over other video providers. Alternatively, Comcast could keep their Internet service throttled down, and develop a proprietary on-Demand system that would occupy the same bandwidth but run independently of their Internet service. Now they have no competitors and get to tariff users for every use. Oh, happy day for Comcast.

All of this brings us, finally, to the mobile wireless market. There are three key points to notice. First, for technological reasons data services took quite some time to arrive in the mobile wireless market. So they got to sit by and watch all this other stuff happen. Second, like DSL and cable, wireless Internet service requires special equipment, meaning the wireless service provider is necessarily also the Internet service provider and has total control over the network. Third, the cast of the characters is fairly similar (AT&T and Verizon, again, plus a couple other big national companies). So what you have are companies that have learned that what they want, more than anything, is control and the ability to tariff users for every bloody thing they do on the network, and who technically have the capability in this market to make it happen. They have fought bitterly to maintain this control and will continue to. These are concentrated markets, so the threat of competition isn't that great, and clearly the fear of loss of control their users is much greater than their fear of competitive harm. And the big wireless companies are all moving in lock-step on this anyway, so there is no real competitive impetus to stop fucking with customers and actually give us what we want. This is the sort of thing that happens in concentrated markets…

On some level they know they'll eventually have to let their users onto the Internet for real (the iPhone is a concession to that reality, although AT&T saw to it that they get their pound of flesh from iPhone users anyway). But the longer they can keep customers off the Internet, or only let them on some borked up version of the Internet, the better. I mean if people can download ring-tones for free, who's going to pay their provider $3 a pop for them? If you can buy MP3's for your phone from iTunes for a buck, why would you pay Verizon twice as much for their proprietary music-to-phone system? Now they can sell you GPS-based mapping, sports scores, online games, SMS, emails with pictures from the camera phone, and every other damned thing you'd get for free on the Internet. If they provided good Internet service or allowed phones with WiFi capability onto the network, this would all come crashing down.

In the end, the simple reality is that the communications market is deeply dysfunctional. It's highly concentrated, has many components that are or were natural monopolies, and is littered with regulatory detritus from past battles with abusive monopolists (too many of which didn't come out the right way for the good guys). These are markets that desperately need oversight and consumer protections. Frankly, a lot of the physical infrastructure would probably be better off in the public sector. The Internet is an incredible economic engine and allows for so much innovation and competition, it's sad to let these crotchety bastards be the gatekeepers. The analogy to superhighways isn't that far off. It doesn't have to be like this. In other countries (generally where regulators have forced open access on the phone companies) people regularly pay about the same we do for Internet service that is ten times faster, they have mobile phones that can do WiFi and are interchangeable among providers, and they have no real issue over net neutrality because Internet providers don't have enough market power to abuse it. I'll be very curious to see what happens if the Democrats take over at the FCC in a couple years...

Another Round of Net Neutrality

The Bush administration has fired its latest salvo on net neutrality, with an in-depth report by the FTC (actual report here). Their conclusion: let's wait and see. It's a 170 page report and covers a lot of ground. I can hardly respond to everything in it, but I would like to make a few comments.

First, the FTC does a solid job of covering the arguments in favor of net neutrality (pages 56-64). They didn't make any effort to soft-peddle the substantive arguments or build the strawman positions many net neutrality critics are so fond of. Most significantly they covered the concern that prioritization of ISP-provided content and other specially licensed preferred content would tend to recreate the walled gardens of the pre-Internet America Online and Prodigy days. They discussed the problem that lack of competition in the last mile makes the market an unreliable regulator of ISP behavior. Also they noted the argument that increasing bandwidth (say to 100 Mb/s) could make the issue of congestion largely disappear (and concerns over net neutrality with it).

The report also summarizes arguments against net neutraity (pages 64-69). Most of them are, I think, pretty weak. They raise the usual point about the necessity of non-neutrality to deal with small numbers of users sucking down large amounts of network resources for filesharing and the like. This really has little to do with net neutrality. Net neutrality is about ISP's throttling the content end of the connection, not the user end. Even under most proposed neutrality regimes ISP's would be free to throttle users (within the terms of their service agreements) when users abuse the network. They also raise the point that different types of data (web pages versus VOIP versus streaming video) may need different service levels. This again, as I pointed out in my last net neutrality post, is not prohibited by net neutrality. It's ok to give VOIP traffic higher priority than web traffic, as long as no particular VOIP provider is preferenced over another.

The net neutrality opponents argue that net neutrality is necessary to allow service providers to capture enough revenue to fund the buildout of faster networks needed for advanced services. This seems facially plausible, but breaks down on further analysis. First, if what the ISP's want is a metering system, so that the users of advanced services bear more of the cost, there is nothing to stop them from simply offering those capabilities directly to end users for a price. Already some providers offer multiple tiers of internet service. Additionally, the net neutrality model doesn't appear to be one well suited to driving growth in network capacity. Non-neutrality revenue is built on bandwidth scarcity. If bandwidth isn't scarce, no content provider pays an ISP for preferred service. I find it hard to believe that if ISP's start to have significant new revenue streams coming in predicated on the scarcity of their network resources, they would turn around and invest that income in expanding their network resources and thereby undercut their new revenue model. In a non-neutral market it pays to under-supply bandwidth.

Finally, perhaps the most critical topic of discussion is the state of competition in broadband internet service (starts on page 98). If internet service was truly a competitive market, there would be no need for net neutrality regulation, as consumer demand could sufficiently regulate the market to prevent abuse by ISP's.

The report starts off by noting how much broadband prices have fallen and how much speed has increased in the last six or seven years. Certainly it's true that this has happened, but it needs to be put into perspective. This is, of course, a business built on technology that is improving at remarkable rates. Look at how much PC's have advanced in the past decade, how much faster the processors are, how much more RAM they have, how much more harddrive capacity, how much better video processing, and how much prices have fallen. By comparison, the rate of change in broadband internet service is glacial. Or you could compare cost and speed improvements in broadband service in the US with broadband services in Japan or Korea or France, where, on a price/performance basis, we've been left in the dust. Yes, the market is improving, but it would be an appalling failure if it didn't, and the rate at which it is improving does not bode well for the state of competition in the broadband market.

The other key point on competition is the matter of new entrants to cover the "last mile". Opponents of net neutrality argue that the cable/phone company duopoly is being challenge by other service providers using satellite, broadband over powerlines (BPL), or wireless systems. Each of these technologies, unfortunately, faces crucial barriers. Satellite is cursed with poor latency. As fast as the speed of light is, bouncing a signal off a satellite down to a receiving station, then back up to the satellite and back down to the user takes time, enough time to make satellite unusable for most interactive advanced services. BPL simply doesn't appear to be making the cut in terms of cost of deployment and performance level to entice many power companies to want to try rolling it out. It simply isn't going to have the cost/performance ratio to make it competitive with cable or DSL any time in the near future (if ever). For wireless the challenge is spectrum. Wireless will be highly competitive in the market for internet service at speeds of 10 Mb/s or less. But it does not appear that there is enough spectrum for widespread use at speeds much greater than that. As people become more accustomed to high speed cable/DSL/fiber service they will find wireless insufficient for standard residential use (and for most uses that implicate net neutrality concerns). Cable and DSL can provide speeds in the 30 Mb/s range (although no US providers offer this at present), and fiber will go up to 100 Mb/s. I'm afraid we are going to be stuck with a duopoly for some time. The market is not going to bail us out of this mess (although it could if federal regulators forced the cable and phone companies to open up their last mile networks to other competitors).

Blocking Innovation

As Vonage and Verizon having been battling in court since early this year over Verizon's claims of having patented some faily obvious elements of VoIP used by Vonage, software patents have again fallen under scrutiny. Tim Berners Lee, the inventor of the World Wide Web, has an OpEd in the New York Times arguing against software patents. He quotes Bill Gates writing in 1991 that if early computer pioneers had understood how to patent their works "the industry would be at a standstill today."

Software has always seemed poorly suited to patents for a number of reasons. Most of what software does is fairly obvious. The really innovative stuff tends to fall into either broad conceptual ideas (like the web) that would be inappropriate for patents or clever algorithms (like quicksort) that are basically mathematical discoveries, abstract intellectual concepts of the sort not traditionally patentable. The rest of it, however complex and difficult, tends to be fairly mundane implementation, requiring little originality (but often a huge effort in coordination).

Existing patent law, in theory, should prevent unoriginal things from being patented. But in reality it does not seem to do a good job of this, particularly in the initial granting of patents. And even if questionable patents can still be challenged and overturned in court, this is a remedy of limited value to small developers and open source projects. Going up against Microsoft or IBM's legal team is often an insurmountable barrier to entry. And both patent examiners and judges appear to struggle with the concepts of computer science in their efforts to determine what is or is not an obvious development. As the OpEd points out, two of Verizon's patents in this case cover the painfully obvious concept of converting between phone numbers and internet addresses.

From the economic analysis side as well, software development is ill-suited to patents. For one, there is, as the OpEd discusses, alternative protection available for software developers in the form of copyrights. Any direct copying already gives rise to a legal claim. And copyright is, as the article notes, considerably less cumbersome and expensive. Also crucial, copyright does not create liability for independent invention, while patent does. So if Verizon has valid patents on basic elements of VoIP they can bar anyone else from offering a VoIP service until those patents expire. With a copyright they could only prevent people from actually copying their system, but they would have no claim against a competitor who developed their own, even if it worked the same way.

It is important to recall the purpose of intellectual property protections: to allow inventors and developers to profit from their investments in new technologies and innovations, thereby promoting more such investments. The need for strong protection is much lower in the software space than for, say, pharmaceuticals. Development costs are relatively low and innovation occurs rapidly enough in software that there is a significant first-mover advantage even without patent protections. As long as competitors cannot rip off a new program wholesale (which copyright prevents) and have to independently develop their own version, the first innovator will be rewarded for her efforts. Look at how YouTube still dominates over other video-sharing systems. Some of its competitors are offering technically superior systems, but YouTube got there first. The software market, before these big companies realized they could patent everything under the sun, was wildly innovative and fiercely competitive. They didn't need patents to achieve that. Patents impose signicant transactional costs and barriers to entry without adding any meaningful beneficial incentives to the software business. This system is a mess, and I hope the Verizon case will help to spark reform.

Al Gore and TV Culture

I was deeply impressed with this excerpt from Al Gore's new book The Assault On Reason from Time Magazine. It's heavily Amusing Ourselves to Death influenced. It makes a half-hearted diversion into being an advocacy paper for net neutrality in the closing paragraphs, but that may just be an artifact of reformatting the book into excerpt form, because prior to that turn the excerpt appeared to have a rather grander design. I'll be curious to see reviews of the whole book. I think Gore diagnosed the problem pretty well here, and I surely hope net neutrality is not the entirety of his answer. I have to concur with Andrew Sullivan, I much prefer the new public intellectual version of Al Gore to the old political candidate version. I hope he doesn't let himself be dragged into the 2008 race.

Opening Up Cellular Networks

Some time back we had a discussion here on the insistence of US cellular providers that wifi features on new phones be disabled before the phones were marketed in the US. In a column for Forbes, Tim Wu argues that the FCC should use the upcoming auction of wireless spectrum gained from the digital TV transition to enforce a right to attach rule. He notes that the exact same question was broached with respect to landline phone service in 1968, where the FCC ruled that AT&T must allow the Carterphone to interoperate with its system (a ruling the paved the way for the creation of MCI and the ultimate demise of AT&T's monopoly). The rule would force carriers who purchased spectrum in the auction to allow any device that conforms to basic specs to operate on their network, knocking down an immense barrier to entry for wireless device manufacturers. This is a good example of a pro-market regulation, carving out space for innovation and competition in an anticompetitive marketplace.

Losing the Independent Press and the Price We Pay

Bill Moyers made waves this week with his special report on the role of the press in the run up to the war in Iraq. His story dovetails nicely with PBS Frontline's ongoing series of reports on the state of the press. Both are available online, and well worth watching. I would start with with Part III of the Frontline series, followed by Moyers's Buying the War.

Moyers has few true revelations and little new information, but it is well worth going back and seeing how badly the press got this war wrong, and revisiting some of the more egregious false claims of the pundits. Those pundits, as Moyers points out, are alive and well in American journalism. Many have been promoted. No one has been called to account.

The Frontline report helps to illustrate why American journalism is failing. No one does real reporting anymore, save a few national newspapers, and even they are cutting back. The newspapers are still highly profitable, but their readership and their revenue is in steady decline, and so sacrifices must be made for them to maintain their high margins and keep the corporate overlords happy. This was very much my concern while researching the media ownership rules last summer. Maintaining the prohibition on TV-newspaper crossownership has long been a central goal of the Democrats with respect to the media ownership rules. While I certainly see the value in that policy, for me the financial well-being of our newspapers has to be the top concern. I'm frankly not sure whether they would be helped or hindered by relaxing the cross-ownership rule. But I think we have to be willing to try anything that might preserve our last bastions of actual journalism.

Of course, the Moyers piece illustrates that even when newspapers have the resources to perform serious research they can fail to use them when caught up in the political conventional wisdom (as was the case of the New York Times and the Washington Post with Iraq). The conventional wisdom is created in part by the efforts of political actors (the White House in particular), but also by the media itself when it parrots the lines fed to them by those political actors. When real journalism is a small slice of the news media universe it seems that it is easily cowed by the overwhelming majority of the press for whom statements issued by the White House define reality. There needs to be a critical mass of independent press for news outlets to gain the courage of numbers to be willing to take risks.

Neither of these two pieces paints the whole picture on its own, the business of the news and the politics of the press, but together I think they give us a good look at what has gone wrong. The newspapers have a long way to go before they can recoup their lost hard-copy profits through their online presence. Given how much the internet has shaken up other industries, it's not clear they ever will. If they don't we can look forward to further setbacks to the quality of journalism in the US. We need to start to consider the degree to which quality reporting is a public good. In my opinion, in a democratically governed nation, journalism is the highest order of public good. The public cannot make informed choices if there is no one to inform them. If the market is unable to provide what we need, we will have to explore other avenues.

Big Lobbying Money + Complicated Subject + Stupid Tech Writer = Gigantic Mess

Subtitle: Robert X Cringely is a bloody idiot.

The job of a tech writer is to explain the universe of high technology to laymen in terms they can understand. To clarify confusing topics, to illuminate, to inform. So why does PBS's Robert X Cringely insist on writing on topics about which he so obviously does not have even a wisp of understanding? This week he writes about net neutrality, a topic inspired, apparently, by his suspicion that his broadband provider may be messing with his faxes over his VOIP service. He goes on to discuss how ISP's already implement quality of service prioritization (QoS), which he refers to as tiered service, which means that we have, in fact, never had network neutrality. It also, he says, means the internet is not a "best efforts" network as all the pundits say it is. All of this is, of course, wrong. That's not unusual for Cringely. What irritates me is that this is an important issue, and the wrong characterization that Cringely just published is exactly the line of bullshit that the lobbyists for incumbent networks are trying to sell. This makes Joe unhappy. So I'm going to take time out of my day to do Cringely's job for him.

It's easiest first to define net neutrality by what it is not:

Net neutrality is not an effort to prohibit QoS prioritization or traffic shaping. If you want to (as Cringely says in the article) give top priority to VOIP, then routing tables, then commercial broadband service, then consumer broadband service, there is absolutely no problem with that. Knock yourself out. If you want to dial down bandwidth on popular filesharing ports because the bittorrent users are clogging your network, again no problem. This is not what net neutrality is about (with one caveat I'll get to below). (And in case anyone cares, traffic prioritization is not, as Cringely believes, mutually exclusive with being a "best efforts" network (which the internet by and large is).)

Net neutrality also does not prohibit tiered service offers. If you want to give you end users a choice between a $20/mo 2mb service, a $40/mo 10mb service, and an $80 100mb service, there's no problem with that.

And here is what net neutrality is:

Net neutrality prohibits discrimination based on source. All data of the same type gets the same treatment regardless of who it comes from. So you can discriminate as much as want between different types of data, as long all packets within each class get the same treatment. The point of this is that a service provider is not allowed to favor its own websites, email, VOIP system, streaming video service, etc. over a competitor's. If you want to give your VOIP system high priority service, then Vonage gets that service too. Similarly, service providers not allowed to preference third-party services who pay them a fee. So charging Vonage for special priority carriage then turfing other VOIP providers isn't allowed either. That's net neutrality.

Where this gets tricky is where a service provider doesn't offer VOIP or streaming video over the internet, but offers an analogous non-internet-based service over the same line (phone service for DSL, cable TV for cable modems). In that case, prioritization by type acts as a proxy for prioritization by source. If you throttle down all VOIP service equally, that still gives preference to your non-internet phone service. To that extent, and only to that extent, net neutrality may have something to say about type-based prioritization.

The cable and phone incumbents have spent boatloads of lobbying money to promote the misconceptions about net neutrality that Cringely put on display in his column. Traffic shaping based on packet type is highly useful, and generally accepted as a desirably practice. Likewise tiered service offers to end users serves an important purpose in the market. If the incumbents can get everyone to believe that these things would be trashed by net neutrality, it makes their job much easier in arguing against it. This lobbying effort has been quite effective, and I see these misconceptions all over the place. It always bothers me, but much more so when I see it from a perceived authority figure like Cringely. Tech writers have a duty to clarify this issue, not to muddy it further.

And, by the way, Cringely, the reason your fax doesn't work over VOIP has nothing to do with net neutrality. VOIP uses lossy compression tuned to work with human voices, not the blips, bleeps, and squawks of a fax modem. Your fax machine doesn't work over VOIP because the fax transmission is being garbled by that compression. Buy yourself a scanner and toss the fax machine, it's obsolete.