mark nottingham

Why Intermediation is Important

Saturday, 27 December 2014

HTTP Standards

A few months ago I went to the Internet Governance Forum, looking to understand more about the IGF and its attendees. One of the things I learned there was a different definition of “intermediary” — one that I think the standards community should pay close attention to.

In Internet protocols, an intermediary is a device that sits between two endpoints and somehow mediates communication between them; in HTTP, this takes the form of “proxies” and “gateways,” and more generally they’re called “middleboxes.” Intermediaries have a long history on the Internet; they’re used to route traffic, optimise it (e.g., caching), make it more reliable, and provide extra services to users. While this kind of intermediation deserves attention, a more broad definition is roughly any party that’s able to affect your communication. In this view, every intermediary is a potential point of control. Note “potential” there — intermediaries have a variety of uses (both purposeful and accidental), and there’s a huge difference between the qualities of a privacy-enhancing proxy and a government-mandated content firewall. So, not only is an “on-path” HTTP proxy an intermediary, but so is your Web site’s DNS registrar — because they can be used to disable access to it (and have been, sometimes with disastrous results). DNS itself is often used as a “policy enforcement point” ( again, with not-so-great results).

Likewise, TLS certificates can be revoked — making a site’s Certificate Authority yet another intermediary (especially if HSTS is in use).

An ISP is an intermediary by this definition because they can cut off net access altogether, or they can filter it, or shape it. In a robust market, ISP-as-intermediary is weak — it’s easy to circumvent the intermediation by using another ISP. It’s not so weak if there’s not much of a market (bringing about “net neutrality” concerns). And, in some cases, you don’t have a choice; e.g., the airport wifi is often the only way to get onto the net when you’re travelling.

Even a Web host can be considered an intermediary; while you can get your data in and out of a WordPress site or Google Docs, there is a certain amount of friction to doing so. If that friction is too great, they have effective control over your communication.

Of course, this cuts both ways; while as a user, you might view intermediation as control, from their perspective, it often forms a kind of liability; for example, if your content breaks the law, a government (yours or another) might lean on your Web host, your DNS registrar, an ISP or someone else to deal with it. Article 19 has an in-depth policy paper that covers the related issues.

How Intermediaries Hurt

As the Internet becomes more attached to our worlds, intermediaries are becoming a primary focus for implementation of policy and control of information online. Putting aside the (huge) issue of whether allowing control over communication is desirable, enforcing policy in protocols that don’t anticipate this use can cause significant security and interoperability issues.

Indeed, we’re already seeing institutional reaction to overreach by some kinds of abusive intermediaries; for example, IAB member Joe Hildebrand’s excellent “Erosion of the moral authority of transparent middle boxes” explores how some network operators have lost the high ground by taking too many liberties with the bits flowing through them:

Some middlebox capabilities are currently implemented using the same mechanisms employed by attackers, including passive capturing of plaintext data, active impersonation, and denial of service. Further, some services are legitimate in one context but illegitimate in another - and the transparent nature of the middleboxes creates security problems separating those problem domains.

This leads to efforts to stop both unwanted attacks and unwelcome intermediation through technical measures like encryption, as called for by the IAB and currently under consideration by the TAG.

Similarly, as off-path intermediation points are used in unanticipated ways, I believe we’re going to see unwanted side effects create pressure to work around them as well. We would do well to consider this in how we create and extend Internet protocols, so as to avoid the worst effects.

More Thoughtful Intermediation

Fundamentally, intermediation introduces a new party to communication. While this is sometimes necessary, it always needs to be done with careful consideration for the original parties to the communication and their inherent rights — something I’d like to see formalised in the IETF.

Specifically, all other things equal, introducing new points of intermediation in protocols should be avoided; there are already more than enough available to impose whatever policy a society might want to, and each new kind of intermediary brings both security and interoperability concerns, because intermediaries unavoidably make a system more complex. When there is a genuine need to introduce a new party to communication, the potential intermediary should be carefully vetted to avoid misaligning its interests with that of the other parties, and it should be technically easy to replace it with another, or remove it.

For example, the newest potential intermediary in the Web stack is Certificate Transparency. As I understand it, it’s a fairly weak one; a third party would need to have influence over all of the logs that the browser is using to deny access. I’d be more comfortable, though, if CT explicitly explored this issue (perhaps in its Security Considerations).

Of course, some intermediaries develop naturally on “top” of the Internet; for example, Google Docs, Dropbox and other services can be seen as intermediaries by this definition. In these cases, it may be appropriate to reduce the friction of moving content between services through standardisation; right now all we’ve got is WebDAV, and that’s pretty poor.

In some places, intermediation is unavoidable, but we need better ways to explicitly indicate that the intermediary is being used as a point of control. In this way, I find Tim Bray’s 451 proposal very interesting, and discussion about it has picked up again recently.

Of course, one of the biggest forms of intermediation is well-represented by the ICANN contingent at IGF; DNS registration itself implies central coordination and therefore a point of control. In this area, DNSChain is the most radial dis-intermediation yet; while it’s not yet ready for broad use, it is interesting to speculate about what impact it could have.