mnot’s blog

Design depends largely on constraints.” — Charles Eames


Thursday, 5 December 2019

On RFC8674, the safe preference for HTTP

It’s become common for Web sites – particularly those that host third-party or user-generated content – to make a “safe” mode available, where content that might be objectionable is hidden. For example, a parent who wants to steer their child away from the rougher corners of the Internet might go to their search engine and put it in “safe” mode.

this entry’s page

Sunday, 13 October 2019

How Multiplexing Changes Your HTTP APIs

When I first learned about SPDY, I was excited about it for a number of reasons, but near the top of the list was its potential impact on APIs that use HTTP.

this entry’s page

Tuesday, 27 November 2018

Designing Headers for HTTP Compression

One of the concerns that often comes up when someone creates a new HTTP header is how much “bloat” it will add on the network. This is especially relevant in requests, when a little bit of extra data can introduce a lot of latency when repeated on every request.

this entry’s page

Wednesday, 7 June 2017

How (Not) to Control Your CDN

In February, Omer Gil described the Web Cache Deception Attack.

this entry’s page

Thursday, 11 May 2017

How to Think About HTTP Status Codes

There’s more than a little confusion and angst out there about HTTP status codes. I’ve received more than a few e-mails (and IMs, and DMs) over the years from stressed-out developers (once at 2am, their time!) asking something like this:

this entry’s page

Thursday, 16 March 2017

The State of Browser Caching, Revisited

A long, long time ago, I wrote some tests using XmlHttpRequest to figure out how well browser caches behaved, and wrote up the results.

this entry’s page

Friday, 22 April 2016

Ideal HTTP Performance

The implicit goal for Web performance is to reduce end-user perceived latency; to get the page in front of the user and interactive as soon as possible.

this entry’s page

Wednesday, 9 March 2016

Alternative Services

The IESG has approved “HTTP Alternative Services” for publication as a Proposed Standard.

this entry’s page

Friday, 18 December 2015

Why 451?

Today, the IESG approved publication of “An HTTP Status Code to Report Legal Obstacles”. It’ll be an RFC after some work by the RFC Editor and a few more process bits, but effectively you can start using it now.

this entry’s page

Tuesday, 18 August 2015

Will there be a Distributed HTTP?

One of the things that came up at the HTTP Workshop was “distributed HTTP” — i.e., moving the Web from a client/server model to a more distributed one. This week, Brewster Khale (of fame) talked about similar thoughts on his blog and at CCC. If you haven’t seen that yet, I’d highly suggest watching the latter.

this entry’s page

Monday, 15 June 2015

HTTP/2 Implementation Status

RFC7540 has been out for about a month, so it seems like a good time for a snapshot of where HTTP/2 implementation is at.

this entry’s page

Wednesday, 18 February 2015

HTTP/2 is Done

The IESG has formally approved the HTTP/2 and HPACK specifications, and they’re on their way to the RFC Editor, where they’ll soon be assigned RFC numbers, go through some editorial processes, and be published.

this entry’s page

Saturday, 27 December 2014

Why Intermediation is Important

A few months ago I went to the Internet Governance Forum, looking to understand more about the IGF and its attendees. One of the things I learned there was a different definition of “intermediary” — one that I think the standards community should pay close attention to.

this entry’s page

Saturday, 7 June 2014

RFC2616 is Dead

Don’t use RFC2616. Delete it from your hard drives, bookmarks, and burn (or responsibly recycle) any copies that are printed out.

this entry’s page

Sunday, 1 June 2014

Chrome and Stale-While-Revalidate

Chrome is looking at adding support for RFC5861’s stale-while-revalidate, which is really cool. I wrote about the details of SwR when it first became an RFC, but its application to browsers is something that’s a new. Seems like a good time to answer a few potential questions.

this entry’s page

Friday, 9 May 2014

If You Can Read This, You're SNIing

When TLS was defined, it didn’t allow more than one hostname to be available on a single IP address / port pair, leading to “virtual hosting” issues; each Web site (for example) now requires a dedicated IP address.

this entry’s page

Monday, 17 March 2014

Trying out TLS for HTTP:// URLs

The IETF now considers “pervasive monitoring” to be an attack. As Snowden points out, one of the more effective ways to combat it is to use encryption everywhere you can, and “opportunistic encryption” keeps on coming up as one way to help that.

this entry’s page

Thursday, 30 January 2014

Nine Things to Expect from HTTP/2

HTTP/2 is getting close to being real, with lots of discussions and more implementations popping up every week. What does a new version of the Web’s protocol mean for you? Here are some early answers:

this entry’s page

Saturday, 4 January 2014

Strengthening HTTP: A Personal View

Recently, one of the hottest topics in the Internet protocol community has been whether the newest version of the Web’s protocol, HTTP/2, will require, encourage or indeed say anything about the use of encryption in response to the pervasive monitoring attacks revealed to the world by Edward Snowden.

this entry’s page

Sunday, 23 June 2013

Five Reasons to Considering Linking in Your HTTP APIs

There’s been a lot of interest in and effort expended upon “hypermedia APIs” recently. However, I see a fair amount of resistance to it from developers and ops folks, because the pragmatic benefits aren’t often clear. This is as it should be, IMO; if you’re not able to describe concrete benefits without hand-waving about the “massive scale of the Web.”

this entry’s page

Wednesday, 15 May 2013

Indicating Problems in HTTP APIs

A common part of HTTP-based APIs is telling the client that something has gone wrong. Most APIs do this in some fashion, whether they call it a “Fault” (very SOAP-y), “Error” or whatever.

this entry’s page

Friday, 4 January 2013

Exploring Header Compression in HTTP/2.0

One of the major mechanisms proposed by SPDY for use in HTTP/2.0 is header compression. This is motivated by a number of things, but heavy in the mix is the combination of having more and more requests in a page, and the increasing use of mobile, where every packet is, well, precious. Compressing headers (separately from message bodies) both reduces the overhead of additional requests and of introducing new headers. To illustrate this, Patrick put together a synthetic test that showed that a set of 83 requests for assets on a page (very common these days) could be compressed down to just one round trip – a huge win (especially for mobile). You can also see the potential wins in the illustration that I used in my Velocity Europe talk.

this entry’s page

Tuesday, 18 December 2012

"Why Don't You Just…"

A proposal by John Graham-Cumming is currently doing the rounds:

this entry’s page

Friday, 7 December 2012

HTTP Status: 101 Switching Protocols

The HTTPbis Working Group met in Atlanta last month; here’s how things are going.

this entry’s page

Tuesday, 4 December 2012

Evolving HTTP APIs

One of the most vexing problems that still seems to be facing people when I talk to them about HTTP APIs is how to handle versioning and extensibility – i.e., how they evolve.

this entry’s page

Monday, 29 October 2012

OPTIONS is Not the Method You're Looking For

Once in a while, people ask me whether they should use the OPTIONS HTTP method, and whether we should try to define formats for discovering resource capabilities with it.

this entry’s page

Monday, 24 September 2012

Caching POST

One of the changes in Apple’s release of iOS6 last week was a surprising new ability to cache POST responses.

this entry’s page

Wednesday, 5 September 2012

Why PATCH is Good for Your HTTP API

A common problem for APIs is partial update; when the client wants to change just one part of a resource’s state. For example, imagine that you’ve got a JSON representation of your widget resource that looks like:

this entry’s page

Saturday, 4 August 2012

HTTP in Vancouver

The HTTPBIS Working Group is in a transitional phase; we’re rapidly finishing our revision of the HTTP/1.1 specification and just getting steam up on our next target, HTTP/2.0.

this entry’s page

Wednesday, 11 July 2012

Bad HTTP API Smells: Version Headers

One thing I didn’t cover in my previous rant on HTTP API versioning is an anti-pattern that I’m seeing a disturbing number of APIs adopt; using a HTTP header to indicate the overall version of the API in use. Examples include CIMI, CDMI, GData and I’m sure many more.

this entry’s page

Monday, 25 June 2012

HTTP API Complexity

@dret: if your scenario is homogeneous and models are harmonized across participants, #REST is of limited utility for you.

this entry’s page

Saturday, 31 March 2012

What's Next for HTTP

We had two great meetings of the HTTPbis Working Group in Paris this week — one to start wrapping up our work on HTTP/1.1, and another to launch some exciting new work on HTTP/2.0.

this entry’s page

Tuesday, 25 October 2011

Web API Versioning Smackdown

A lot of bits have been used over on the OpenStack list recently about versioning the HTTP APIs they provide.

this entry’s page

Friday, 21 October 2011

Why ESI is Still Important, and How to Make it Better

More than ten years ago, I was working at Akamai and got involved in the specification of Edge Side Includes (ESI), sort of a templating language for intermediaries.

this entry’s page

Friday, 2 September 2011

RFC6266 and Content-Disposition

HTTPbis published RFC6266 a little while ago, but the work isn’t finished.

this entry’s page

Sunday, 28 August 2011

Better Browser Caching

In discussing my whinge about AppCache offline with a few browser vendory folks, I ending up writing down my longstanding wishlist for making browser caches better. Without further ado, a bunch of blue-sky ideas;

this entry’s page

Wednesday, 24 August 2011

Distributed Hungarian Notation doesn't Work

It used to be that when you registered a media type, a URI scheme, a HTTP header or another protocol element on the Internet, it was an opaque string that was a unique identifier, nothing more.

this entry’s page

Friday, 5 August 2011

HTTP Pipelining Today

Last week, highlighted how mobile browsers use HTTP pipelining.

this entry’s page

Wednesday, 27 July 2011


FYI, I’ve implemented Content Security Policy on this site. If your’e a Mozilla user, please tell me if you have any problems.

this entry’s page

Monday, 11 July 2011

What Proxies Must Do

The explosion of HTTP implementations isn’t just in clients and servers. An oft-overlooked but important part of the Web ecosystem is the intermediary, often called just a “proxy”*.

this entry’s page

Sunday, 19 June 2011

Fixing AppCache

HTML5’s AppCache mechanism is one confused little puppy. Purporting to be for taking web applications offline — a compelling and useful thing — it’s more often used by performance-hungry sites that want to use it as an online cache.

this entry’s page

Friday, 27 May 2011

Linked Cache Invalidation

After designing and deploying Cache Channels, it quickly became apparent that one Web cache invalidation mechanism wasn’t able to cover the breadth of use cases.

this entry’s page

Wednesday, 18 May 2011

On HTTP Load Testing

A lot of people seem to be talking about and performing load tests on HTTP servers, perhaps because there’s a lot more choice of servers these days.

this entry’s page

Monday, 4 April 2011

HTTP POST: IETF Prague Edition

this entry’s page

Wednesday, 9 March 2011

htracr in Two Minutes

I made a quick and dirty screencast to show off some of the newer features in htracr.

this entry’s page

Tuesday, 1 March 2011

Last Call: Content-Disposition

The IESG has received a request from the Hypertext Transfer Protocol Bis WG (httpbis) to consider the following document:

this entry’s page

Saturday, 27 November 2010

Digging Deeper with htracr

There’s a lot of current activity on the binding between HTTP and TCP; from pipelining to SPDY, the frontier of Web performance lives between these layers.

this entry’s page

Friday, 1 October 2010

HTTP Roundup: What’s Up with the Web’s Protocol

I’m going to try to start blogging more updates (kick me if I don’t!) about what’s happening in the world of HTTP.

this entry’s page

Friday, 23 July 2010

Thou Shalt Use TLS?

Since SPDY has surfaced, one of the oft-repeated topics has been its use of TLS; namely that the SPDY guys have said that they’ll require all traffic to go over it. Mike Belshe dives into all of the details in a new blog entry, but his summary is simple: “users want it.”

this entry’s page

Wednesday, 30 June 2010

Падручнік па кэшаванню

Patricia Clausnitzer has kindly translated the Caching Tutorial to Belarusian. Thanks!

this entry’s page

Thursday, 3 June 2010

Why Our New TV Doesn't Like the Web

A while back we used an absurd amount of reward points from our credit card to get some Myer gift certificates, and on the weekend these miraculously turned into a new TV, the Sony 32EX600.

this entry’s page

Thursday, 6 May 2010

RFC5861: HTTP Stale Controls

On a bit of a roll, RFC5861: HTTP Stale Controls has (finally) been published as an Informational RFC.

this entry’s page

Wednesday, 5 May 2010

Thoughts on Archiving HTTP

Steve Souders and others have been working for a while on HAR, a HTTP Archive format.

this entry’s page

Wednesday, 7 April 2010

RFC5785: Well-Known URIs

One of the nagging theoretical problems in the Web architecture has been finding so-called “site-wide metadata”; i.e., finding something out about a Web site before you access it. We wrestled with this in P3P way back when, and the TAG took it up after that.

this entry’s page

Wednesday, 10 March 2010

Caching-Tutorial für Webautoren und Webmaster

Thomas Hühn has graciously translated the caching tutorial into German. Thanks!

this entry’s page

Thursday, 18 February 2010

Are Resource Packages a Good Idea?

Resource Packages is an interesting proposal from Mozilla folks for binding together bunches of related data (e.g., CSS files, JavaScript and images) and sending it in one HTTP response, rather than many, as browsers typically do.

this entry’s page

Friday, 15 January 2010

WS-REST (heh, heh)

If you haven’t seen it already, check out the Call for Papers for the First International Workshop on RESTful Design (WS-REST 2010), where I’m on the program committee, along with many of the usual suspects.

this entry’s page

Wednesday, 16 December 2009

HTTP + Politics = ?

Australia has apparently decided, through its elected leaders, to filter its own Internet connection.

this entry’s page

Friday, 13 November 2009

Will HTTP/2.0 Happen After All?

A couple of nights ago, I had a casual chat with Google’s Mike Belshe, who gave me a preview of how their “ Let’s make the Web faster” effort looks at HTTP itself.

this entry’s page

Friday, 30 October 2009

Traffic Server

A long time ago*, the word in high-performance proxy-caching was Inktomi’s Traffic Server. It was so fast it was referred to being “carrier grade” and this could be said without people smirking, and it was deployed by the likes of AOL, when AOL was still how most people accessed the Internet.

this entry’s page

Sunday, 12 July 2009

RED gets a blog

Just FYI, for those interested: RED now has a blog detailing news and other developments. I’ll still post about it here occaisionally, but most RED-related things are going over there…

this entry’s page

Thursday, 25 June 2009

The Resource Expert Droid

A (very) long time ago, I wrote the Cacheability Engine to help people figure out how a Web cache would treat their sites. It has a few bugs, but is generally useful for that purpose.

this entry’s page

Wednesday, 17 June 2009


The caching tutorial is now available in Chinese, courtesy of Che Dong (and apologies for taking so long in linking to it!).

this entry’s page

Friday, 12 June 2009

What to Look For in a HTTP Proxy/Cache

Part of my job is maintaining Yahoo!’s build of Squid and supporting its users, which use it to serve everything from the internal Web services that make sites go to serving Flickr’s images.

this entry’s page

Friday, 5 June 2009

Opera Turbo

HTTP performance is a hot topic these days, so it’s interesting that Opera has announced a “turbo” feature in Opera 10 Beta;

this entry’s page

Friday, 29 May 2009

Most Revealing Google Wave Comment

Everybody’s atwitter (yeah, sue me) about the Google Wave developer preview. Lots of new stuff there, but for me the most revealing comment, almost a throwaway, was here:

this entry’s page

Tuesday, 14 April 2009

Counting the ways that rev="canonical" hurts the Web

I had a lovely holiday weekend in Canberra with the family, without Web access. Perhaps I’ll blog about that soon — Canberra being in my opinion one of the nicest overlooked cities in the world — but that will have to wait. Going offline for a few days always brings a certain dread of what one’s inbox will hold when you get back, and this one was no exception.

this entry’s page

Tuesday, 24 February 2009

Caching When You Least Expect it

There’s a rule of thumb about when a HTTP response can be cached; the Caching Tutorial says:

this entry’s page

Wednesday, 18 February 2009

Stop it with the X- Already!

UPDATE: RFC6648 is now the official word on this topic.

this entry’s page

Monday, 27 October 2008

Dev-Friendly Web Caching

Ryan Tomayko announces Rack::Cache, a HTTP cache for Ruby’s generic Web API;

this entry’s page

Thursday, 16 October 2008


Metadata discovery is a nagging problem that’s been hanging around the Web for a while. There have been a few stabs at this problem (including at least one by yours truly), but no real progress.

this entry’s page

Friday, 4 July 2008

The WS-Empire Strikes Back... feebly

Here’s a gem on a little-used mailing list:

this entry’s page

Thursday, 22 May 2008

The Pitfalls of Debugging HTTP

Some folks at work were having problems debugging HTTP with LWP ’s command-line GET utility; it turned out that it was inserting Link headers — HTTP headers, mind you — for each HTML <link> element present.

this entry’s page

Thursday, 20 March 2008

Moving Beyond Methods in REST

Having complained before about the sad state of HTTP APIs, I’m somewhat happy to say that people seem to be getting it, producing more capable server-side and client-side tools for exposing the full range of the protocol; some frameworks are even starting to align object models with resource models, where HTTP methods map to method calls on things with identity. Good stuff.

this entry’s page

Monday, 3 March 2008


Not many people that I know outside of IETF circles realise that a new *DAV effort has started up; CardDAV.

this entry’s page

Wednesday, 6 February 2008

Another Kind of HTTP Negotiation

Here’s one that I’ve been wondering about for a while, for the LazyWeb (HTTP Geek Edition);

this entry’s page

Monday, 21 January 2008

Watching WADL (and other rambling thoughts)

I’m following the discussion of RESTful Web description in general, and WADL in particular, with both difficulty and interest (see Patrick and Joe’s thoughts for a nice contrast).

this entry’s page

Friday, 4 January 2008

Cache Channels

The stale-while-revalidate and stale-if-error extensions aren’t the only fiddling we’ve been doing with the HTTP caching model. Now that Squid 2.7 is starting to see daylight, I can explain about a much more ambitious project — Cache Channels.

this entry’s page

Wednesday, 12 December 2007

Two HTTP Caching Extensions

We use caching extensively inside Yahoo! to improve scalability, latency and availability for back-end HTTP services, as I’ve discussed before.

this entry’s page

Sunday, 9 December 2007

Why Revise HTTP?

I haven’t talked about it here much, but I’ve spent a fair amount of time over the last year and a half working with people in the IETF to get RFC2616 — the HTTP specification — revised.

this entry’s page

Friday, 2 November 2007

WADL Documentation XSLT Updated

I’ve updated the WADL documentation stylesheet, primarily to;

this entry’s page

Saturday, 8 September 2007


Feed Paging and Archiving (nee Feed History) has finally made it to a standards-track RFC.

this entry’s page

Tuesday, 7 August 2007

ETags, ETags, ETags

I’ve been hoping to avoid this, but ETags seem to be popping up more and more often recently. For whatever reason, people latch onto them as a litmus test for RESTfulness, as the defining factor of HTTP’s caching model, and much more.

this entry’s page

Saturday, 28 July 2007

URI Templates Redux

URI Templates -01 is now an Internet-Draft.

this entry’s page

Wednesday, 20 June 2007

The State of Proxy Caching

A while back I wrote up the state of browser caching, after writing a quick-and-dirty XHR-based test page, with the idea that if people know how their content is handled by common implementations, they’d be able to trust caches a bit more.

this entry’s page

Tuesday, 15 May 2007

Expires vs. max-age

I occasionally get a question from readers of the caching tutorial about whether to use the Expires header or Cache-Control: max-age to control a response’s freshness lifetime.

this entry’s page

Sunday, 29 April 2007

Squid is My Service Bus

The QCon presentation ( slides) was ostensibly about how we use HTTP for services within Yahoo’s Media Group. When I started thinking about the talk, however, I quickly concluded that everyone’s heard enough about the high-level benefits of HTTP and not nearly enough details of what it does on the ground. So, I decided to concentrate on one aspect of the value that we get from using HTTP for services; intermediation, as an example.

this entry’s page

Sunday, 15 February 2004

Caching Tutorial Update

I’ve published a revision of the Caching Tutorial for Web Authors and Webmasters, the first non-trivial edit in some time almost since I wrote it in 1998. That said, there aren’t any substantial changes; this is mostly tweaking and incorporation of new information.

this entry’s page

Saturday, 28 June 2003

Caching is often enough

I feel compelled to respond to Norm Walsh’s thoughts on caching.

this entry’s page

Monday, 5 May 2003


I’ve finally gotten sick enough of a project that I’ve been working on for waaaay too long to release it to the unsuspecting^H^H^H general public.

this entry’s page

Tuesday, 8 April 2003

HTTP header sniffing

LiveHTTPHeaders for Mozilla is the best HTTP header sniffer I’ve seen yet; up till now, I’ve been using WebTee, but for most purposes, this is much better. Enjoy.

this entry’s page