mnot’s blog

Design depends largely on constraints.” — Charles Eames

Sunday, 28 August 2011

Caching HTTP Web

Better Browser Caching

In discussing my whinge about AppCache offline with a few browser vendory folks, I ending up writing down my longstanding wishlist for making browser caches better. Without further ado, a bunch of blue-sky ideas;

Cache Contexts

Most current HTTP caches — whether browser or proxy — put everything they cache into a single “bucket” that shares configuration and storage space.

While that makes sense for a HTTP proxy run by your ISP, it doesn’t make as much for browsers, because Web sites expect the browser to behave in predictable ways.

I think that browsers should have an identifiable, separate cache context for each Web site that is visited, so that they can be independently limited, configured and interacted with. Ideally this would be by default, although I suppose it could also be opt-in by the site.

This has a lot of follow-on effects which I’ll explore below.

Cache Eviction

When a typical Web cache runs out of room, it performs eviction (i.e., it makes more room) on a per-object basis; i.e., it looks for the oldest (Least Recently Used) or least used (Least Frequently Used) response and gets rid of it.

That’s fine for a big shared proxy cache (although many forests of trees have been wasted on cache eviction algorithm research, I’ll spare my poor readers the details). For a private, single-user cache, however, it may not be so great.

As I mentioned in the AppCache article, Web sites often don’t like it when they’ve got some things in cache but not others, because a few things have been evicted, leading to unpredictable performance. If browser caches aren’t universal, but instead bucketed into contexts, it allows eviction algorithms to operate on two levels; inside the context, and globally.

Inside a given context (say, for, we can limit the site’s cache size; let’s say we’ll give all sites 1M by default, and 10M if they’re bookmarked. When more than the limit needs to be cached for that site, we perform eviction and remove a few of that site’s cached responses (probably LRU, although see below for ideas about hinting).

Then, when the overall cache size (ALL of the contexts) gets bigger than we’d like, we evict the least most recently used contexts (probably omitting bookmarked sites).

This gives more predictable caching to your Web site; you know that you’ll get a bucket of a certain size, as long as the user has visited the site recently, rather than the current situation of having things from your site randomly evicted, based upon the user’s behaviour on other sites.

Cache Inspection and Interaction

Next, you’d need a way for sites to control eviction within their context. The simple way is a cache-control response directive; e.g.,

Cache-Control: max-age=43200, priority=5

Where “priority” tells the browser cache how important this is compared to other things in the same bucket. There’d need to be more detail here, of course, but the idea is to make it possible to prioritise certain things in the context, so that sites can control caching to the level where navigation assets never get evicted when they’re fresh, while allowing bigger things to be cached more ephemerally, in case the user replays a video or goes back a page.

That handles the simple cases admirably, I think. However, for full its-getting-crazy-in-here cache control, you’d need some JavaScript APIs to allow a site to inspect, understand and manipulate what it has in cache, and the cache context itself.

To start with, this means a JS API to inspect the cache context and implement custom eviction algorithms.

// The context for this window's site, including not only
// cached objects on this page, but also everything else for
// the site.
var my_context = window.CacheContext;

// How much space, in bytes, the context is allowed. Read-only (for now).
var ctx_capacity = my_context.capacity;

// How much space is currently used, in bytes.
var in_use = my_context.used;

// The contents of the context, as an array;
my_context.contents.forEach(function(cached_item) {
  // each item is an object that has a bunch of interesting properties:
  cached_item.url // not brain surgery to figure this one out
  cached_item.age // cache age according to the HTTP ageing algorithm
  cached_item.freshness // how fresh it is, again according to HTTP
  cached_item.cache_control // cache-control response directives, as an object
  cached_item.last_modified // yup, when it was last modified, as per hdrs
  cached_item.size // how big it is, in bytes
  cached_item.status_code // HTTP status code of the response
  cached_item.priority // eviction policy weight (read/write)
  cached_item.make_stale() // mark the item as stale in the cache
  cached_item.validate() // reload the cached entry, making fresh if successful
} );

// A callback that's triggered when the cache needs to evict
// something from this context. If the number of bytes used
// is still over the context's capacity after this returns,
// or if the callback is not handled, the cache will
// automatically evict the lowest priority objects until
// space is available.
my_context.onEvict(function() {...} );

Put together, this would enable sites to actively inspect and manage their cache. While we’re at it, it’d be cool to have DOM extensions for IMG / SCRIPT / LINK rel=”stylesheet” / A tags (others?) to give access to the cached_item object, as well as a way to get at it from XHR.

As you can probably tell, this is all just sketching in the clouds (so to speak) at this point, but I do believe this is the direction we’ll end up going if we get serious about browser caching. The important question, for me, is whether a full API like this is necessary, or if the 99% case can be met with contexts + priority.

Request Cache Control

HTTP already defines a way to control caches directly, of course; Request Cache-Control directives. For example,

GET /data/thing HTTP/1.1
Cache-Control: max-age=600, max-stale=30

should work in browser caches when sent with XHR (returning something that’s at most 600 seconds old, even if it’s up to 30 seconds stale), but it doesn’t.

Likewise, browsers should obviously support:

GET /other/thing HTTP/1.1
Cache-Control: no-store

Supporting request Cache-Control in XmlHttpRequest would give Web developers much finer-grained control over cache behaviour.

Fallback for the Live Web

HTML5 AppCache already has the ability to define “fallback” pages for offline apps, and the “real” Web should have this ability too, so that users aren’t subjected to confusing browser messages when there’s an intermittent network problem. One way to do this would be to specify a few URIs as hints in the browser-hints file.

Linked Cache Invalidation

Being able to specify the relationships between resources and have caches do intelligent things based upon them is incredibly powerful; it makes caches suddenly useful for a lot more than static assets. So, I’d like to see browsers implement Linked Cache Invalidation.

That’s it for now; it’s not much more than a sketch, and an incomplete one at that, but I thought it’d be interesting to see what people like or not about the general direction here.