mnot’s blog

Design depends largely on constraints.” — Charles Eames

HTTP Entries

Wednesday, 15 May 2013

Indicating Problems in HTTP APIs

A common part of HTTP-based APIs is telling the client that something has gone wrong. Most APIs do this in some fashion, whether they call it a “Fault” (very SOAP-y), “Error” or whatever. Most of them define a new format for just this purpose; for examples, see Amazon’s, OpenStack’s, Twitter’s, Facebook’s, and SalesForce’s. Twitter's is fairly representative:  {"errors":[{"message":"Sorry, that page does not exist","code":34}]} Here, they associate a human-readable message and an error code with the error. That’s a good start, but...

this entry’s page (4 comments)

Friday, 4 January 2013

Exploring Header Compression in HTTP/2.0

One of the major mechanisms proposed by SPDY for use in HTTP/2.0 is header compression.  This is motivated by a number of things, but heavy in the mix is the combination of having more and more requests in a page, and the increasing use of mobile, where every packet is, well, precious.  Compressing headers (separately from message bodies) both reduces the overhead of additional requests and of introducing new headers. To illustrate this, Patrick put together a synthetic test that...

this entry’s page (5 comments)

Tuesday, 18 December 2012

"Why Don't You Just…"

A proposal by John Graham-Cumming is currently doing the rounds: HMURR (pronounced ‘hammer’) introduces a new pipelining mechanism with explicit identifiers used to match requests and responses sent on the same TCP connection so that out-of-order responses are possible. The current HTTP 1.1 pipelining mechanism requires that responses be returned in the same order as requests are made (FIFO) which itself introduces a head-of-line blocking problem. This seems attractive at first glance; rather than starting a whole new protocol, why not just incrementally improve an...

this entry’s page

Friday, 7 December 2012

HTTP Status: 101 Switching Protocols

The HTTPbis Working Group met in Atlanta last month; here’s how things are going. HTTP/1.1 We’re now out of Working Group Last Call on all of our “core” documents, so the editors are working through the issues that brought up. As soon as that’s done, we’ll go to IETF Last Call, and hopefully soon after well have a number of new RFCs defining HTTP/1.1. Here, you can see the upswing in number of issues during our WGLC period: To see...

this entry’s page (1 comment)

Tuesday, 4 December 2012

Evolving HTTP APIs

One of the most vexing problems that still seems to be facing people when I talk to them about HTTP APIs is how to handle versioning and extensibility — i.e., how they evolve. I tend to think about this a lot and talk to quite a few people about it, since I’m intimately familiar with the approaches to versioning that the HTTP protocol itself takes, and with the general attitude taken to it in the broader Internet architecture by the...

this entry’s page (6 comments)

Monday, 29 October 2012

OPTIONS is Not the Method You're Looking For

Once in a while, people ask me whether they should use the OPTIONS HTTP method, and whether we should try to define formats for discovering resource capabilities with it. I usually say “no.” This is a personal position, nothing “official”. That said, the conversation usually goes like this: What’s Wrong with OPTIONS? A few things. First and foremost, it can’t be cached. Usually, people want to use OPTIONS to discover some metadata about the server (the “OPTIONS *” case) or...

this entry’s page (7 comments)

Monday, 24 September 2012

Caching POST

One of the changes in Apple's release of iOS6 last week was a surprising new ability to cache POST responses. Lots has been said about this, but some people reading RFC2616 have come away scratching their head about whether this is actually a bug or not. The HTTP spec says this about POST: Responses to this method are not cacheable, unless the response includes appropriate Cache-Control or Expires header fields.  Which, on the face of it, seems to say that...

this entry’s page (4 comments)

Wednesday, 5 September 2012

Why PATCH is Good for Your HTTP API

A common problem for APIs is partial update; when the client wants to change just one part of a resource’s state. For example, imagine that you’ve got a JSON representation of your widget resource that looks like: {   "name": "abc123",   "colour": "blue",   "count": 4 } and you want to update the “count” member’s value to “5”. Now, you could just PUT the entire thing back with the updated value, but that requires a recent GET of its...

this entry’s page (28 comments)

Saturday, 4 August 2012

HTTP in Vancouver

The HTTPBIS Working Group is in a transitional phase; we’re rapidly finishing our revision of the HTTP/1.1 specification and just getting steam up on our next target, HTTP/2.0. So, we met in Vancouver this week as part of IETF84. Here’s a quick summary. HTTP/1.1 Five years ago, we held a Birds of a Feather meeting at IETF69, discussing whether we should attempt to revise RFC2616, the specification of HTTP/1.1. It’s turned out to be a massive job, but we’re close...

this entry’s page

Wednesday, 11 July 2012

Bad HTTP API Smells: Version Headers

One thing I didn't cover in my previous rant on HTTP API versioning is an anti-pattern that I'm seeing a disturbing number of APIs adopt; using a HTTP header to indicate the overall version of the API in use. Examples include CIMI, CDMI, GData and I'm sure many more. These APIs are designed so that the client can send a request header advertising what version of the protocol they support: GET /foo HTTP/1.1 Host: api.example.org BadApiVersion: 1.2 and the server will...

this entry’s page (2 comments)

Monday, 25 June 2012

HTTP API Complexity

@dret: if your scenario is homogeneous and models are harmonized across participants, #REST is of limited utility for you. Erik’s tweet just now reminded me of something I’d been wanting to write about for a while; complexity in HTTP APIs, and their effect on how you use the protocol. One Client, One Server The simplest HTTP API is one where you have a known client and server talking to each other; they may be managed by separate people or the...

this entry’s page (3 comments)

Saturday, 31 March 2012

What's Next for HTTP

We had two great meetings of the HTTPbis Working Group in Paris this week — one to start wrapping up our work on HTTP/1.1, and another to launch some exciting new work on HTTP/2.0. HTTP/1.1 The HTTP/1.1 specification (RFC2616) was standardised in the mid-to-late 90’s. At the time, there was a lot of pressure to produce something quickly, because the Web was getting popular, and swamping a lot of networks in the process. As a result, the spec is somewhat...

this entry’s page (2 comments)

Tuesday, 25 October 2011

Web API Versioning Smackdown

A lot of bits have been used over on the OpenStack list recently about versioning the HTTP APIs they provide. This over-long and rambling post summarises my current thoughts on the topic, both as background for that discussion, as well as for review in the wider community. The Warm-up: Software vs. Web Versioning Developers are used to software versioning; e.g., for every release, you bump an identifier. There are usually major versions, minor versions, and sometimes things like package identifiers....

this entry’s page (15 comments)

Friday, 21 October 2011

Why ESI is Still Important, and How to Make it Better

More than ten years ago, I was working at Akamai and got involved in the specification of Edge Side Includes (ESI), sort of a templating language for intermediaries. In that time, interest in ESI has grown, waned and been reborn. As far as I can tell, it's implemented not only by Akamai and Oracle (the main forces behind it), but also in Varnish, Squid, and lots of other places too. Back then, I had a strong suspicion that it'd die because...

this entry’s page (9 comments)

Friday, 2 September 2011

RFC6266 and Content-Disposition

HTTPbis published RFC6266 a little while ago, but the work isn’t finished. This is the RFC that clarifies how the Content-Disposition header is used in HTTP; in a nutshell, while basic file downloads worked OK, there wasn’t any broad interoperability between browsers for non-ASCII filenames. Julian Reschke did the hard work of coming up with a test suite to find how how bad things were, filing bugs with browsers (such as here and here), and finally writing the draft that eventually...

this entry’s page (3 comments)

Sunday, 28 August 2011

Better Browser Caching

In discussing my whinge about AppCache offline with a few browser vendory folks, I ending up writing down my longstanding wishlist for making browser caches better. Without further ado, a bunch of blue-sky ideas; Cache Contexts Most current HTTP caches — whether browser or proxy — put everything they cache into a single “bucket” that shares configuration and storage space. While that makes sense for a HTTP proxy run by your ISP, it doesn’t make as much for browsers, because...

this entry’s page

Wednesday, 24 August 2011

Distributed Hungarian Notation doesn't Work

It used to be that when you registered a media type, a URI scheme, a HTTP header or another protocol element on the Internet, it was an opaque string that was a unique identifier, nothing more. Sure, there are some substructures (e.g., vnd. and prs. in media types) to aid in avoiding collisions, but they don’t actually do anything. And even so, they need to be used judiciously (e.g., the problems inherent in x-). However, it’s now becoming fashionable to...

this entry’s page (4 comments)

Friday, 5 August 2011

HTTP Pipelining Today

Last week, Blaze.io highlighted how mobile browsers use HTTP pipelining. I’ve been active in trying to get pipelining more widely deployed, but to date I haven’t tested mobile browsers much. So, one VM and two test pages (20 images on each) later, I asked my twitter peeps to hit it with their phones while I was watching with htracr. The results confirm what they saw; mobile browsers do pipeline, sometimes aggressively. A normal build of Firefox (and pretty much...

this entry’s page (2 comments)

Wednesday, 27 July 2011

CSP

FYI, I’ve implemented Content Security Policy on this site. If your’e a Mozilla user, please tell me if you have any problems. (No, I’m not particularly worried about XSS here; I just want to get a feel for it; see my feedback so far.)...

this entry’s page

Monday, 11 July 2011

What Proxies Must Do

The explosion of HTTP implementations isn’t just in clients and servers. An oft-overlooked but important part of the Web ecosystem is the intermediary, often called just a “proxy”*. These days, it’s pretty easy for anyone to build a proxy using Python, Ruby, Perl, Java or Node.JS, and there are a bunch of frameworks that can help you do this, such as node-http-proxy. Additionally, there are lots of off-the-shelf proxies that you can use, from the widely-known Squid and Traffic...

this entry’s page (5 comments)

Sunday, 19 June 2011

Fixing AppCache

HTML5’s AppCache mechanism is one confused little puppy. Purporting to be for taking web applications offline — a compelling and useful thing — it’s more often used by performance-hungry sites that want to use it as an online cache. Unfortunately, this means that those sites will be popping up dialogues like this: Even more unfortunately, if you click “Don’t Allow”, you’ll just be asked to approve it again next time you navigate to the site. And again. And again....

this entry’s page (4 comments)

Friday, 27 May 2011

Linked Cache Invalidation

After designing and deploying Cache Channels, it quickly became apparent that one Web cache invalidation mechanism wasn’t able to cover the breadth of use cases. In a nutshell, Cache Channels trades off immediacy for reliability; that is, while cache invalidations don’t take place right away (there’s a 10-30 second window), you know that they’ll be respected, because of how the protocol is designed. That’s great if you need to (for example) invalidate news articles with months of TTL because...

this entry’s page (4 comments)

Wednesday, 18 May 2011

On HTTP Load Testing

A lot of people seem to be talking about and performing load tests on HTTP servers, perhaps because there’s a lot more choice of servers these days. That’s great, but I see a lot of the same mistakes being made, making the conclusions doubtful at best. Having spent a fair amount of time benchmarking high-performance proxy caches and origin servers for my day job, here are a few things that I think are important to keep in mind. It’s...

this entry’s page (18 comments)

Monday, 4 April 2011

HTTP POST: IETF Prague Edition

Last week found lots of HTTP-ish folks in Prague for IETF 80. In short, the good bits: HTTPbis HTTPbis had a cracker of a meeting, with three RFC2616 authors and representatives of all five major browsers in the room. First and foremost, the Content-Disposition draft has been approved for publication as an RFC by the IESG. This is great news for interoperability, with growing browser support for the revised specification. With one critical issue scheduled for a fix in...

this entry’s page (3 comments)

Wednesday, 9 March 2011

htracr in Two Minutes

I made a quick and dirty screencast to show off some of the newer features in htracr. htracr demo from mnot on Vimeo. It’s still a work in progress (you can see at least one visible bug if you’re looking!)....

this entry’s page

Tuesday, 1 March 2011

Last Call: Content-Disposition

The IESG has received a request from the Hypertext Transfer Protocol Bis WG (httpbis) to consider the following document: 'Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP)' <draft-ietf-httpbis-content-disp-06.txt> as a Proposed Standard The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2011-03-14. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain...

this entry’s page

Saturday, 27 November 2010

Digging Deeper with htracr

There’s a lot of current activity on the binding between HTTP and TCP; from pipelining to SPDY, the frontier of Web performance lives between these layers. To get more visibility in exactly what’s happening down there, I decided to throw together a little tool to show how HTTP uses TCP: htracr. In a nutshell, It’s a packet sniffer written in JavaScript, thanks to node_pcap, and it uses RaphaelJS to visualise what’s going on. It’s still very young (lots of...

this entry’s page (2 comments)

Friday, 1 October 2010

HTTP Roundup: What’s Up with the Web’s Protocol

I’m going to try to start blogging more updates (kick me if I don’t!) about what’s happening in the world of HTTP. HTTPbis The effort to revise the core HTTP specification (RFC 2616) is going nicely, albeit slowly. Given the nature of the work, slow is better than rushed. In late July, some of the HTTPbis WG editors met in (then) sunny Münster before IETF78 in Maastricht. The result of that were drafts -10 and -11, closing a number...

this entry’s page (8 comments)

Friday, 23 July 2010

Thou Shalt Use TLS?

Since SPDY has surfaced, one of the oft-repeated topics has been its use of TLS; namely that the SPDY guys have said that they’ll require all traffic to go over it. Mike Belshe dives into all of the details in a new blog entry, but his summary is simple: “users want it.” I don’t think it’s that simple. Trust I trust my ISP, to a point; I have a business relationship with them, so I don’t worry too much...

this entry’s page (7 comments)

Wednesday, 30 June 2010

Падручнік па кэшаванню

Patricia Clausnitzer has kindly translated the Caching Tutorial to Belarusian. Thanks!...

this entry’s page (4 comments)

Thursday, 3 June 2010

Why Our New TV Doesn't Like the Web

A while back we used an absurd amount of reward points from our credit card to get some Myer gift certificates, and on the weekend these miraculously turned into a new TV, the Sony 32EX600. Overall, we really like it; while I’m still trying to find the exact recipe to successfully encode video to feed to it via DLNA, it’s beautiful to look at, and (unlike most TVs these days), the UI is a pleasure to use. Except. One...

this entry’s page (4 comments)

Thursday, 6 May 2010

RFC5861: HTTP Stale Controls

On a bit of a roll, RFC5861: HTTP Stale Controls has (finally) been published as an Informational RFC. As discussed before in “Two HTTP Caching Extensions,” these are very useful ways to hide latency and errors from your end users. While they’re most useful in HTTP gateway caches (a.k.a. reverse proxy caches / accelerators), very latency-sensitive sites might find them useful as well when working with “normal” proxy caches. Both are implemented in Squid 2.7. Not only does Squid...

this entry’s page (5 comments)

Wednesday, 5 May 2010

Thoughts on Archiving HTTP

Steve Souders and others have been working for a while on HAR, a HTTP Archive format. I love the idea behind HAR, but as I expressed on the mailing list (so far no response), I have a hard time believing that different implementations will parse and process HTTP and then serialise it into HAR in exactly the same way. My use case is incorporating HAR support into RED, both for import and export. However, RED needs byte-for-byte access to...

this entry’s page (8 comments)

Wednesday, 7 April 2010

RFC5785: Well-Known URIs

One of the nagging theoretical problems in the Web architecture has been finding so-called “site-wide metadata”; i.e., finding something out about a Web site before you access it. We wrestled with this in P3P way back when, and the TAG took it up after that. The easy solution to this is to define a static “well-known” URL — like /robots.txt. However, having a third party come along and squat on part of every Web site’s URL namespace is less than...

this entry’s page (3 comments)

Wednesday, 10 March 2010

Caching-Tutorial für Webautoren und Webmaster

Thomas Hühn has graciously translated the caching tutorial into German. Thanks! See also the Chinese, Czech and French translations. To help the translators keep up with changes, I've started hosting the raw document on Github, which can also be used to log issues....

this entry’s page

Thursday, 18 February 2010

Are Resource Packages a Good Idea?

Resource Packages is an interesting proposal from Mozilla folks for binding together bunches of related data (e.g., CSS files, JavaScript and images) and sending it in one HTTP response, rather than many, as browsers typically do. Intuitively, this seems to make sense; less HTTP requests is good, right? Maybe, maybe not. AFAICT, there aren’t any metrics comparing RP vs. traditional sites (has anyone done this?). In any case, a few concerns come to mind about this approach to making...

this entry’s page (18 comments)

Friday, 15 January 2010

WS-REST (heh, heh)

If you haven’t seen it already, check out the Call for Papers for the First International Workshop on RESTful Design (WS-REST 2010), where I’m on the program committee, along with many of the usual suspects. Submissions due February 8, 2010, 23.59 Hawaii time. If only I were there to receive them…...

this entry’s page (1 comment)

Wednesday, 16 December 2009

HTTP + Politics = ?

Australia has apparently decided, through its elected leaders, to filter its own Internet connection. Since many, many other people are discussing whether this is advisable or indeed effective, I’ll focus here on what this will do to HTTP, and by extension the Web. What’s on the Table Reading the white paper, there are three different technologies for filtering the Web on the table; “pass-by hybrid”, whereby the ISP’s router will shunt traffic to target IP addresses to another box...

this entry’s page (3 comments)

Friday, 13 November 2009

Will HTTP/2.0 Happen After All?

A couple of nights ago, I had a casual chat with Google’s Mike Belshe, who gave me a preview of how their “Let’s make the Web faster” effort looks at HTTP itself. SPDY (nee FLIP) is an alternate application protocol that’s in Chromium, but buried so deeply that you have to enable it with a command-line option (—use-flip). AFAICT there aren’t even any public servers that support it yet, but it’s still a very exciting development. Why? In a...

this entry’s page (6 comments)

Friday, 30 October 2009

Traffic Server

A long time ago*, the word in high-performance proxy-caching was Inktomi’s Traffic Server. It was so fast it was referred to being “carrier grade” and this could be said without people smirking, and it was deployed by the likes of AOL, when AOL was still how most people accessed the Internet. Then, in 2001, Yahoo! bought Inktomi. They did this because they wanted to be a player in Search, and they happened to get a proxy/caching product for free....

this entry’s page (6 comments)

Sunday, 12 July 2009

RED gets a blog

Just FYI, for those interested: RED now has a blog detailing news and other developments. I’ll still post about it here occaisionally, but most RED-related things are going over there… First (substantial) post: Using RED on a page’s assets....

this entry’s page

Thursday, 25 June 2009

The Resource Expert Droid

A (very) long time ago, I wrote the Cacheability Engine to help people figure out how a Web cache would treat their sites. It has a few bugs, but is generally useful for that purpose. However, as I’ve got more involved in using HTTP for non-browser things, it’s become apparent that more than caching is important when you’re examining a resource to see how it behaves; things like partial content, syntax checking and other esoteric but important details. Very...

this entry’s page (15 comments)

Wednesday, 17 June 2009

面向站长和网站管理员的Web缓存加速指南

The caching tutorial is now available in Chinese, courtesy of Che Dong (and apologies for taking so long in linking to it!). Norwegian should be coming soon......

this entry’s page (1 comment)

Friday, 12 June 2009

What to Look For in a HTTP Proxy/Cache

Part of my job is maintaining Yahoo!’s build of Squid and supporting its users, which use it to serve everything from the internal Web services that make sites go to serving Flickr’s images. In that process, I often am asked “what about X?”, where X is another caching or load balancing product (yes, Squid can be used as a load balancer). For example, Varnish, or lighttpd. Generally, these comparisons come down to three factors; performance, features and manageability. Almost...

this entry’s page (3 comments)

Friday, 5 June 2009

Opera Turbo

HTTP performance is a hot topic these days, so it’s interesting that Opera has announced a “turbo” feature in Opera 10 Beta; Ever felt a Web site was loading slowly? Do you think it will happen again? Think again: Opera Turbo is a compression technology that provides significant improvements in browsing speeds over limited-bandwidth connections like a crowded Wi-Fi in a cafe or browsing through your mobile phone while commuting. They go on to give more details in their...

this entry’s page (20 comments)

Friday, 29 May 2009

Most Revealing Google Wave Comment

Everybody’s atwitter (yeah, sue me) about the Google Wave developer preview. Lots of new stuff there, but for me the most revealing comment, almost a throwaway, was here: Did we mention we use Squid? In other words, Google’s hot new, Web 2.0-on-steroids HTML5 XMPP++ open social platform buzzword generator works well with, indeed relies upon (for the purposes of making a nice demo) a more than 15-year-old piece of nearly-forgotten (in some quarters) Web infrastructure. That’s pretty cool....

this entry’s page (7 comments)

Tuesday, 14 April 2009

Counting the ways that rev="canonical" hurts the Web

I had a lovely holiday weekend in Canberra with the family, without Web access. Perhaps I’ll blog about that soon — Canberra being in my opinion one of the nicest overlooked cities in the world — but that will have to wait. Going offline for a few days always brings a certain dread of what one’s inbox will hold when you get back, and this one was no exception. That’s because while I was watching the kids rolling down...

this entry’s page (30 comments)

Tuesday, 24 February 2009

Caching When You Least Expect it

There’s a rule of thumb about when a HTTP response can be cached; the Caching Tutorial says: If the response’s headers tell the cache not to keep it, it won’t. If the request is authenticated or secure, it won’t be cached. If no validator (an ETag or Last-Modified header) is present on a response, and it doesn’t have any explicit freshness information, it will be considered uncacheable. And, generally, this is true; most implementations won’t both caching something that...

this entry’s page (6 comments)

Wednesday, 18 February 2009

Stop it with the X- Already!

Sometimes, it seems like every time somebody has a great idea for a new HTTP header, media type, or pretty much any other protocol element, they do the same thing. Rather than trying to figure out how to fit into how the rest of the world operates, getting adequate review and socialising their proposal, they just stick a bloody X- on the front and ship it. The IETF has no-one to blame but itself for this situation, of course....

this entry’s page (19 comments)

Monday, 27 October 2008

Dev-Friendly Web Caching

Ryan Tomayko announces Rack::Cache, a HTTP cache for Ruby’s generic Web API; The basic goal is standards-based HTTP caching that scales down to the early stages of a project, development environments, light to medium trafficked sites, stuff like that. HTTP’s caching model is wildly under-appreciated in the Ruby web app community and my hope is that making its benefits more accessible will lead to wider understanding and acceptance. I commented and, of course, focused solely on one aspect of...

this entry’s page (4 comments)

Thursday, 16 October 2008

/site-meta

Metadata discovery is a nagging problem that’s been hanging around the Web for a while. There have been a few stabs at this problem (including at least one by yours truly), but no real progress. This is both unfortunate and worrisome, because as the next generation of Web-based protocols informed by REST, Web 2.0 and the like roll out, they’re going to need a way to find and talk about metadata on the Web in an automated fashion. And...

this entry’s page (17 comments)

Friday, 4 July 2008

The WS-Empire Strikes Back... feebly

Here’s a gem on a little-used mailing list: As most of you know, over the last several years fairly good progress has been made on standardizing Web services. Many Web services specifications have, in fact, been standardized in W3C (i.e. SOAP 1.2, WSDL 2.0, WS-Addressing, WS-Policy, etc). There is still some work to be done. Accessing data about a resource through Web services is an area of the Web services architecture that has yet to be fully realized. Some...

this entry’s page (9 comments)

Thursday, 22 May 2008

The Pitfalls of Debugging HTTP

Some folks at work were having problems debugging HTTP with LWP’s command-line GET utility; it turned out that it was inserting Link headers — HTTP headers, mind you — for each HTML <link> element present. Blurgh. This brought to mind some other peculiarities that can make debugging HTTP more complex and REALLY ANNOYING… Curl automagically adds a Pragma: no-cache to requests, so that you don’t have to worry about scaling the Web or getting decent performance. LiveHTTPHeaders shows you...

this entry’s page (25 comments)

Thursday, 20 March 2008

Moving Beyond Methods in REST

Having complained before about the sad state of HTTP APIs, I’m somewhat happy to say that people seem to be getting it, producing more capable server-side and client-side tools for exposing the full range of the protocol; some frameworks are even starting to align object models with resource models, where HTTP methods map to method calls on things with identity. Good stuff. However, something’s been bugging me for a long time about this. While there’s a nice internal logic...

this entry’s page (11 comments)

Monday, 3 March 2008

DAV WTF?

Not many people that I know outside of IETF circles realise that a new *DAV effort has started up; CardDAV. An address book access protocol leveraging the vCard data format. The Internet-draft draft-daboo-carddav will be the starting point. The WG is explicitly cautioned to keep the base specification feature set small with an adequate extension mechanism, as failure to do so was a problem for previous PAB efforts (ACAP). Draft-daboo looks like it’s taken a page out of CalDAV...

this entry’s page (4 comments)

Wednesday, 6 February 2008

Another Kind of HTTP Negotiation

Here’s one that I’ve been wondering about for a while, for the LazyWeb (HTTP Geek Edition); PUTs and POSTs can result in the creation of new resources, or changes to the state of existing ones. The response to both can contain one of a status message about what happened, or a representation of the resource (or most important one, if more than one was created/modified); often signalled by the Content-Location header, or nothing (i.e., just the status line and...

this entry’s page (25 comments)

Monday, 21 January 2008

Watching WADL (and other rambling thoughts)

I’m following the discussion of RESTful Web description in general, and WADL in particular, with both difficulty and interest (see Dare, Patrick and Joe’s thoughts for a nice contrast). Difficulty because there’s so much of it, and it’s hard to give each piece the attention it deserves. Interest because it goes to the heart of the harder parts of REST — e.g., “hypertext as the engine of application state.” On top of that, I was one of the people...

this entry’s page (1 comment)

Friday, 4 January 2008

Cache Channels

The stale-while-revalidate and stale-if-error extensions aren’t the only fiddling we’ve been doing with the HTTP caching model. Now that Squid 2.7 is starting to see daylight, I can explain about a much more ambitious project — Cache Channels. In HTTP, there are generally two ways to keep something in a cache fresh; using a freshness lifetime (e.g., Cache-Control: max-age), and validation (e.g., If-Modified-Since). Together, they do a really good job of making the Web seem faster and reducing load...

this entry’s page (4 comments)

Wednesday, 12 December 2007

Two HTTP Caching Extensions

We use caching extensively inside Yahoo! to improve scalability, latency and availability for back-end HTTP services, as I’ve discussed before. However, there are a few situations where the plain vanilla HTTP caching model doesn’t quite do the trick. Rather than come up with one-off solultions to our problems, we tried going in the other direction; finding the most general solution that still met our needs, in the hopes of meeting others’ as well. Here are two of them (with...

this entry’s page (16 comments)

Sunday, 9 December 2007

Why Revise HTTP?

I haven’t talked about it here much, but I’ve spent a fair amount of time over the last year and a half working with people in the IETF to get RFC2616 — the HTTP specification — revised. That effort reached a milestone last week when the HTTPbis Working Group had its first face-to-face meeting in Vancouver. It’s still early days, but we’ve already made good progress; based on what we saw in the room, for example, it looks like...

this entry’s page (9 comments)

Friday, 2 November 2007

WADL Documentation XSLT Updated

I've updated the WADL documentation stylesheet, primarily to; Fix a bug with finding and displaying XML Schema Make it compatible with xsltproc (and hopefully most other XSLT1.0 processors that understand EXSLT node-set) Generate valid XHTML The hard part was getting support for xsltproc; while switching over to the node-set function was easy, it uncovered other bugs around how libxml handles copying namespace nodes; an ugly and tiresome hack was required to work around it, but it seems to work....

this entry’s page

Saturday, 8 September 2007

5005

Feed Paging and Archiving (nee Feed History) has finally made it to a standards-track RFC. For many non-traditional (read: non-blog) applications of Atom, I think archived feeds in particular are going to be vital. I’m already using it in some places which should be seeing the light of day before too long; e.g., sending events to HTTP caches — a problem space that people have been noodling on for a very long time indeed. I’m also hoping that blog...

this entry’s page (3 comments)

Tuesday, 7 August 2007

ETags, ETags, ETags

I’ve been hoping to avoid this, but ETags seem to be popping up more and more often recently. For whatever reason, people latch onto them as a litmus test for RESTfulness, as the defining factor of HTTP’s caching model, and much more. So, let me counter: they’re not all that. In fact, there are a number of pitfalls you need to be wary of if you use them. First, depending on how they’re generated, you might find different boxes...

this entry’s page (8 comments)

Saturday, 28 July 2007

URI Templates Redux

URI Templates -01 is now an Internet-Draft. After sitting on the spec for a while and trying to figure out an elegant solution to the encoding problem, we decided to take the simple route and see how it sticks. The only encoding that the template spec itself does is for characters that are outside the range of those allowed in URIs. Everything else is application-specifc. So, for example, this variable; foo = "/thing?@ <--stüff-->" with this template: http://www.example.com/{foo} will...

this entry’s page (5 comments)

Wednesday, 20 June 2007

The State of Proxy Caching

A while back I wrote up the state of browser caching, after writing a quick-and-dirty XHR-based test page, with the idea that if people know how their content is handled by common implementations, they’d be able to trust caches a bit more. The other half that they need to know about, of course, is proxy caching; depending on who you listen to, somewhere between 20% and 50% of clients on the internet are behind some kind of proxy, and...

this entry’s page (13 comments)

Tuesday, 15 May 2007

Expires vs. max-age

I occasionally get a question from readers of the caching tutorial about whether to use the Expires header or Cache-Control: max-age to control a response’s freshness lifetime. Some people claim that Expires is better, because it’s defined by HTTP/1.0, whereas Cache-Control only came about in HTTP/1.1. Since there are still HTTP/1.0 agents out there, the reasoning goes, this is the safer path. The problem with that line of reasoning is that HTTP versions aren’t black and white like this;...

this entry’s page (7 comments)

Sunday, 29 April 2007

Squid is My Service Bus

The QCon presentation (slides) was ostensibly about how we use HTTP for services within Yahoo’s Media Group. When I started thinking about the talk, however, I quickly concluded that everyone’s heard enough about the high-level benefits of HTTP and not nearly enough details of what it does on the ground. So, I decided to concentrate on one aspect of the value that we get from using HTTP for services; intermediation, as an example. If your service is struggling to...

this entry’s page (6 comments)

Sunday, 15 February 2004

Caching Tutorial Update

I’ve published a revision of the Caching Tutorial for Web Authors and Webmasters, the first non-trivial edit in some time almost since I wrote it in 1998. That said, there aren’t any substantial changes; this is mostly tweaking and incorporation of new information. The bigger modifications include: - Replacing “Object” with “Representation” as appropriate - New information on Gateway Caches and Content Delivery Networks - Incorporation of various corrections and additional information received - Creative Commons licensing I’m sure there...

this entry’s page

Saturday, 28 June 2003

Caching is often enough

I feel compelled to respond to Norm Walsh's thoughts on caching. It's important to distinguish between the capabilities of a specific product (such as WWWoffle) and the technology that it implements (caching). I would agree that the general state of cache implementation leaves much to be desired, both in clients and in proxies. However, I would not write off the technology wholesale. For example, Norm claims that you can't populate a cache. Untrue; it's simple to populate any cache, simply...

this entry’s page

Monday, 5 May 2003

Tarawa

I've finally gotten sick enough of a project that I've been working on for waaaay too long to release it to the unsuspecting^H^H^H general public. Tarawa is, in short, a HTTP server API that tries to apply some of the lessons we've learned recently in discussions regarding the Web architecture and REST. This release is for Python; the API is currently very Python-specific, but I expect that to change. It's also very alpha, but should be functional for playing with....

this entry’s page

Tuesday, 8 April 2003

HTTP header sniffing

LiveHTTPHeaders for Mozilla is the best HTTP header sniffer I've seen yet; up till now, I've been using WebTee, but for *most* purposes, this is much better. Enjoy....

this entry’s page

Creative Commons