Yet More New HTTP Specs
Wednesday, 8 June 2022
The HTTP “core” documents were published on Monday, including a revision of HTTP semantics, caching, HTTP/1.1, HTTP/2, and the brand-new HTTP/3. However, that’s not all that the HTTP community has been up to.
Today the RFC Editor published another set of HTTP specifications. They reflect a variety of work that’s happened over the last few years; publication was held up by references to the core work, so many of these already have active implementations. Let’s take a quick look.
As always, see the HTTP Working Group specifications page for a full listing of relevant HTTP documents and links.
Building Protocols with HTTP
HTTP isn’t just used with Web browsers, of course. Back in the early 00’s, the IETF saw enough proposals for new things built on top of HTTP – e.g., WebDAV, CalDAV, CardDAV, IPP – that it decided to write down some principles as “On the Use of HTTP as a Substrate”, referred to as BCP56 (for “Best Current Practice”). This was such a big topic of discussion that there were also joke RFCs about it.
Since then, however, use of HTTP for APIs and similar things has exploded, Roy’s dissertation popularised the principles of REST, and even more IETF protocols started being built on top of HTTP.
The advice in BCP56 wasn’t particularly relevant or even useful any more. So, a few years back the Working Group began to write a replacement, now published as RFC9205: Building Protocols with HTTP.
This document covers things like how to refer to HTTP in your document, specify client and server behaviour, use URLs, methods, status codes and header fields, and coexist with browsing safely.
The new BCP56 is still focused on advice for IETF specs that use HTTP, but if you’re using HTTP in other standards bodies or even just building an API, it might be relevant to your work too.
Extensible Priorities
The defining feature of HTTP/2 is the ability to have multiple requests on a single connection at the same time. While that makes the protocols’ use of the network much more efficient, it also requires the server to know how to prioritise the different outstanding responses to send, because that can impact perceived performance. For example, if a server sends images before fonts and stylesheets, it might block rendering.
While a server can use heuristics (for example, “send stylesheets and javascript first”) to prioritise responses, that only gets you so far. If the client can send hints about what should be prioritised based upon its needs, that allows much finer-grained decisions to be made, and leads to better perceived performance.
HTTP/2 built in a prioritisation scheme that was tree-based, and it was quite capable; not only could it fully represent the graph of priorities between different exchanges on the connection, it could even allow an intermediary to combine different client connections into a single forward connection while preserving the relative priorities of each client.
This scheme was capable, but complex – it wasn’t implemented or used widely, and confused a lot of people. Even when we designed it, the Working Group wasn’t sure that it was the right solution; in the end, we decided between it and a simpler scheme based on the outcome of a coin flip, because we didn’t have enough information about what the right path forward was.
When HTTP/3 came along, it became clear we needed a new prioritisation scheme – not only because of the complexity and adoption problems, but also because HTTP/2’s prioritisation scheme couldn’t be easily ported to the new protocol.
The result of a lot of work in this area is RFC9218: Extensible Prioritization Scheme for HTTP. This is a simpler but extensible way for servers to send priority hints to clients, and it works both in HTTP/3 and HTTP/2, since the recent update to HTTP/2 deprecated its prioritisation scheme.
Read more from Lucas Pardue – one of the specification’s authors – in his blog entry about it.
Cache-Status
Figuring out how caches treat your content can be frustrating – especially when there are multiple layers in CDNs and reverse proxies. Mehdi Daoudi, Catchpoint’s CEO was so frustrated he blogged about it in 2017, saying “Each CDN out there uses different debug headers and some don’t send anything back.”
While there are a number of ad hoc headers (e.g., X-Cache-Status) out there, they aren’t well-defined and often omit a lot of critical information. The obvious thing to do, then, was standardize a header that is well-aligned with HTTP’s caching model.
RFC9211: The Cache-Status HTTP Response Header Field does just that. For example:
Cache-Status: ExampleCache; hit; ttl=376
tells you that there was a hit in ExampleCache and that there’s 376 seconds of freshness remaining.
Tim Perry from HTTP Toolkit writes more about it in his blog entry. It’s not certain that this will get adoption, so if you’re a CDN or reverse proxy customer and care about debugability, agitate with your vendors for support.
Proxy-Status
The companion to Cache-Status is RFC9209: The Proxy-Status HTTP Response Header Field. As multiple layers of intermediation have become more common in HTTP, it’s become important to understand what each “hop” does with the message. Proxy-Status allows this, with particular focus on disambiguating where and why error messages were created.
For example, this response tells you who generated the 504, and why;
HTTP/1.1 504 Gateway Timeout
Proxy-Status: ExampleCDN; error=connection_timeout
Again, broad vendor support is needed to see the value from this header. If you think it’d be useful, please tell them.
Targeted Cache-Control
It’s pretty common for CDNs and reverse proxies to have special control headers for their caches, so that servers can target directives just at them (unlike Cache-Control, which applies to all caches on the path).
An early attempt to define a common way to do this was the Surrogate-Control header. However, it wasn’t well-specified, is arguably over-complicated, and wasn’t widely adopted.
So, RFC9213: Targeted Cache Control shows how to define a new header field with the same semantics as Cache-Control, but targeted at a single entity (or kind of entity). It also defines the first targeted field, CDN-Cache-Control, which looks like this:
CDN-Cache-Control: max-age=3600, stale-while-revalidate=60
This specification seems to be getting good take-up already, with blog posts from Cloudflare and Akamai describing their support. Since the spec allows other such headers to be defined, we might also see new ones for reverse proxies or even individual implementations, or single CDNs.
Bootstrapping WebSockets with HTTP/3
RFC9220: Bootstrapping WebSockets with HTTP/3 is a small but very necessary specification that describes how to use a HTTP/3 connection for WebSockets. It’s a complement to the equivalent RFC for HTTP/2.
Expect-CT
Finally, RFC9163: Expect-CT Extension for HTTP describes a mechanism to help clients conform to new Certificate Transparency requirements in browsers. Scott Helme wrote in depth about Expect-CT a while back, but it’s probably not something that will see much use; as author Emily Stark mentioned on Twitter, those requirements are now firmly in place, so it’s not as useful any more. Really, this RFC is just there for documentation.