mark nottingham

Watching WADL (and other rambling thoughts)

Monday, 21 January 2008

HTTP APIs

I’m following the discussion of RESTful Web description in general, and WADL in particular, with both difficulty and interest (see Patrick and Joe’s thoughts for a nice contrast).

Difficulty because there’s so much of it, and it’s hard to give each piece the attention it deserves. Interest because it goes to the heart of the harder parts of REST — e.g., “hypertext as the engine of application state.” On top of that, I was one of the people pushing (or at least noodling) on Web description way back when, and did my best to help Marc get WADL into a usable state when it was under more active development, so I have a personal interest as well.

What Do Clients Do?

The first hurdle that a RESTful description format faces is probably the biggest; how it’s used by clients. My experience is that WADL provides most of its value on the server-side (e.g., for in-development service modelling, documentation, and review, as well as limited code generation), but much discussion keeps on circling around to the client side, perhaps because of the well-worn footpath off the cliff that WSDL provided.

If clients use a WADL file to generate static code that calls the described service without checking to see if the WADL has changed, they’re going to be tightly coupled to the WADL definition, and therefore no better than WSDL or any other interface description. Blech.

On the other hand, if you use the WADL at runtime to dynamically create the URLs and representations you send and parse the ones you receive, it’s all good, and in this way WADL is acting like a Web format should — using hypertext (in this case, a generic XML format) as the engine of application state. In this way, it’s no different than the APP service document format or HTML forms, except that WADL is less application-specific in the first case, and more flexible in the latter.

So, perhaps a good way to get this idea into people’s heads is to remember that it’s only good for as long as the response that contained the WADL file is fresh, in the HTTP caching sense; just as an HTML form has to be updated when it becomes stale, so does a description file. Furthermore, you can still follow a link and get a 4xx or 5xx error, which trumps the description every time.

Whether that gets through remains to be seen; client-side WADL-aware tools are few and far between, and the legacy of WSDL hangs over WADL darkly.

Metadata and Policy

Another of the big hopes I had for WADL was as a means to convey metadata and policy, much like robots.txt, p3p.xml, site maps and so on do today, but with much finer granularity. This would not only benefit clients, but also intermediaries; for example, Akamai uses URISpace to give their network more information about origin servers.

So far, this hasn’t eventuated; there doesn’t seem to be much in the way of WADL extensions.

I do notice, however, that the concept of a resource type does seem to be getting currency in a variety of places, including in the Semantic Web world. It would be interesting to evaluate WADL as just a format for defining a single resource type, which is declared with a URI in a response header; that leads to lots of interesting things, I think. Timely enough, Marc has talked about Jersey doing pretty much just this. Hmm indeed.

Formats are Hard, Remember?

The fly in the ointment for all the folks that want all-singing, all-dancing client-side dynamic binding is describing the formats. XML Schema just isn’t up to the task, and I have my doubts about Relax NG. People will keep chipping away at the generic data format problem, but I don’t hold high hopes here, and in the end, this is the real limitation of using WADL — or any other description language — on the client side.

In particular, you’d need to establish a common understanding of the data model between the parties communicating that’s separate from the serialisation, and bind that to a representation at runtime. While this is possible, all of the tools that I’ve seen do it at design time, baking the schema into the code (please pipe up if you know differently, please). That’s because most of the tools in this space are for heavyweight static languages.

WADL’s XPath shortcuts for variable bindings are attractive because of this, but unfortunately they’re one-way; they tell you how to parse a message, but not how to create one. Some form of templating might help, I suppose.

Coming from the other direction, another RESTful constraint is to have a limited set of media types. This worked really well for the browser Web, and perhaps in time we’ll come up with a few document formats that describe the bulk of the data on the planet, as well as what you can do with it.

However, I don’t mean “XML” or even “RDF” by “format” in that sentence; those are the easy parts, because they’re just meta-formats. The hard part is agreeing upon the semantics of their contents, and judging by the amount of effort its taken for things like UBL, I’d say we’re in for a long wait.

Two Roads to RESTful Services

Because of this, I think there will be two ways to get a (somewhat) RESTful Web service into the world, for the foreseeable future.

One (I’ll resist calling it the “high” road, as we’ve been there before) will be to work with a group of people to identify a broad problem space, standardise one (or a few) media types, defining their semantics, and an application-specific format that glues them together into an interface. Atom Publishing Protocol is a great example of this, and it certainly has legs.

The other will be to skip the huddle, define your own formats and semantics, and throw it over the wall, knowing that you’ll get your problem solved in the short term, but without the considerable leverage of a widely adopted and understood format and interface.

To be clear, I’d encourage people to use the former route wherever possible (sometimes going so far as to stretch the intent of existing formats, a la uFormats), but in some cases that just doesn’t work. While the world will eventually see a [insert esoteric problem domain] format standardised some day if there’s good cause, it’s not fair to ask the people who need it now to wait that long. Hell, I’d like a standard for blog templates, so that Moveable Type and ecto can use the same thing, but despite Atom’s focus on blogs, we don’t have that yet (and there are good reasons why that’s so).

It’s also important to recognise that standards don’t just appear magically when lots of people get together around a blank piece of paper; as in the greater world, evolution is the best path to good design, and the second road acknowledges this by allowing experimentation and small-scale deployment and testing.

So, the real question at hand is whether it’s a good thing or not to give people tools to help them down that second path, and whether WADL in particular gives enough of a leg up to be worthwhile.

I’ve seen lots of developers using WADL to design and document their resources to good effect, so my gut feeling is “yes.” However, that value could be cancelled out if some client-side static binding tools come around the corner. Because of this, and since no “good” client-side tools exist yet, I continue to see no need to publish a WADL description yet, lest I encourage people to drive without looking out the window.