mnot’s blog

Design depends largely on constraints.” — Charles Eames

Thursday, 20 March 2008

Moving Beyond Methods in REST

Having complained before about the sad state of HTTP APIs, I’m somewhat happy to say that people seem to be getting it, producing more capable server-side and client-side tools for exposing the full range of the protocol; some frameworks are even starting to align object models with resource models, where HTTP methods map to method calls on things with identity. Good stuff.

However, something’s been bugging me for a long time about this. While there’s a nice internal logic to mapping HTTP methods to object methods, it doesn’t realise the power of having generic semantics.

Consider a resource;

class Person (Resource):
    def GET(self):
        # do acls...
        # get the representation out of some persistent store
        # translate to the format asked for
        return representation
    def PUT(self, representation):
        # do acls...
        # translate the representation to the appropriate format
        # put the representation into some persistent store
        # cook up a status message
        return status_representation
    def DELETE(self):
        # do acls...
        # delete the resource from some persistent store
        # cook up a status message
        return status_representation
    def POST(self, representation):
        return representation

In this interface, GET, PUT and DELETE all have well-defined semantics. So well-defined that they really shouldn’t need application-specific code; after all, they’re just manipulating state in well-known ways.

In fact, I’d posit that you can specify the behaviour of any RESTful resource by describing a) the processing that POST does, and b) any side effects of PUT and DELETE.

There are a lots of caveats around that, of course. You need to define access control for the methods, and specify if authentication is required. You need to specify the formats that the application can work with (potentially with differing answers for input and output, and possibly with appropriate translations). You’ll need to specify the processing that happens around query parameters (e.g., in filtering output for GET).

The thing is, none of those have an implementation that’s specific to this particular resource; instead, they’re better abstracted out, so that the implementation looks something like this;

@store_type("mysql") # tell the Resource what implements GET, PUT and DELETE
@acl("choose your ACL poision") # tell who / when access is allowed, per-method and finer-grained
class Person (Resource):
    store_format = PersonML
    def POST(self, representation):
        # operate on the store...
        return representation
    def PUT_effect(self, representation):
        # called IFF the presented representation is storable, 
        # but before it is available; raising an exception will back it out
        return status_representation
class PersonML(Format):
    translations = {
        'application/xml': (self.to_xml, self.from_xml),
        'application/json': (self.to_json, self.from_json),
    }
    def to_xml(self, native_input):
        # do whatever you've got to do
        return xml_output
    def from_xml(self, xml_input):
        # do whatever you've got to do
        return native_output
    ...

I haven’t incorporated a way to handle query parameters here, but you get the idea.

The tantalising part of this approach is that it can be implemented close to your persistence layer; all you need is hooks in the right places for side effects and POST processing. In fact, as long as you’re willing to be flexible on consistency, you can almost do it with mod_rewrite (calling a completely separate script as well as PUTting/DELETEing the state doesn't yet seem to be possible; ping me if you can figure out how to), and stuff like Apple's FSEvent looks very, very interesting in this light.

Is anybody aware of anything along these lines out there in an existing tool or framework? I’ve been meaning to write some code along these lines for some time; if you'd like to help out, please drop me a line.

The other place that this view has impact is in describing RESTful applications, if you believe in doing such things. WADL, for example, gives GET PUT and DELETE equal weight with POST, when I’ve always suspected it would be more elegant if you took the abstraction up a notch and talked about state, rather than methods.


Filed under: HTTP Protocol Design Web Web Services

11 Comments

Mike Amundsen said:

I fully agree. Direct HTTP Methods are the way to go - not mapping to arbitrary functions. I've been working on a framework that does that. It's built using C# on Windows, but the pattern is generic enough to be easily ported.

- Base classes with HTTP Methods as functions
- URI routing at the class level
- support for PUT w/ ETags/If-Modified-Since (including creates)
- support for media-types (via conneg) at the class level
- support for caching and cache-invalidation
- support for HTTP Auth (uri+user+method)

Check it out @ http://exyus.com and http://exyus.googlecode.com

I'd appreciate any comments/feedback.

Thursday, March 20 2008 at 3:45 PM +10:00

Eric Allam said:

Your basically talking about a Rails app using a RESTful plugin such as make_resourceful or rest_controller.

Thursday, March 20 2008 at 4:21 PM +10:00

Mark Nottingham said:

Eric,

Looking at some search results on those, I still see some HTTP methods sprinkled here and there, and it looks like they actually build a higher-level interface on top of REST (something very CRUD-like, with additions like 'index') rather than boiling it down to fewer (not more) things to do. I'm not saying that's not useful, it's just not what I'm talking about here, really.

Mike - thanks, will take a look!

Thursday, March 20 2008 at 4:57 PM +10:00

Pete Svensson said:

Hi Eric,

Have you seen if the JCR project Apache Sling can handle these things through server-side scripting? I'm a little bit involved in the project, but not enough to make a definite statement.

Also, I recommend to check out the SMD specification that will be part of Dojo 1.1, when it comes to describing web services in a restful way;
http://www.sitepen.com/blog/2008/03/19/pluggable-web-services-with-smd/

Cheers,
PS

Thursday, March 20 2008 at 8:18 PM +10:00

Jeffrey Winter said:

All the method definitions should really include a Map of HTTP headers.

Thursday, March 20 2008 at 11:17 PM +10:00

Chris Dent said:

This looks really good.

I've been approaching something similar (very basic methods, separate transformer modules or classes) and it is resulting in some very simple code that is easy to understand and just as importantly easy to test.

Where my stuff differs is that I just cannot get down with directly tying URIs to objects or classes. I'm not really sure why. In some ways it makes perfect sense especially when there is confidence that it's gonna be HTTP the whole way.

But in other ways it feels like I'm making a significant coupling that limits reusability. So rather than dispatch to "data" classes, I tend to dispatch to modules with functional controllers that _use_ "data" classes. The major benefit with this in my current experimentation is that are more entry points for testing.

Thursday, March 20 2008 at 11:29 PM +10:00

Robert Thau said:

The Rails REST-level stuff is actually a superset of what you've got. Your methods are roughly what Rails expects the controller to provide for a singleton resource. However, some resources naturally come in collections, which is where the extra complication comes in; in those cases, 'index' gives you a list, etc.; that's where the plugins become helpful in supplying the boilerplate. Note also the support for nested resources (e.g., all the comments on a particular blog post).

One thing you might want to consider, regardless, is creation of new resources, e.g., a new entry on a blog. (Individual entries might be resources associated with URLs like /entries/367, /entries/368, etc.) In the REST style, new resources are typically created by POST. But when you're adding a new entry, you're obviously not POSTing to any existing one. Rails handles this as a POST request to the URL associated with the collection of blog entries (in this simple case, that's just a POST to /entries), which typically turns into a :create message to the entries controller.

Friday, March 21 2008 at 12:26 AM +10:00

Justin Sheehy said:

"I've always suspected it would be more elegant if you took the abstraction up a notch and talked about state, rather than methods."

I couldn't agree more.

"Is anybody aware of anything along these lines out there in an existing tool or framework?"

Yes.

A couple of months ago, my little team created something called "webmachine". It's all about resource state in the HTTP sense.

However, we don't do it in the way that your sample implementation indicates, of making PUT and POST explicit and everything else (like auth,conditional requests, conneg, and so on) a separate decoration. Instead, a resource definition module implements a set of functions describing the answers to various questions about state, which are called by webmachine in different order and combination depending on the request and on the known state based on preceding functions called on this request. Examples of such functions are get_content, resource_exists, authorized, etag_matches, acceptable_content_type, and so on. You only implement the functions relevant to your resource, as defaults are provided for anything left out; get_content and resource_exists are all one really needs to implement a basic GET-only server.

We've built a production application on top of it which we'll be selling soon. Webmachine isn't the product, it was an enabler so that we could write well-behaved web applications quickly in a way that didn't feel wrong.

We've planned all along on making it open-source and I will be doing so (and writing more about it) very soon. Mark, if you want to discuss it pre-release feel free to send me email.

Friday, March 21 2008 at 6:58 AM +10:00

Erik Johnson said:

Nice post! I have 2 points. First about state, you have to decide what (if any) constraints a particular state places on representations. In other words, I can set (or change) the customer ID on an unbooked order but not on a confirmed (or cancelled) order. Let's assume the system decides to be an ass and doesn't let you include values in representations you are not allowed to set. That means that different states for order 123 have different payload formats. If the resource contains read-only data, then you wind up with asymmetry between GET and PUT for the same resource/state.

This is all fine with me, except that I've never gotten a good feeling that GET/PUT asymmetry for the same URI is kosher REST. FWIW, we decided that transitioning states should be done using PUT to a state-specfic URI with no representation in the payload. There was a programming-model reason for this. It clarified the business intent and avoided developer confusion in sorting out side-effects of changes relavent to one state vs. another. So, you can't change the customer and cancel the order at the same time.

My second (and WAY more obtuse) point is about your observation about WADL. If resources contain links to other resources in any normative way, your app might want to give callers a way to navigate those links up front. In WADL (and other things like it), the URI template schemes seem too monolithic in the face of URI address spaces that have compositional features. Applications that let callers retrieve resources through relationships are a good example (e.g., uri://order(123)/line(1)/item/part-class/gl-account/business-division).

No one has yet invented a URI template scheme mapped to an information model (of any sort) -- only a list with wildcard substitutions. But if such an animal comes forth, wouldn't that affect the design of your programming model?

Friday, March 21 2008 at 11:29 AM +10:00

Mark Baker said:

For a project a few years ago, for handling RDF (XML & Turtle), I wrote an RDFServlet which handled GET, PUT, and DELETE itself - in the framework - and only required that the developer provided the logic for POST.

Saturday, April 5 2008 at 8:35 AM +10:00

Jerome Louvel said:

Hi Mark,

In the Restlet framework (http://www.restlet.org), we use a very similar approach. There is a base Resource class that you extend and where you declare the available variants.

When a request targets your resource (via template based URI routing), a new instance if created. At that time the underlying persistence store can be access to load the state of the resource (or be prepare to load it).

For a GET, the engine will select the best variant via content negotiation and call you resource instance back, asking it to represent itself for the given variant.

Our Resource API looks like this:
- represent(Variant) : Representation <- GET/HEAD
- storeRepresentation(Representation) <- PUT
- removeRepresentations() <- DELETE
- acceptRepresentation(Representation) <- POST

At this level of abstraction in our API, we tried to not focus on the method name but on the actual semantic related to the state of the resource instead (store, remove, represent, etc.).

See the Javadocs for details:
http://www.restlet.org/documentation/1.1/api/org/restlet/resource/Resource.html

See this tutorial for a more complete explanation:
http://www.restlet.org/documentation/1.1/firstResource

Best regards,
Jerome

Sunday, October 19 2008 at 3:43 AM +10:00

Creative Commons