mark nottingham

Profiles

Tuesday, 17 April 2012

HTTP APIs

Erik Wilde - otherwise known as dret - has published an Internet-Draft for a “profile” link relation type:

This specification defines the ‘profile’ link relation type that allows resource representations to indicate that they are following one or more profiles.

Why am I excited? Three words:

Media Type Proliferation

For better or worse, everyone and his dog is minting “RESTful” APIs. One byproduct of this is the need to identify formats, so they’re going off and creating new media types. Lots of them. Sometimes, for every variation of their syntax (e.g., if a new element is added) some people feel the compulsion to identify it with a media type. This quickly gets unwieldy, obviously, and registering that many media types won’t do. Enter the profile relation:

HTTP/1.1 200 OK
Content-Type: application/json
Link: <http://example.org/profiles/myjsonstuff>; rel="profile"
Connection: close

{
  "stuff": 0,
  "moar-stuff": 1
}

Here, a Link response header is used to say that the application/json content conforms to a particular profile. What that profile is is completely defined by the person who owns that link, so it can place constraints on the contents, advertise conventions, and so forth. This means that you don’t need to define and register a new media type for every message type in your RESTful API (a dubious combination of terms anyway); using a URI means that you can place constraints on the format, advertise extensions and compose them without using top-level media types (which were never intended for that anyway). Media types aren’t so good for this; because they’re a flat, highly-structured space, it’s difficult to describe variations in format with any nuance. Also, by defining a new media type, you lose information about the generic formatting conventions of JSON or XML — something that people are trying to shove into +json and +xml extensions.

Wait, Haven’t We Seen This Before?

Profile was born in HTML, where it’s used by microformats.

<head profile="http://microformats.org/profile/hcard">

Here, the hCard profile tells people consuming the HTML — a format that they already known about which is advertised by the media type — that it conforms to some additional constraints (in this case, a particular arrangement of HTML classes). In Microformats, you can combine multiple profiles to indicate that more than one profile is in effect in the document:

<head profile="http://microformats.org/profile/hcard
               http://microformats.org/profile/hcalendar">

Tell Me More.

The same ideas can be used in JSON formats, and once we figure out conventions for linking in JSON, we can embed the profile links inside the document too. Since doing so will leverage the concepts in Web Linking, if we do it right, that means that a profile can be applied to a context that’s part of a document — for example, one object in JSON. E.g.,

{
  "thing": { "a": 1, "b": 2 },
  "owner": {
      "_profile": "http://example.net/name",
      "firstName": "Bob",
      "lastName": "Roberts
  }
}

This has some pretty cool implications for collections in JSON — more on that soon. Note that using link conventions in a document is a profile itself. With profiles, you can say that the http://example.com/b profile extends the http://example.net/a profile, or you can mint a new http://example.org/c profile that combines them; it’s up to you. You could even specify query parameters to control variability in the format, if you really want to.

But, What Should a Profile Be?

Since it’s a URI, it’s possible to dereference a profile URI — just like an XML namespace document. I suspect that it’ll become useful to define a lightweight document format describing a profile, its relationships to other profiles, etc. I’m not sold on pointing to a schema yet (JSON or otherwise). This is kind of the nice thing about profiles; because they don’t require a specific form, you can make them mean what you want to. The challenge, I think, is to come up with conventions and tools that promote good practice and reinforce the Web, rather than turning this into just another layer of media types, or something like WSDL. Erik is still working on his draft, and people (including me) are giving feedback.


21 Comments

http://openid.elfisk.dk/jornwildt said:

I think profiles work well when servers want to describe their content to the client, just as you describe here. But it doesn’t work well the other way around. There is no way for the client to request a specific profile from the server when interacting with the resource.

Thus the client and server cannot evolve independently of each other; the client cannot be improved and ask for a newer and richer profile, and the server cannot make breaking changes to the JSON format without breaking older clients.

Had a media type been used instead then content negotiation would have solved the problem.

It is of course possible to invent an “Accept-Link-Profile” header for profile negotiation, but why, when its a solved problem with media types?

Having said that I must admit that I am still in doubt about how to avoid media type bloat. Well, I don’t even know if media type bloat is a real or perceived problem.

/Jørn

Tuesday, April 17 2012 at 5:23 AM

James M Snell said:

I’m certainly a fan of this approach but it would be good to reconcile this with the “implements” link relation I proposed previously that is still active ( https://datatracker.ietf.org/doc/html/draft-snell-additional-link-relations-01.html). Their purposes overlap significantly. It would be great to have a single relation for both. Also, i’m curious why it wouldn’t be more preferable to include the profile as an optional parameter in media type.. E.g. application/json; profile=”http://…/a”… This allows the profile detail to flow more naturally with the content.

Tuesday, April 17 2012 at 7:42 AM

James M Snell said:

Was thinking about it further once I got home and posted some thoughts here: http://chmod777self.blogspot.com/2012/04/on-profile-link-relation.html … basically, while I can see where this is going, and agree with the basic premise and need, I don’t think the profile link, as defined, is the right approach to take. As always, I’m open to being convinced otherwise, but I don’t see this actually solving the problem of proliferation that you’re warning about.

Tuesday, April 17 2012 at 9:55 AM

Erik Wilde said:

so it seems that application/json;profile=”http://…/a” as suggested by james (and i like the idea) is a no-go because media type need to register parameter names for those parameters to be permitted. at least that’s what mark said ;-)

but the ‘profile’ draft could (and maybe should) be updated to say that for new media type registrations, authors should consider to register a profile parameter for the media type, and then declare that URIs appearing in that profile parameter or on a ‘profile’ link should be treated as equivalent. and then we probably need some language specifying what to say is multiple of these profile values are present and they conflict.

Wednesday, April 18 2012 at 2:11 AM

Manu Sporny said:

Is there a time-frame on this link relation being standardized? As in, when would it be stable?

We are currently considering application/ld+json for the JSON-LD document MIME type and potentially application/ld-frame+json for JSON-LD frames:

http://json-ld.org/spec/ED/json-ld-syntax/20120318/#iana-considerations

We were also going to use this to pull in JSON-LD contexts that are out of band:

Link: <http://example.org/contexts/a>; rel=”describedby”

but would be happy to use this instead (keeping application/ld+json as the MIME type):

Link: <http://example.org/contexts/a>; rel=”profile”

So, there are three things we are contemplating: 1) JSON-LD is a fully compliant sub-set of JSON - a valid JSON-LD document is a valid JSON document - should we register a new application/ld+json MIME type for it? and 2) JSON-LD has the concept of a “context”, which tells JSON-LD processors how to interpret the JSON keys (an some values) in the document - it seems like the new profile relation might be a better fit for that? and 3) JSON-LD frames are used to re-structure the JSON-LD document into a well-known form expected by the developer so that an algorithm can be run against the re-framed/structured JSON-LD data - we were going to use a MIME type for specifying a frame via HTTP, but could possibly use a profile instead?

It seems like we’d want to do #2, but achieve #1 and #3 via the creation of new MIME types. Thoughts?

Wednesday, April 18 2012 at 3:11 AM

Mark Baker said:

The proliferation of media types is not a problem! The problem is a proliferation of formats.

The only way I know how to solve that problem is by reusing the generic, extensible formats, like HTML, Atom, RDF/*.

Wednesday, April 18 2012 at 3:41 AM

James M Snell said:

Looking over RFCs 2046 and 4288, I see nothing that requires that parameter names have to be registered before they can be used. If a media type registration includes optional or required parameters, it must fully define those parameters and include them in the registration, but there’s nothing that says additional unregistered parameters cannot be used. On the contrary, RFC 2046 explicitly states that unknown parameter names are simply to be ignored, and RFC 4288 states only that parameters must not be defined in a way that changes the semantics of the base content type; they can only provide additional information about it. Now.. in practice, the story may be rather different, but that’s what the specs themselves say… meaning, in theory, we defining a new profile parameter for application/json and application/xml would be feasible.. particularly if defined within an Internet-Draft that eventually made it’s way to an RFC (note that RFC5023 added the optional type parameter to the application/atom+xml media type defined in RFC4287 so there is precedence for adding parameters after the fact).

Wednesday, April 18 2012 at 5:14 AM

Erik Wilde said:

thanks, @jasnell, for digging into the actual spec texts. it never fails to amaze me how much exegesis is part of this whole spec game… anyway, now that @mnot claimed we cannot just tell people to use profile media type parameters, and you have said the opposite, i probably have to do the digging myself, too. and i am pretty sure it comes down to how to interpret certain text passages and words, that’s at least what often happens. but regardless of my own exegesis exercise, it would be really nice and useful if we could do that. the only potential problem is that some media type already may have a profile parameter, and then we would have a conflict, but i am pretty sure that how to handle that conflict would be up to exegesis as well.

@manusporny, it all depends on how vocal people are to support this idea and provide constructive comments, and how much pushback we might get. pushback so far has been minimal, so things look not too bad. i’d be interested to try to push this pretty fast, but you never know with IETF things, suddenly things happen and then, one year later, you’re wondering what was going on…

Thursday, April 19 2012 at 5:33 AM

Erik Wilde said:

@mnot, using a media type profile parameter might be a particularly interesting and useful place to serialize this, because of the visibility on the HTTP layer itself. so if media type parameters can be bolted on retroactively, then maybe we should at least discuss that option and what it might give us and what the “MIME guys” have against it.

Thursday, April 19 2012 at 5:40 AM

Jan Algermissen said:

Hi Mark N,

I am with Mark B on this one: http://www.markbaker.ca/blog/2008/02/media-type-centralization-is-a-feature-not-a-bug/

Proliferation of types is an inherent problem which is addressed by a central review process / by mediation thourgh discussion. Shoving the problem elsewhere (e.g. to profile identifiers) does not make the problem go away.

It would only lead to proliferation of profiles.

Jan

Wednesday, April 25 2012 at 7:01 AM

Jan Algermissen said:

Hi Mark,

the discussion is twofold I guess. I do not think that enhancing ‘host’ media types (e.g. HTML) with ‘sub’ formats (e.g. hCard) is the way to go in order to enable semantically richer interactions between systems.

For example, I do not think that representing an order list as an Atom-feed with application/foo.order-typed entries. If my domain inherently makes use of the notion of order lists I definitely think I need an order list type[1]. Otherwise there will be out of band knowledge.

However, it is certainly a nice thing to be able to embed stuff into more generic formats and to tell consumers about it using a profile mechanism. No doubt there.

I just do not think it solves a media type bloat problem we get once semantically richer applications are to be created using HTTP. The media type centralism in HTTP (well, REST) is the feature that allows the server to evolve. Because it removes ownership of the contract from the server owner to a central space.

The rest of the puzzle is IMHO left to expertise, review, discussion and feeding stuff that has been tried out and appeared useful back into the central specs. Removing the need for contract centralization essentially means to take away the benefit of exactly that feedback.

Am I making some sense at least?

[1] Which can be part of a ‘domain type’ such as application/procurement

Thursday, April 26 2012 at 1:39 AM

James M Snell said:

Mark, I’m largely in agreement with you on that last comment, but I don’t think proliferation of profile URIs that are independently of media type will actually solve the problem… all it does it shift it from one relatively rigid extensibility point to another that is somewhat less rigid. The more I mull it over, in fact, that more I’m thinking that a revamp to the mime media type system – like what I was stewing over in my blog posts – is what really needs to happen to address the real challenge. However, that kind of change would be a major challenge to take on – not an impossible one tho if we had people committed to making it happen.

Friday, April 27 2012 at 3:59 AM

James M Snell said:

Then actually solving the problem of type proliferation is not actually the goal here as much as it is attempting to allow the proliferation to happen in a somewhat more flexible way and hoping, somehow, that doing so makes the overall situation better.

So, let’s call this exactly what it is… a mechanism for identifying the additional semantics that have been applied to a particular resource representation. Uncontrolled proliferation would still happen, but it would manifest in Link headers and link fields rather than in media types. And, it would be done in a way that eliminates the possibility for profile-based server-side conneg since there’s no Accept-Link-Profile type of mechanism defined.

Friday, April 27 2012 at 5:03 AM

Jan Algermissen said:

Hi Mark,

to me, the focus of media types is on global (as opposed to server-owned) contract. Even if a media type originates from a service provider, the very fact to establish it as a media type is a commitment to not change the contract at will as one will usually do with a server-owned API spec. This is why I think media types are a superior mechanism to minting profile URIs.

That said, maybe your concern can be addressed by reusing existing syntaxes (HTML,Atom,…) and simply packaging them with Joe’s domain specific constraints (e.g. use of certain micro formats or making the presence of certain links mandatory). Such combinations constitute contracts that can easily named with a media type identifier.

This comes with the additional benefit of having all the parsing tools ready.

Jan

Friday, April 27 2012 at 6:53 AM

Mike Schinkel said:

@Jan - Definitely agree about media-types being global. OTOH, the problem with existing syntaxes is that many of the well know ones are really complex (HTML and Atom, to name too) and to use them potentially adds a huge amount of complexity for prospective clients.

However, there is an existing syntax that is really simple, easy to grok, and not difficult for prospective clients: JSON. Funny that, it’s basically become the defacto-standard.

Maybe what’s needed are some more well-known media types that add just one or two facets to JSON for specific use-cases that are designed to be reusable like JSON is reusable vs. be-all-end-all formats like Atom. Atom is great, but it’s not a format for ad-hoc parsing when time budgeted in the project to write the code to interface is about one afternoon.

Wednesday, July 4 2012 at 5:01 AM