mnot’s blog

Design depends largely on constraints.” — Charles Eames

Saturday, 7 February 2004

Protocol Design

Messages vs. Files

Jon Udell is thinking about the benefits of data being globally available, rather than localised to a machine. I’m in complete agreement; in the last two years, I’ve used Linux, Windows and Mac OSX on the desktop, leading me to be ruthless about data portability.

However, I’m not sure how this leads him to this conclusion;

A general solution would require OSs that work like Groove, and applications that send messages rather than write files.

Why do messages — which in developers’ minds, inevitably means “short-lived” — have an advantage over files? Making your OS message-based seems to just add complexity, not make data more portable and ubiquitous.

I’m very happy that I can use a tool like Unison to move my files back and forth; if they’re messages that live somewhere in limbo, the system becomes much less hackable (in the good sense) to the average developer or sysadmin.

Thinking about messages puts the emphasis on behaviour, rather than data, even though time and time again, data has proven to be more long-lived, flexible and easier to mould to unintended purposes. Why throw that away?

UPDATE: A number of responses; Mark Baker, Patrick Logan and Sean McGrath.

Sean hits the nail on the head (as usual); it’s very true that messages can be thought of as documents, and vice versa, but if the fundamental model of a system isn’t working with documents, you have to reinvent a lot of infrastructure.

Filesystems, after all, are one of the few things that define and hold together operating systems; they allow for unintended uses of data. If you expose everything through a specialized API, everyone who comes into contact with the system — developers, administrators, and users — has to learn a new means of accessing it.

I know that you’re thinking “well, they’ll have to learn a specialized file format anyway,” but that’s not the point; they already have tools for working with files, and they can manipulate the files and see the results as they do so. Try that with an API; the best you can do is something like an interactive interpreter a la Python’s, and it’s still a much steeper learning curve.

Put another way, I can’t think of any good reason why you wouldn’t want to expose persistent state as a file. that doesn’t mean that there can’t be other interfaces, but why lock it up in them?

P.S. Sean, I’m not sure specialised RPOST/RPUT methods are necessary; I think it can be done with a pattern, or maybe a few extra headers.

One Comment

PJ said:

Messages are data. Ideally they’re platform-independent, which includes storage-method independent (they could be entries in a database or files in a filesystem or sheets of paper in a filing cabinet). In one sense, files in a filesystem are messages… from a past incarnation of an app to a future incarnation of itself or (hopefully, because we like open formats) some other app. Mr. Udell’s point about wanting an OS based on messages is, I think, that he wants to be able to reconstitue not only static state (ie. data files), but also running state… imagine your desktop writing out its running state into the cloud such that when you drop it on the floor and have to do a ‘restore from cloud’ you get your complete data state restored, including which windows you had open and where and how far into that mp3 file you had listened. One (fairly crude) way to implement this would be for your desktop to write files out into the filesystem and then essentially continuously run Unison to keep the ‘cloud’ up to date. More elegant would be a local queue that your desktop would write events to and that would watch your local datastore for changed files and would sync with the cloud as necessary.

Basically I think the two of you are in violent agreement. :)

Monday, February 9 2004 at 9:40 AM