Web Entries
Tuesday, 4 December 2012
One of the most vexing problems that still seems to be facing people when I talk to them about HTTP APIs is how to handle versioning and extensibility — i.e., how they evolve. I tend to think about this a lot and talk to quite a few people about it, since I’m intimately familiar with the approaches to versioning that the HTTP protocol itself takes, and with the general attitude taken to it in the broader Internet architecture by the...
Wednesday, 5 September 2012
A common problem for APIs is partial update; when the client wants to change just one part of a resource’s state. For example, imagine that you’ve got a JSON representation of your widget resource that looks like: { "name": "abc123", "colour": "blue", "count": 4 } and you want to update the “count” member’s value to “5”. Now, you could just PUT the entire thing back with the updated value, but that requires a recent GET of its...
Saturday, 4 August 2012
The HTTPBIS Working Group is in a transitional phase; we’re rapidly finishing our revision of the HTTP/1.1 specification and just getting steam up on our next target, HTTP/2.0. So, we met in Vancouver this week as part of IETF84. Here’s a quick summary. HTTP/1.1 Five years ago, we held a Birds of a Feather meeting at IETF69, discussing whether we should attempt to revise RFC2616, the specification of HTTP/1.1. It’s turned out to be a massive job, but we’re close...
Tuesday, 17 April 2012
Erik Wilde - otherwise known as dret - has published an Internet-Draft for a “profile” link relation type: This specification defines the ‘profile’ link relation type that allows resource representations to indicate that they are following one or more profiles. Why am I excited? Three words: Media Type Proliferation For better or worse, everyone and his dog is minting “RESTful” APIs. One byproduct of this is the need to identify formats, so they’re going off and creating new media types....
Saturday, 14 April 2012
When you’re designing HTTP APIs, you need to keep a lot of concerns in mind. Stealing a page from XP, let’s look at some possible personas and their user stories for HTTP-based APIs: Joe at HugeCorp Let’s start with the people who consume the APIs. Joe uses Java (or some similar statically typed language). He’s used to having good tool support, with IDEs that manage all of the sticky details for him, and language features that keep him out of...
Friday, 13 April 2012
When people create HTTP APIs, one of the common decisions is about what format to use, usually revolving around “JSON or XML?” The thinking often goes like this: JSON is simple, easy to use, and “cool”; clients using dynamic languages will love it BUT, many people (especially those using static languages) are invested in XML So, I’ll just support both! Unfortunately, it’s not that easy; just because HTTP allows you to negotiate for formats doesn’t mean it’s a good idea...
Friday, 25 November 2011
To be a full-fledged format on the Web, you need to support links -- something sorely missing in JSON, which many have noticed lately. In fact, too many; everybody seems to be piling on with their own take on how a link should look in JSON. Rather than adding to the pile (just yet), I thought I'd look around a bit first. What am I looking for? Primarily, a way to serialise typed links (as defined by RFC5988, "Web Linking") into...
Friday, 21 October 2011
More than ten years ago, I was working at Akamai and got involved in the specification of Edge Side Includes (ESI), sort of a templating language for intermediaries. In that time, interest in ESI has grown, waned and been reborn. As far as I can tell, it's implemented not only by Akamai and Oracle (the main forces behind it), but also in Varnish, Squid, and lots of other places too. Back then, I had a strong suspicion that it'd die because...
Wednesday, 12 October 2011
Since joining Rackspace to help out with OpenStack, one of the hot topics of conversation I’ve been involved in has been extensibility and versioning. I think most of my readers (yes, all six of you) are fairly familiar with, if not tired of (hi, Dave!) the various arguments and counter-arguments in this space. However, there is one new-ish bit; how to do distributed extensibility in JSON. That’s because OpenStack’s API allows vendors to add extensions in various ways, in an...
Friday, 2 September 2011
HTTPbis published RFC6266 a little while ago, but the work isn’t finished. This is the RFC that clarifies how the Content-Disposition header is used in HTTP; in a nutshell, while basic file downloads worked OK, there wasn’t any broad interoperability between browsers for non-ASCII filenames. Julian Reschke did the hard work of coming up with a test suite to find how how bad things were, filing bugs with browsers (such as here and here), and finally writing the draft that eventually...
Sunday, 28 August 2011
In discussing my whinge about AppCache offline with a few browser vendory folks, I ending up writing down my longstanding wishlist for making browser caches better. Without further ado, a bunch of blue-sky ideas; Cache Contexts Most current HTTP caches — whether browser or proxy — put everything they cache into a single “bucket” that shares configuration and storage space. While that makes sense for a HTTP proxy run by your ISP, it doesn’t make as much for browsers, because...
Wednesday, 24 August 2011
It used to be that when you registered a media type, a URI scheme, a HTTP header or another protocol element on the Internet, it was an opaque string that was a unique identifier, nothing more. Sure, there are some substructures (e.g., vnd. and prs. in media types) to aid in avoiding collisions, but they don’t actually do anything. And even so, they need to be used judiciously (e.g., the problems inherent in x-). However, it’s now becoming fashionable to...
Friday, 5 August 2011
Last week, Blaze.io highlighted how mobile browsers use HTTP pipelining. I’ve been active in trying to get pipelining more widely deployed, but to date I haven’t tested mobile browsers much. So, one VM and two test pages (20 images on each) later, I asked my twitter peeps to hit it with their phones while I was watching with htracr. The results confirm what they saw; mobile browsers do pipeline, sometimes aggressively. A normal build of Firefox (and pretty much...
Wednesday, 27 July 2011
FYI, I’ve implemented Content Security Policy on this site. If your’e a Mozilla user, please tell me if you have any problems. (No, I’m not particularly worried about XSS here; I just want to get a feel for it; see my feedback so far.)...
Monday, 11 July 2011
The explosion of HTTP implementations isn’t just in clients and servers. An oft-overlooked but important part of the Web ecosystem is the intermediary, often called just a “proxy”*. These days, it’s pretty easy for anyone to build a proxy using Python, Ruby, Perl, Java or Node.JS, and there are a bunch of frameworks that can help you do this, such as node-http-proxy. Additionally, there are lots of off-the-shelf proxies that you can use, from the widely-known Squid and Traffic...
Sunday, 19 June 2011
HTML5’s AppCache mechanism is one confused little puppy. Purporting to be for taking web applications offline — a compelling and useful thing — it’s more often used by performance-hungry sites that want to use it as an online cache. Unfortunately, this means that those sites will be popping up dialogues like this: Even more unfortunately, if you click “Don’t Allow”, you’ll just be asked to approve it again next time you navigate to the site. And again. And again....
Friday, 27 May 2011
After designing and deploying Cache Channels, it quickly became apparent that one Web cache invalidation mechanism wasn’t able to cover the breadth of use cases. In a nutshell, Cache Channels trades off immediacy for reliability; that is, while cache invalidations don’t take place right away (there’s a 10-30 second window), you know that they’ll be respected, because of how the protocol is designed. That’s great if you need to (for example) invalidate news articles with months of TTL because...
Wednesday, 18 May 2011
A lot of people seem to be talking about and performing load tests on HTTP servers, perhaps because there’s a lot more choice of servers these days. That’s great, but I see a lot of the same mistakes being made, making the conclusions doubtful at best. Having spent a fair amount of time benchmarking high-performance proxy caches and origin servers for my day job, here are a few things that I think are important to keep in mind. It’s...
Monday, 4 April 2011
Last week found lots of HTTP-ish folks in Prague for IETF 80. In short, the good bits: HTTPbis HTTPbis had a cracker of a meeting, with three RFC2616 authors and representatives of all five major browsers in the room. First and foremost, the Content-Disposition draft has been approved for publication as an RFC by the IESG. This is great news for interoperability, with growing browser support for the revised specification. With one critical issue scheduled for a fix in...
Tuesday, 22 March 2011
Now, you can test any URL to instantly determine if it’s RESTful. You’re welcome....
Tuesday, 1 March 2011
The IESG has received a request from the Hypertext Transfer Protocol Bis WG (httpbis) to consider the following document: 'Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP)' <draft-ietf-httpbis-content-disp-06.txt> as a Proposed Standard The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2011-03-14. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain...
Friday, 1 October 2010
I’m going to try to start blogging more updates (kick me if I don’t!) about what’s happening in the world of HTTP. HTTPbis The effort to revise the core HTTP specification (RFC 2616) is going nicely, albeit slowly. Given the nature of the work, slow is better than rushed. In late July, some of the HTTPbis WG editors met in (then) sunny Münster before IETF78 in Maastricht. The result of that were drafts -10 and -11, closing a number...
Friday, 23 July 2010
Since SPDY has surfaced, one of the oft-repeated topics has been its use of TLS; namely that the SPDY guys have said that they’ll require all traffic to go over it. Mike Belshe dives into all of the details in a new blog entry, but his summary is simple: “users want it.” I don’t think it’s that simple. Trust I trust my ISP, to a point; I have a business relationship with them, so I don’t worry too much...
Thursday, 3 June 2010
A while back we used an absurd amount of reward points from our credit card to get some Myer gift certificates, and on the weekend these miraculously turned into a new TV, the Sony 32EX600. Overall, we really like it; while I’m still trying to find the exact recipe to successfully encode video to feed to it via DLNA, it’s beautiful to look at, and (unlike most TVs these days), the UI is a pleasure to use. Except. One...
Thursday, 6 May 2010
On a bit of a roll, RFC5861: HTTP Stale Controls has (finally) been published as an Informational RFC. As discussed before in “Two HTTP Caching Extensions,” these are very useful ways to hide latency and errors from your end users. While they’re most useful in HTTP gateway caches (a.k.a. reverse proxy caches / accelerators), very latency-sensitive sites might find them useful as well when working with “normal” proxy caches. Both are implemented in Squid 2.7. Not only does Squid...
Wednesday, 5 May 2010
Steve Souders and others have been working for a while on HAR, a HTTP Archive format. I love the idea behind HAR, but as I expressed on the mailing list (so far no response), I have a hard time believing that different implementations will parse and process HTTP and then serialise it into HAR in exactly the same way. My use case is incorporating HAR support into RED, both for import and export. However, RED needs byte-for-byte access to...
Wednesday, 7 April 2010
One of the nagging theoretical problems in the Web architecture has been finding so-called “site-wide metadata”; i.e., finding something out about a Web site before you access it. We wrestled with this in P3P way back when, and the TAG took it up after that. The easy solution to this is to define a static “well-known” URL — like /robots.txt. However, having a third party come along and squat on part of every Web site’s URL namespace is less than...
Wednesday, 10 March 2010
Thomas Hühn has graciously translated the caching tutorial into German. Thanks! See also the Chinese, Czech and French translations. To help the translators keep up with changes, I've started hosting the raw document on Github, which can also be used to log issues....
Thursday, 18 February 2010
Resource Packages is an interesting proposal from Mozilla folks for binding together bunches of related data (e.g., CSS files, JavaScript and images) and sending it in one HTTP response, rather than many, as browsers typically do. Intuitively, this seems to make sense; less HTTP requests is good, right? Maybe, maybe not. AFAICT, there aren’t any metrics comparing RP vs. traditional sites (has anyone done this?). In any case, a few concerns come to mind about this approach to making...
Friday, 15 January 2010
If you haven’t seen it already, check out the Call for Papers for the First International Workshop on RESTful Design (WS-REST 2010), where I’m on the program committee, along with many of the usual suspects. Submissions due February 8, 2010, 23.59 Hawaii time. If only I were there to receive them…...
Wednesday, 16 December 2009
Australia has apparently decided, through its elected leaders, to filter its own Internet connection. Since many, many other people are discussing whether this is advisable or indeed effective, I’ll focus here on what this will do to HTTP, and by extension the Web. What’s on the Table Reading the white paper, there are three different technologies for filtering the Web on the table; “pass-by hybrid”, whereby the ISP’s router will shunt traffic to target IP addresses to another box...
Friday, 13 November 2009
A couple of nights ago, I had a casual chat with Google’s Mike Belshe, who gave me a preview of how their “Let’s make the Web faster” effort looks at HTTP itself. SPDY (nee FLIP) is an alternate application protocol that’s in Chromium, but buried so deeply that you have to enable it with a command-line option (—use-flip). AFAICT there aren’t even any public servers that support it yet, but it’s still a very exciting development. Why? In a...
Friday, 30 October 2009
A long time ago*, the word in high-performance proxy-caching was Inktomi’s Traffic Server. It was so fast it was referred to being “carrier grade” and this could be said without people smirking, and it was deployed by the likes of AOL, when AOL was still how most people accessed the Internet. Then, in 2001, Yahoo! bought Inktomi. They did this because they wanted to be a player in Search, and they happened to get a proxy/caching product for free....
Thursday, 13 August 2009
Although I'm a bit concerned to see so many references to "Web 2.0", it's very exciting to see Australia talking about opening up government. They've asked for feedback, so I've submitted my AU.02, and will try to attend the Melbourne roadshow to see what's up. Actually, no I won't; I'm already attending a school council meeting that night. Damn....
Friday, 3 July 2009
The Stockholm IETF meeting is shaping up to be an interesting one (and not just because it’s in such a beautiful city). As announced on the mailing list, we are having a HTTPbis working group meeting. It looks like all of the editors will be there as well, so we’ll have a chance to get good feedback from the community, as well as move forward on the documents in between other meetings. Additionally, I’m helping to arrange a couple...
Thursday, 25 June 2009
A (very) long time ago, I wrote the Cacheability Engine to help people figure out how a Web cache would treat their sites. It has a few bugs, but is generally useful for that purpose. However, as I’ve got more involved in using HTTP for non-browser things, it’s become apparent that more than caching is important when you’re examining a resource to see how it behaves; things like partial content, syntax checking and other esoteric but important details. Very...
Wednesday, 17 June 2009
The caching tutorial is now available in Chinese, courtesy of Che Dong (and apologies for taking so long in linking to it!). Norwegian should be coming soon......
Friday, 12 June 2009
Part of my job is maintaining Yahoo!’s build of Squid and supporting its users, which use it to serve everything from the internal Web services that make sites go to serving Flickr’s images. In that process, I often am asked “what about X?”, where X is another caching or load balancing product (yes, Squid can be used as a load balancer). For example, Varnish, or lighttpd. Generally, these comparisons come down to three factors; performance, features and manageability. Almost...
Friday, 5 June 2009
HTTP performance is a hot topic these days, so it’s interesting that Opera has announced a “turbo” feature in Opera 10 Beta; Ever felt a Web site was loading slowly? Do you think it will happen again? Think again: Opera Turbo is a compression technology that provides significant improvements in browsing speeds over limited-bandwidth connections like a crowded Wi-Fi in a cafe or browsing through your mobile phone while commuting. They go on to give more details in their...
Friday, 29 May 2009
Everybody’s atwitter (yeah, sue me) about the Google Wave developer preview. Lots of new stuff there, but for me the most revealing comment, almost a throwaway, was here: Did we mention we use Squid? In other words, Google’s hot new, Web 2.0-on-steroids HTML5 XMPP++ open social platform buzzword generator works well with, indeed relies upon (for the purposes of making a nice demo) a more than 15-year-old piece of nearly-forgotten (in some quarters) Web infrastructure. That’s pretty cool....
Tuesday, 14 April 2009
I had a lovely holiday weekend in Canberra with the family, without Web access. Perhaps I’ll blog about that soon — Canberra being in my opinion one of the nicest overlooked cities in the world — but that will have to wait. Going offline for a few days always brings a certain dread of what one’s inbox will hold when you get back, and this one was no exception. That’s because while I was watching the kids rolling down...
Tuesday, 24 February 2009
There’s a rule of thumb about when a HTTP response can be cached; the Caching Tutorial says: If the response’s headers tell the cache not to keep it, it won’t. If the request is authenticated or secure, it won’t be cached. If no validator (an ETag or Last-Modified header) is present on a response, and it doesn’t have any explicit freshness information, it will be considered uncacheable. And, generally, this is true; most implementations won’t both caching something that...
Wednesday, 18 February 2009
Sometimes, it seems like every time somebody has a great idea for a new HTTP header, media type, or pretty much any other protocol element, they do the same thing. Rather than trying to figure out how to fit into how the rest of the world operates, getting adequate review and socialising their proposal, they just stick a bloody X- on the front and ship it. The IETF has no-one to blame but itself for this situation, of course....
Friday, 21 November 2008
There are lots of new “Web 2.0” specs emerging — many beginning with “o” — that are both exciting and concerning. Exciting because the Web is still evolving and still being applied to new problems, but concerning because the Web is big and has many moving parts. While it’s easy to design something new to address one use case or or set of interests, it’s much harder to make sure it works across the breadth of the Web and...
Monday, 27 October 2008
Ryan Tomayko announces Rack::Cache, a HTTP cache for Ruby’s generic Web API; The basic goal is standards-based HTTP caching that scales down to the early stages of a project, development environments, light to medium trafficked sites, stuff like that. HTTP’s caching model is wildly under-appreciated in the Ruby web app community and my hope is that making its benefits more accessible will lead to wider understanding and acceptance. I commented and, of course, focused solely on one aspect of...
Thursday, 16 October 2008
Metadata discovery is a nagging problem that’s been hanging around the Web for a while. There have been a few stabs at this problem (including at least one by yours truly), but no real progress. This is both unfortunate and worrisome, because as the next generation of Web-based protocols informed by REST, Web 2.0 and the like roll out, they’re going to need a way to find and talk about metadata on the Web in an automated fashion. And...
Friday, 4 July 2008
Here’s a gem on a little-used mailing list: As most of you know, over the last several years fairly good progress has been made on standardizing Web services. Many Web services specifications have, in fact, been standardized in W3C (i.e. SOAP 1.2, WSDL 2.0, WS-Addressing, WS-Policy, etc). There is still some work to be done. Accessing data about a resource through Web services is an area of the Web services architecture that has yet to be fully realized. Some...
Thursday, 20 March 2008
Having complained before about the sad state of HTTP APIs, I’m somewhat happy to say that people seem to be getting it, producing more capable server-side and client-side tools for exposing the full range of the protocol; some frameworks are even starting to align object models with resource models, where HTTP methods map to method calls on things with identity. Good stuff. However, something’s been bugging me for a long time about this. While there’s a nice internal logic...
Monday, 3 March 2008
Not many people that I know outside of IETF circles realise that a new *DAV effort has started up; CardDAV. An address book access protocol leveraging the vCard data format. The Internet-draft draft-daboo-carddav will be the starting point. The WG is explicitly cautioned to keep the base specification feature set small with an adequate extension mechanism, as failure to do so was a problem for previous PAB efforts (ACAP). Draft-daboo looks like it’s taken a page out of CalDAV...
Sunday, 17 February 2008
It’s 7am, I’m sitting in the Auckland Koru Club on my way home and reading the minor kerfuffle regarding PATCH with interest. For me, the critical difference between PATCH and POST is generality; PATCH is a generic method (as all good HTTP methods should be), while POST is not (the exception that proves the rule). As such, it should be possible to take a PATCH request and a current representation of the resource it’s being applied to and —...
Wednesday, 6 February 2008
Here’s one that I’ve been wondering about for a while, for the LazyWeb (HTTP Geek Edition); PUTs and POSTs can result in the creation of new resources, or changes to the state of existing ones. The response to both can contain one of a status message about what happened, or a representation of the resource (or most important one, if more than one was created/modified); often signalled by the Content-Location header, or nothing (i.e., just the status line and...
Monday, 21 January 2008
I’m following the discussion of RESTful Web description in general, and WADL in particular, with both difficulty and interest (see Dare, Patrick and Joe’s thoughts for a nice contrast). Difficulty because there’s so much of it, and it’s hard to give each piece the attention it deserves. Interest because it goes to the heart of the harder parts of REST — e.g., “hypertext as the engine of application state.” On top of that, I was one of the people...
Friday, 4 January 2008
The stale-while-revalidate and stale-if-error extensions aren’t the only fiddling we’ve been doing with the HTTP caching model. Now that Squid 2.7 is starting to see daylight, I can explain about a much more ambitious project — Cache Channels. In HTTP, there are generally two ways to keep something in a cache fresh; using a freshness lifetime (e.g., Cache-Control: max-age), and validation (e.g., If-Modified-Since). Together, they do a really good job of making the Web seem faster and reducing load...
Wednesday, 12 December 2007
We use caching extensively inside Yahoo! to improve scalability, latency and availability for back-end HTTP services, as I’ve discussed before. However, there are a few situations where the plain vanilla HTTP caching model doesn’t quite do the trick. Rather than come up with one-off solultions to our problems, we tried going in the other direction; finding the most general solution that still met our needs, in the hopes of meeting others’ as well. Here are two of them (with...
Sunday, 9 December 2007
I haven’t talked about it here much, but I’ve spent a fair amount of time over the last year and a half working with people in the IETF to get RFC2616 — the HTTP specification — revised. That effort reached a milestone last week when the HTTPbis Working Group had its first face-to-face meeting in Vancouver. It’s still early days, but we’ve already made good progress; based on what we saw in the room, for example, it looks like...
Friday, 2 November 2007
I've updated the WADL documentation stylesheet, primarily to; Fix a bug with finding and displaying XML Schema Make it compatible with xsltproc (and hopefully most other XSLT1.0 processors that understand EXSLT node-set) Generate valid XHTML The hard part was getting support for xsltproc; while switching over to the node-set function was easy, it uncovered other bugs around how libxml handles copying namespace nodes; an ugly and tiresome hack was required to work around it, but it seems to work....
Saturday, 8 September 2007
Feed Paging and Archiving (nee Feed History) has finally made it to a standards-track RFC. For many non-traditional (read: non-blog) applications of Atom, I think archived feeds in particular are going to be vital. I’m already using it in some places which should be seeing the light of day before too long; e.g., sending events to HTTP caches — a problem space that people have been noodling on for a very long time indeed. I’m also hoping that blog...
Tuesday, 7 August 2007
I’ve been hoping to avoid this, but ETags seem to be popping up more and more often recently. For whatever reason, people latch onto them as a litmus test for RESTfulness, as the defining factor of HTTP’s caching model, and much more. So, let me counter: they’re not all that. In fact, there are a number of pitfalls you need to be wary of if you use them. First, depending on how they’re generated, you might find different boxes...
Saturday, 28 July 2007
URI Templates -01 is now an Internet-Draft. After sitting on the spec for a while and trying to figure out an elegant solution to the encoding problem, we decided to take the simple route and see how it sticks. The only encoding that the template spec itself does is for characters that are outside the range of those allowed in URIs. Everything else is application-specifc. So, for example, this variable; foo = "/thing?@ <--stüff-->" with this template: http://www.example.com/{foo} will...
Wednesday, 20 June 2007
A while back I wrote up the state of browser caching, after writing a quick-and-dirty XHR-based test page, with the idea that if people know how their content is handled by common implementations, they’d be able to trust caches a bit more. The other half that they need to know about, of course, is proxy caching; depending on who you listen to, somewhere between 20% and 50% of clients on the internet are behind some kind of proxy, and...
Wednesday, 13 June 2007
It's a little thing, but I'm very pleased to see that Safari 3 will check with you before you discard a page where you've entered data on a form; A nice step forward....
Tuesday, 1 May 2007
Martin Arlitt makes an exciting announcement; It is my pleasure to announce two new versions of httperf: 0.8.1 and 0.9.0. version 0.8.1 fixes the known bugs in version 0.8, which was released almost 7 years ago. The primary new feature in 0.9.0 is improved portability. It has been tested on HP-UX 11i (64-bit PA-RISC and IA-64) Red Hat Enterprise Linux AS (AMD64 and IA-64) SUSE Linux 10.1 (i386) openSUSE 10.2 (i386) OpenBSD 4.0 (i386) FreeBSD 6.0 (AMD64) Solaris 8...
Sunday, 29 April 2007
The QCon presentation (slides) was ostensibly about how we use HTTP for services within Yahoo’s Media Group. When I started thinking about the talk, however, I quickly concluded that everyone’s heard enough about the high-level benefits of HTTP and not nearly enough details of what it does on the ground. So, I decided to concentrate on one aspect of the value that we get from using HTTP for services; intermediation, as an example. If your service is struggling to...
Thursday, 5 April 2007
We’ve announced the program for this years’ Developers’ Track, and I’m very excited about the lineup. For example, Ryan Boyd from Google will be presenting about GData right before Pasha Sadri talks about Yahoo! Pipes. These are two cutting-edge uses of feeds, and with a little luck we might even be able to get them to field some joint questions in the middle. Also on the topic of feed syndication, Elias Torres is scheduled to talk about Apache Abdera...
Tuesday, 27 February 2007
I think that most of the debate about REST focuses on the wrong things, leading developers down the garden path at the expense of their productivity and the success of their projects. Time and time again, I’ve seen folks who are new to REST get caught up on small stuff like this; False Choice: “High” vs. “Low” REST Lots of blog space has been devoted to figuring out what PUT and DELETE are good for, leading many to believe...
Wednesday, 7 February 2007
Yahoo! (finally!) released Pipes as a beta today; congrats to the very talented team that put this together. Niall gives the geeks-eye view, and to be clear, this is not going to be the next great consumer Web site; your grandmother is not going to go out and build pipes. However, I do think it’s going to be a big wake-up call for the “Enterprise” software industry. This tool does more to deliver on their promises of non-programmers slicing...
Wednesday, 7 February 2007
A reminder: proposals for the Developers’ Track at WWW2007 should be in by February 16th. We’re looking for Web-focused presentations, demos and tutorials for and by developers. I’m particularly interested in seeing what’s happening in Syndication, HTTP, XML, and Web Frameworks. Submissions need to include an abstract and either a) rough slides, b) a description of or screenshots from a demo, or c) a paper. More info at the CfP. P.S. We also have some limited funding (courtesy of...
Monday, 4 December 2006
Uche calls it; So the SOA wars are heating up. More and more smart people are pointing out that the emperor has no clothes; but stakes is still crazy high. Some folks haven’t yet made all their money from SOA. So how do the stakeholders respond? With cold-blooded threats. … and here’s where it started. Priceless. I think that most “enterprises” (whatever that means) do need a shake-up, and maybe some new blood. But not to buy into an...
Thursday, 30 November 2006
One of the perceived deficiencies of JSON is that it doesn’t have a schema language. I say “perceived” because the problems that a schema language brings often outweigh the benefits; after all, look at the mess that XML Schema is in. Even that said, schemas are useful for documentation and QA. So, I’m finding the work that Robert Cerny has done very interesting; it’s basically schemas for JSON, in JSON (or very nearly so). For example, here’s the schema...
Friday, 27 October 2006
There are plenty of reasons to hate HTTP Cookies, but there’s one thing that especially annoys me; their syntax. Specifically, the Netscape spec allowed an “expires” field that contains the date the cookie should be discarded; Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/; expires=Wednesday, 09-Nov-99 23:12:40 GMT What’s wrong with that? Well, HTTP uses commas to delimit different instances of the same header, saying: It MUST be possible to combine the multiple header fields into one “field-name: field-value” pair, without changing the semantics...
Thursday, 19 October 2006
Dave Johnson writes up a nice summary of the issues of adding new elements to HTML for declarative Ajax, something that I ran into when doing HInclude. Basically, IE doesn’t give you access to any non-HTML element that’s not in a namespace, and you have to access it by the full QName, so your hands are tied somewhat (e.g., the user can’t swap in a different prefix), but it’s doable. E.g., var includes = document.getElementsByTagName("hx:include"); if (includes.length == 0)...
Tuesday, 17 October 2006
My team at Yahoo! is looking for a mid-level developer (5-10 years experience) to help build our HTTP/REST toolkit, among other things. The job is mostly coding Perl and PHP, with some C or C++ skills desirable. There’s also the opportunity for design input and work on related projects (e.g., we work closely with Dave Beckett’s team). We’re also looking for an engineering manager. This is a technical role requiring at least five years of management experience. If you’re...
Friday, 13 October 2006
A couple of interesting things have happened recently; first, Jonathan Marsh has a new job; WSO2 is a year-old startup which provides support services around Apache’s Axis 2 Web services application server. The company’s CEO, Sanjiva Weerawarana, on Saturday announced that it has hired Jonathan Marsh from Microsoft to head up development efforts in mash-ups. “Jonathan will drive the development of our 3rd major product; WSO2 Tellurium, which will be a server-side mashup platform. (Somewhat akin to what a...
Wednesday, 4 October 2006
As mentioned a while back, there are a variety of places where it would be useful to be able to describe the structure of a URI, rather than just convey a URI itself. I took a stab at this in the Link Header draft, and have also been working in the background with DeWitt Clinton, Joe Gregorio, Marc Hadley, Dave Orchard, and James Snell on a more general specification, URI Templates, the first draft of which we (finally!) got...
Sunday, 1 October 2006
Hot on the heels of the last batch, Stefan pointed me to Jesse Skinner’s addDOMLoadEvent, which seems to avoid the problems I found earlier (you know you’re in for some debugging when you’re cutting-and-pasting code from blog comments!). As a result, I’ve updated HInclude and URL Templates (both in new homes). For a demo of how well the new HInclude works, take a look through some photos and note when the included footer loads. Thanks, Stefan and Jesse! Next,...
Thursday, 28 September 2006
I’ve updated the url_template.js and json_form.js libraries to fix some bugs, to make the demo I gave at XTech run more smoothly. It should work well on Safari, Mozilla and IE6 (despite some glitches at a showing inside Y! the other day; the demo gods were not smiling). It does not work in Opera; it seems like the more I use XHR in that browser, the more bugs I find. I’m thinking of updating the XHR tests to catch...
Saturday, 16 September 2006
Apple’s shipping an iSight camera in just about everything these days, and one of the coolest apps to use it is Delicious Library. If you follow that to its logical conclusion, everything should be barcode-enabled, by Web-enabling it. For example, if you go to a wine site and scan a bottle that’s sitting on your shelf, Safari should be smart enough to automatically navigate to a page with details for that bottle (e.g., a map of where it was...
Monday, 21 August 2006
There have been some interesting developments in Web caching lately, from a performance perspective; event loops are becoming mainstream, and there are lots of new contenders on the scene. Fortuitously, I’ve been benchmarking proxies with an eye towards the raw performance of their HTTP stacks (i.e., how well they handle connections and parse headers, serving everything from memory), for work, so that we can select a platform for internal service caching. In particular, I’ve been looking at the maximum...
Friday, 18 August 2006
Many thanks to J.J. Solari for translating the Caching Tutorial to French!...
Monday, 14 August 2006
Timbl has this great term “Webizing” that he uses to talk about giving existing systems the benefits of the Web architecture. Despite the first part of “Web 2.0”, I think AJAX is in severe need of some serious Webizing. Case in point, script.aculo.us has a great autocomplete function, but it uses POST by default. What’s up with that? GET will do just fine, and gives you caching for free. Furthermore, it forces you into using query strings (e.g., http://example.com/suggest?q=AAPL),...
Friday, 30 June 2006
If you boil down the BNF in both RFC2396 and RFC3986, path segments can contain the following characters without percent-encoding them: ALPHA DIGIT ! $ & ' ( ) * + , - . : ; = @ _ ~ Query components can contain these: ALPHA DIGIT ! $ & ' ( ) * + , - . / : ; = ? @ _ ~ Which means that " < > [ \ ] ^ ` { | }...
Saturday, 24 June 2006
Hugo has finally blogged the big news. He’s left one of the coolest jobs in the world — working for the W3C — to come to another one of ‘em, working for Yahoo. I’m really looking forward to continuing to work with him; there’s lots to do! (Is it clear now why he's been poking around the Yahoo properties on his blog so much? ;)...
Thursday, 22 June 2006
Recently, there’s been a resurgence for the Link element in HTML; everything from Microformats to Atom autodiscovery is using it. This isn’t surprising; as machines start processing Web documents more, it’s necessary to use hyperlinks — the foundation of the Web — to tie resources together, without getting in users’ faces. Such linking is useful beyond HTML, of course; Atom defines its own concept of link relationships, and uses them in the Atom Publishing Protocol. Usually, embedding this kind...
Tuesday, 20 June 2006
A friend (who shall remain anonymous) pointed me to Microsoft’s announcement today regarding their foray into robotics, of all things. My eyes glazed over until they rested upon the Microsoft Robotics Application Model; The term “REST” was originated by Roy Fielding. It abstracts, and to a degree, formalizes the Web architecture. REST builds on the notion introduced by Tim Berners-Lee that a URI refers to a resource and all interactions with that resource happens by exchanging state. One can...
Friday, 9 June 2006
See if your aggregator can subscribe to this feed (username/password: test/test) and post the results in comments.While you’re there, it would be interesting to know what happens if you shut down / log out of your aggregator, get back in, and then refresh....
Thursday, 25 May 2006
When I joined Yahoo, one of the biggest adjustments I had to make was to their use of “Web Services”. There, that phrase means any kind of machine-to-machine communication using HTTP; SOAP isn’t assumed (or preferred). This is diametrically opposed to the vendor and standards worlds, where “Web Services” means the stack of SOAP, WSDL and friends. Much confusion ensued on my part, because outside of the vendor and standards hothouse, real-world developers are still using the phrase to...
Tuesday, 16 May 2006
I just finished my XTech presentation, “Web 2.0 on Speed”. here are the slides [pdf]; I’m going to try to s5 them soon. There isn’t much new in this talk; it’s just a synthesis of a few different observations; 1. Static Web servers are fast. You might say ‘duh’, but people often overlook the obvious; with event-looped Web servers like lighttpd (see c10k for more information), you can get 2-3 orders of magnitude more requests/second over LAMP and similar...
Thursday, 11 May 2006
Updated 2006-06-03 One of the big problems that Web developers have with HTTP caching is that they don’t know how the caches behave; while the specs say one thing, the actual behaviour of the cache often significantly deviates — usually because the cache’s developer or operator thinks they can do better. The easiest way to overcome this obstacle is to measure the behaviour of the caches. In particular, thanks to XmlHttpRequest, it’s fairly easy to test a browser’s cache by...
Thursday, 11 May 2006
Yaron publicly says what he’s doing at Microsoft (scroll down); I hear that HTTP stuff is pretty cool. If anyone cares you can peruse a bunch of blog entries on my website (www.goland.org) where I walk through a number of key enterprise scenarios and show that nothing more than HTTP+XML is required. And yes, I still work for Microsoft. In fact, one of my jobs is to write the best practices for the design of all external interfaces for...
Wednesday, 10 May 2006
Anne-Thomas Manes extolls the virtues of WS-*; The single, most important feature that inspires my enthusiasm about WS-* is that it has universal support from all the major vendors. Ah, there we are; major vendors. What she’s basically saying here is that if you’re silly enough to have invited one of these vampires into your home, you’ll have the option of selecting other vampires to replace them at will, and that your vampire will be able to talk with...
Sunday, 23 April 2006
It’s official; I’ve got a last-minute slot at XTech, talking about all things Web caching....
Thursday, 20 April 2006
Back at the W3C Technical Plenary, I argued that Working Groups need to concentrate on making more Web-friendly specifications. Here’s an example of one such lapse causing security problems on today’s Web. Safety in HTTP HTTP methods have a property known as safety; RFC2616 describes it in section 9.1.1; Implementors should be aware that the software represents the user in their interactions over the Internet, and should be careful to allow the user to be aware of any actions...
Thursday, 13 April 2006
I’ve had a lyric running through my head for the last day or so, thanks to a couple of bugs. I am thinking it’s a sign that the freckles / In our eyes are mirror images and when / We kiss they’re perfectly aligned — The Postal Service, Such Great Heights Let me explain. Apache Apache’s ap_meets_conditions in http_protocol.c is responsible for handling conditional HTTP requests. If you send Apache an If-Modified-Since, this code will figure out whether or not...
Friday, 7 April 2006
It’s become axiomatic in some circles — especially in WS-* land, as well as in many other uses of XML — that the preferred (or only) means of offering extensibility is through URI-based namespaces, along with a flag to tell consumers when an extension needs to be understood (a.k.a. mustUnderstand). The reasoning is that extensibility should be as easy as possible. By leveraging one registry — DNS — you can use URIs to allow anyone to create your own...
Thursday, 6 April 2006
I’m a little confused by Mark Baker’s stance regarding SOAP; he seems to encourage the Web services world to use SOAP on top of HTTP in a fashion compatible with HTTP. As I asked Chris Ferris recently, what’s the point? UPDATE: Mark responds. While I agree that evangelising HTTP is good, and that teaching certainly brings benefits to the teacher as well, I’m concerned that people will only see HTTP through the imperfect, low-resolution lens of SOAP, not realising...
Monday, 20 March 2006
True to form, Don’s using his witty charm and good looks (such as they are ;) to shape discussion of a topic… in this case, REST, where he splits the RESTifarian world into two; “hi” and “lo.” To get a more accurate picture of what we’ve done so far, I’ll break this category in two: Lo-Rest, which is the use of HTTP GET (or equiv) for information retrieval/query, and Hi-Rest, which is characterized by the use of HTTP PUT...
Thursday, 16 March 2006
There’s some excitement out there about “Cookie-less HTTP Authentication.” While it’s tempting to say that cookies are evil (the fru-it of the dev-il), using them for authentication isn’t actually that bad, from a REST standpoint; the main loss is that you can’t use standard mechanisms to log in, instead relying on an HTML form. This brings problems with non-browser clients, such as RSS aggregators, WebDAV and CalDAV clients, and XSLT stylesheets. In short, you lose some of the network...
Wednesday, 15 March 2006
Microsoft and friends (of the keep your enemy closer variety, I suspect) have submitted WS-Transfer to the W3C. I found the Team comment interesting; e.g., WS-Transfer can therefore be seen as an underlying protocol-independent version of HTTP, i.e. bringing the capabilities and properties of the Web and HTTP in contexts where HTTP is not used. The use of WS-Transfer is not limited to non-HTTP transports, and can also be used when HTTP is used as a communication tunnel. As...
Wednesday, 1 March 2006
Over the weekend, I submitted a new draft of Feed History. The big (and hopefully, last) change this time is the use of the “previous” and “subscription” Atom link relations, rather than extension elements. Of course, just because they’re in the Atom namespace doesn’t mean that they can’t be used in RSS. The algorithm for recreating the state of a feed is also specified much more carefully; please have a look and make sure it makes sense, and does...
Saturday, 18 February 2006
Have you ever posted a comment to a blog, found it missing, so you re-posted it, only to find two entries? Annoying, huh? Aaron pinged me the other day with this problem, and I responded that the Right way to do this is to POST to the same resource (i.e., the blog entry), so that the POST invalidates the cache. HTTP has this to say about the matter; Some HTTP methods MUST cause a cache to invalidate an entity....
Tuesday, 7 February 2006
Interesting; there are not one but two sessions at the upcoming ETech about taking Web applications offline. Given the current bent of O’Reilly conferences — speed dating for VCs and their willing prey — this is a pretty sure sign that we’re going to see another startup hypedfunded. This isn’t the first time this topic has been broached, of course, but we haven’t seen too many serious efforts at it. Adam Bosworth noodled on the topic back when we...
Monday, 23 January 2006
I’ve been playing around with some ideas that use XMLHttpRequest recently, but I keep on bumping up against implementation inconsistencies on IE vs. Safari vs. Opera vs. Mozilla. Although the interface exposed is pretty much the same, what it does in the background is very different, especially with regards to HTTP. For example, some implementations will handle redirects and cache validation for you, while others will pass through the HTTP status codes, expecting you to pick up the pieces....
Wednesday, 11 January 2006
It took two years, but Apple has finally taken steps to limit Safari’s content-sniffing ways; Safari now displays certain documents that have text/plain headers as plain text rather than treating them as HTML. — About the Mac OS X 10.4.4 Update Why “certain” documents, though? My trivial test is handled correctly now, but what about others? What’s the heuristic now?...
Monday, 9 January 2006
On the heels of mod_cgi, PHP now does the right thing (at least in 5.1) when setting the Allow header. mod_dav is still broken, though. Backstory here....
Saturday, 24 December 2005
The useful end of RFC 3864 (at least regarding HTTP) is finally* here. When you need to know where a particular header is defined there’s now one place to do it; IANA’s Message header registry and repository have been filled with HTTP-related headers by RFC 4229. The content was almost completely automatically generated, by scraping the pertinent information from the RFC citations, using cwm to infer some things about it, processing that n3 with a specialised script and then...
Saturday, 26 November 2005
The first in an occasional series about the real-world benefits of REST and the Web architecture, as applied to HTTP. I used to work for a fairly huge company as a Web/Internet guru. One day, I got sucked into a meeting with a visiting executive who was talking about rolling out a new set of servers to allow customer service reps access internal documentation. The requirement was to make large-ish PDF files available on the internal network world-wide nearly...
Monday, 7 November 2005
More and more people are getting turned on to the advantages of using REST as a higher-level abstraction for networked applications, often comparing it favourably to SOAP and Web services. However, as many have pointed out, this is a bad comparison to make; REST is an architectural style, while SOAP is a protocol. So, what should be compared? When this question comes up, I’ve found it helpful to draw a table with different architectural styles in the columns, and...
Sunday, 30 October 2005
Stumbled across this, from Ian Bicking; My problem with a lot of MVC web frameworks is that they are really a way of codifying one developers internal thinking about a web application, and they don’t map well when they are used by another web developer; or they require both the code and the developers mind to be reshaped. This is fine, until another framework comes along that requires another shift in mindset and code. Or you have a problem (or...
Wednesday, 26 October 2005
Does anybody know of a program or service that will look at a calendar file (e.g., vCalendar, iCalendar, hCalendar) and publish the entries on it as an RSS feed, where each entry in the feed has a link to that one calendar entry? It strikes me that this would be useful for public calendars where you don’t want to see the whole thing dumped into your personal calendar, but instead want to selectively add events to your calendar as they...
Sunday, 23 October 2005
I don't know if this has already been done (it's not exactly rocket science), but for the benefit of those who want to emulate the W3C's cool ,tools functions with mod_rewrite; # tools RewriteRule ^(.*),validate http://validator.w3.org/check?uri=http://%{HTTP_HOST}/$1 [L,R] RewriteRule ^(.*),checklink http://validator.w3.org/checklink?uri=http://%{HTTP_HOST}/$1 [L,R] RewriteRule ^(.*),rchecklink http://validator.w3.org/checklink?recursive=on&uri=http://%{HTTP_HOST}/$1 [L,R] RewriteRule ^(.*),text http://cgi.w3.org/cgi-bin/html2txt?url=http://%{HTTP_HOST}/$1 [L,R] RewriteRule ^(.*),cssvalidate http://jigsaw.w3.org/css-validator/validator?usermedium=all&uri=http://%{HTTP_HOST}/$1 [L,R] RewriteRule ^(.*),links http://www.google.com/search?as_lq=http://%{HTTP_HOST}/$1 [L,R] RewriteRule ^(.*),blinks http://blogsearch.google.com/blogsearch?scoring=d&q=link:http://%{HTTP_HOST}/$1 [L,R] RewriteRule ^(.*),cacheability http://www.ircache.net/cgi-bin/cacheability.py?descend=on&query=http://%{HTTP_HOST}/$1 [L,R] For example, you can validate this page....
Saturday, 22 October 2005
Why is it that Web browsers — Amaya excluded — don’t support PUT and DELETE? After all, if there are enough VCs foolish enough to part with their money for something like Flock, surely we could at least support all of HTTP’s methods. It’s not just browsers, of course; heck, even Python’s new, mega-complete urllib2 library restricts itself to GET and POST. Other languages are similarly limited. So, why does the implemented Web restrict itself to half of CRUD? It’s...
Wednesday, 19 October 2005
Roy Fielding has just closed a bug that’s been around since 1996, and which I’ve previously lamented here; The block has now been deleted from all active branches of httpd (1.3.35, 2.0.56, 2.1+). Thanks for sending in the more complicated patches, but I feel that CGI scripts should be able to handle all methods by now (or be fixed to do so properly). Broken CGI scripts, you’ve been warned. Now if PHP would get its act together, we could implement...
Tuesday, 18 October 2005
I’ve raved before about how useful the XSLT document() function is, once you get used to it. However, the stars have to be aligned just so to use it; the Web site can’t use cookies for anything important, and the content you’re interested in has to be available in well-formed XML. While that’s all fine and good on some higher-plane, utopian, RESTful, stateless, DTD- and Schema- described, Cool URIish Web, it’s not the useful on the Web that most of...
Wednesday, 5 October 2005
Does anybody else chortle quietly when they see “2.0-this” and “2.0-that”? It’s getting absurd; first we had “Web 2.0” (never mind that this term has been used for years in different ways, and that what they’re referring to is really just the original Web being used to its full potential) and now “Identity 2.0.” What gives? The whole things smacks of marchitecture. To top it off, linear versioning (a la 1.0, 2.0, etc.) is very un-Web. I guess people...
Monday, 5 September 2005
Feed History draft -04 is out, with the only major change being the replacement of fh:stateful with fh:incremental, with corresponding changes throughout the document, to make the concepts a bit clearer. This revision also makes cardinality, relative URIs and white space handling more explicit, and adds an acknowledgements section as promised. On the implementation front, here’s a quick-n-dirty Python script that demonstrates reconstruction of an incremental feed (RSS or Atom); while it’s more prototype code than something you’d want...
Thursday, 1 September 2005
I took a pass at a revision of the RSS Tutorial for Content Publishers and Webmasters on the train this morning, as I realised it was dreadfully out of date. It now includes Atom, and RSS 2.0, rather than 0.94. Corrections and suggestions are always welcome; in particular, I’m interested in any links you might have for the Tools or More Information sections (although I’ll only list free — preferably Open Source — products and services)....
Monday, 15 August 2005
I’ve had a few e-mails asking how I got this site’s RSS feed to include its history, so here are the instructions for doing it in Moveable Type (the software that I use to manage this site). If you have instructions for other feed-generating software, please either leave them in comments below, or send me an e-mail. In a nutshell, there are two easy steps; 1. Link your archives together The feed needs to be archived for it to...
Monday, 15 August 2005
Draft -03 of Feed History: Enabling Stateful Syndication is now available. Significant changes include: Added fh:archive element, to indicate that an entry is an archive Allow subscription feed to omit fh:stateful if fh:prev is present Clarified that fh doesn’t add ordering semantics, just allows you to reconstruct state Cleaned up text, fixed examples, general standards hygiene See this site’s feed for an example. There’s going to be at least one more draft, as I neglected to acknowledge people who...
Saturday, 13 August 2005
When I worked in the financial industry, I quickly noticed that Excel spreadsheets contain the bulk of the data in the enterprise. It may make IT execs tear their hair out, but having the data nearby and ready for analysis is sloppy, but oh-so-effective. The challenge is to make the data reusable elsewhere. Unfortunately, spreadsheets are a mish-mash of structured but meaningless data; there’s no easy way to tell which columns contain data and which ones are headers. To...
Wednesday, 10 August 2005
For some time, I’ve noticed that people defining XML formats spend an inordinate amount of time talking about the structure of the format. This is especially apparent in standards working groups, where hours — no, days — can be spent agonizing over whether to make something an attribute or an element. Part of this is obviously stylistic; people have different thoughts on what makes good XML, and they’re fight the same battles over and over again. I’ve often thought...
Monday, 8 August 2005
Some folks at IONA have written a paper entitled Where HTTP Fails SOAP. I had a chance to look at this before I got it published, and their conclusions make a lot of sense — if you accept the premise that SOAP (and Web services) is about integration with existing applications. In other words, if you’re using SOAP to integrate with existing systems, you’re probably going to use whatever interfaces are already available on the box; if there’s a...
Saturday, 16 July 2005
After more than five years, syndication is maturing rapidly. It’s being used for more than blogging — whether it be stock quotes, system logs, or order lists — and even blogging will change in nature as it gets more popular; people will be using blogs to fundamentally change the way they do business, inside and outside the firewall. In the enterprise, people are going to look at syndication as a pub/sub replacement. While feeds over HTTP won’t be able...
Friday, 15 July 2005
It’s been covered before elsewhere, but just a friendly reminder: ‘feed’ URIs are bad for the Web, as are any that are used solely for dispatch (e.g., ‘itms’, ‘pcast’). I’m looking at you, Apple. Interestingly, Apple also supports doing Web dispatch the right way; with media types. I know this because going to my test page — a zero-length file with a generic extension, but a ‘application/atom+xml’ media type — gets dispatched to my RSS reader in Safari on...
Saturday, 9 July 2005
While a lot of companies are exploring blogs as a means of building communities, Intuit* (makers of Quicken, TurboTax, etc.) has skipped directly to the next logical step; using Wikis. TaxAlmanac is a Wiki about taxes, for tax professionals; they can browse the current Internal Revenue Code for free, ask a question and share information they’ve learned about one of the most obfuscated organisations in the world, the IRS. Why? In their words, So, what’s the catch? There is...
Friday, 8 July 2005
You can describe just about anything with sufficient precision in plain English, given enough words. In practice, this doesn’t happen; specialised fields — whether science, finance or art — develop specialised jargon as a shorthand for concepts that are well-understood in that field. It gives greater precision, easier flow of ideas, and yes, it raises the bar to entry for newcomers. The trade-off is worth it, usually; although it would be genuinely useful if a layman could understand the...
Tuesday, 24 May 2005
The W3C has just started a mailing list for discussion of Web description formats; This mailing list is dedicated to discussion of Web description languages based on URI/IRI and HTTP, and aligned with the Web and REST Architecture. Unlike WSDL (Web Services Description Language), such languages are not targeted towards description of Web Services. What’s interesting is that many Web Services people — such as David Orchard, Marc Hadley and myself (although I always think of myself as an...
Sunday, 22 May 2005
There’s been quite a kerfuffle over Google’s Web Accelerator, because it prefetches Web content. It’s amusing to see these issues recycle over time; in the late nineties, prefetching was one of the biggest areas of research in Web caching. There were lots of papers written (search for “prefetch”), and even a commercial implementation; CacheFlow was the prefetching cache (they called it “active content”), and it was pretty controversial in the industry. After a long period of back-and-forth and considerable...
Wednesday, 18 May 2005
Marc Hadley has released WADL in the wild, and I’m intrigued; based on a first look, I’d say it’s the most promising Web (as opposed to Web Services) description language yet. Why? First of all, it’s very resource-oriented; you can clearly and easily see the deliniation between different described resources. It encourages good practices, and supports generative URIs both in query strings and in path segments, which I think is crucial. It’s simple and easy to read. Initial Feedback...
Thursday, 12 May 2005
I happened to look at the HTTP headers returned from Google News just now (what can I say, I’m a HTTP geek), and I noticed something unusual; Last login: Thu May 12 16:52:59 on console Welcome to Darwin! mnot-laptop:~> telnet news.google.com 80 Trying 64.233.161.147... Connected to news.l.google.com. Escape character is '^]'. HEAD / HTTP/1.1 Host: news.google.com HTTP/1.1 200 OK Set-Cookie: PREF=ID=d33570687b199641:TM=1115954884:LM=1115954884:S=EKAAPop2tSe_wM0T; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com Content-Type: text/html; charset=ISO-8859-1 Server: NFE/0.8 Cache-Control: private, x-gzip-ok="" Content-Length: 0 Date: Fri,...
Monday, 9 May 2005
There’s a lot of cool apps emerging for GreaseMonkey (and GreaseMonkIE and PithHelmet, for IE and Safari respectively). It seems like these extensions have a love/hate relationship with the Web, philosophically. On the one hand (with L O V E sprawled across the knuckles), GM is a great example of taking advantage of the representational nature of the Web. SOAP-heads would call this “loose coupling” or “document-oriented.” The fact that you can rock up and modify somebody else’s content...
Friday, 29 April 2005
A while back, I published a series of entries (1,2,3,4) about would-be Web Description Formats, with the intent of figuring out which (if any) is suitable, or whether a new one is required. I’d like to keep this moving, by documenting some requirements for such a format. Although a few requirements and use cases did come up in that discussion, I'd like to work through a more general set of questions that need to be asked as a way...
Sunday, 24 April 2005
Should cookies be shared between your RSS aggregator and your Web browser? If they were, sites would be able to automatically personalise the feeds you subscribe to; would people be interested in that, or see it as an intrusion in their privacy? The possibilities are certainly interesting; if I knew who you were, I could not only modify entries to suit you, but I could also insert an entry into just your feed, saying anything from “Hi” to “Do...
Sunday, 3 April 2005
Web metadata discovery is not a new topic, and one on which the final word has not been spoken. However, one of the most basic means of discovering something about a resource, the HTTP OPTIONS method, is not widely enabled by current implementations. I’m immediately interested in this because it would be nice to use it in conjunction with POE, but this isn’t the only use case; things like privacy policies, robot policies, configurations and site descriptions need to...
Monday, 21 March 2005
A while back, I wrote up a description of a pattern for avoiding messages like “click submit only once.” I didn’t do much after that, because I’ve been a bit busy, and because I wanted to do some implementation of a more general HTTP framework before I wrote a more formal document. One delay led to another, and to make a long story short, I never got around to writing that specification… until now. Why now? The short answer...
Tuesday, 22 February 2005
I love the XSLT document function. With it, you can access the whole Web from a stylesheet; this gives a lot of flexibility, in the right situation. For example, my local library’s online system is based upon iPac (now sold as the Horizon Information Portal, I think), a common packaged library management system. One of its nifty features is letting you keep a list of books (“My List”) that you’d like to eventually check out of the library. In...
Monday, 7 February 2005
Werner makes an excellent point; [W]e need to continue to take care that we do not consider The Model to be The Truth. The web based internet is a massive organic process that is similar to Nature, and we can develop models to observe its phenomena. We can use these models to build our tools on, but we have keep in mind that we cannot use the model force the organic process to behave the way we want it...
Wednesday, 19 January 2005
More than a year after my modest suggestion, Google takes a step to fix comment spam. Hopefully, other people who re-publish Web content (like mailing list archives) will start doing this as well. Perhaps the most interesting thing about this was how it was done. Google has added an HTML rel attribute value unilaterally (OK, they did co-ordinate with a few blog product vendors). Is this a good or a bad thing? The “right” way to add a rel...
Friday, 17 December 2004
The Australian Bureau of Statistics has released an SVG-based "animated population pyramid" that very nicely visualises the change in that country's population over time.
Wednesday, 15 December 2004
I’m thinking about whether it would be a good idea to have a media type for Python source files, call it “text/python.” The main benefit that I see to doing this is the definition of a fragment identifier syntax; i.e., what the bit after the ‘#’ refers to. This would allow URIs to point to specific functions and classes in Python source files, which would be very useful when documenting code. It would also allow some cool import tricks...
Sunday, 10 October 2004
In a recent post, Don gave his take on the enlightening nature of WS-Transfer; Honestly, WS-Transfer has been in the oven for quite a while. It’s been interesting to see people’s reaction to it. Stage 1. What good is this? Stage 2. Ahh, the idea of uniform application protocols seems pretty useful. Stage 3. Wow, I can model my entire universe using less verbs/operations than I have fingers on one hand. Why would I ever author another WSDL portType?...
Monday, 27 September 2004
I was very interested to see the reaction that people had to WS-Transfer over the last few days. While the SOAP Resource Representation Header had opprobrium heaped upon it (""), Transfer passed by with nothing more than a few nodding heads and people saying "aha." In my view, WS-Transfer deserves a lot more of that criticism; if anything, the Resource Representation Header tries to supplant MIME, not HTTP.
Wednesday, 8 September 2004
An update to the Internet-Draft that provides initial values for the HTTP Header Message Registries is now available.
Thursday, 26 August 2004
It's no secret that HTTP authentication isn't used as often as it should be.
Saturday, 31 July 2004
A few days ago I blogged a straw-man API for client-side HTTP based on dictionaries. This turns out to be well-aligned with a project I’ve had on the back burner for a while; coming up with some Python APIs for HTTP that are usable, encourage good practice, and well-aligned with the specifications. So, a first prototype is now available, with three modules defined; http.message, which defines common constructs for HTTP messages and their payloads, http.status, which enumerates the defined HTTP...
Monday, 26 July 2004
From the Daily Python URL comes another noteworthy API for XML; XMLFragment. I haven’t tried it yet (it doesn’t appear to be separately available, hint, hint), but I like the look of it. There are two interesting things going on here. First of all, XMLFragment basically gives up on modelling the complexity of XML in the language, instead punting to XPath. I think that’s a reasonable choice; it’s arguably more intuitive and simple than anything you could do with an...
Sunday, 18 July 2004
Timbl has talked about Web-izing databases and languages; what about operating systems? Despite Microsoft’s legal troubles brought about trying to integrate the browser into Windows, it’s a good idea. Here’s one for the LazyWeb: create a Mac OS X Contextual Menu plug-in with a very simple function — “Copy URL.” When used on a file or folder that’s served by the built-in Web server (e.g., in /Library/WebServer/Documents/ or ~/Sites), it should place the appropriate HTTP URL on the clipboard. For...
Thursday, 1 July 2004
When Tim O’Reilly gave his keynote at eWorld this year, one of his major points was that Internet-based mapping (e.g., Yahoo maps, Mapquest) had failed to take off, despite their obvious utility, because they were walled gardens; unlike eBay and Amazon, they don’t integrate user data and third-party applications very well. For example, if I want to put maps of things on my Web site, I can only do what amounts to a cut-and-paste; I can’t layer in my own...
Wednesday, 16 June 2004
Check out the Web Hypertext Application Technology Working Group; it looks like our last, best hope for extending the browser platform to grow the Web....
Monday, 14 June 2004
One thing about Web description formats that hasn’t seen much discussion yet is how people intend to use them. The WSDL Working Group has a Usage Scenarios document and a Requirements document, but unfortunately they only talk about the kinds of Web services they want to describe, not how the descriptions themselves are to be used. So, here are the use cases for Web description formats that I’m aware of, along with examples and some of my conclusions (skip ahead...
Saturday, 5 June 2004
Because Web sites often don’t make information available to us in the way we’d like, we have to bring the mountain to Mohammed and scrape screens. I’ve played around with this in the past with xpath2rss, a scraping tool that allows you to generate RSS feeds from HTML. Jon Udell has done likewise with his excellent LibraryLookup experiment (Jon, if you read this, please add Peninsula Library). In many cases, scraping the HTML is less than half the battle; the...
Sunday, 30 May 2004
Tim Bray is trying out “purple number signs” on his Web site to make fragment identifiers ubiquitous and easy to find. This site has something along similar lines, through this CSS: [id]:hover:after { content: " #" attr(id) " "; font-size: 50%; color: #ccc; text-decoration: none; } which means that everything with an id attribute gets its anchor advertised when you mouse over it; try the front page and the RSS tutorial for examples. I prefer this to the “purple” approach,...
Friday, 28 May 2004
RFC 3744 has been published: This document specifies a set of methods, headers, message bodies, properties, and reports that define Access Control extensions to the WebDAV Distributed Authoring Protocol. This protocol permits a client to read and modify access control lists that instruct a server whether to allow or deny operations upon a resource (such as HyperText Transfer Protocol (HTTP) method invocations) by a given principal. A lightweight representation of principals as Web resources supports integration of a wide range...
Monday, 3 May 2004
It looks like the HTTP PATCH method proposal might be based on Delta Encoding, which is IMO one of the cooler and lesser-known HTTP technologies. I've heard of a few implementations of delta, including one by Patrick McManus a long time ago that I believe shows up in Navisite's DeltaEdge [pdf]. It’s a win-win; less spec work has to be done, implementations can take advantage of existing code if they have it, and it increases the exposure of Delta Encoding,...
Tuesday, 27 April 2004
In the past, I’ve talked about reusing WSDL as a format for describing Web resources, as well as coming up with a bespoke format. One path that I’ve overlooked so far is reusing WebDAV to describe Web resources. The WebDAV protocol defines and allows you to describe the properties of a Web resource; e.g., the last-modified time is such a property, and so is the fact that it requires authentication. WebDAV also defines a way to get a representation of...
Monday, 19 April 2004
One of the things that people find compelling about Web services is its promise of asynchrony. “HTTP is only request/response, and therefore synchronous; it’s terrible for long-lived business processes, where the server needs to contact the client at some arbitrary time in the future” they say. A single HTTP request/response is indeed synchronous, so that that level this argument holds. In the common case, one party (the server) can’t send messages to the other (the client) without receiving a corresponding...
Friday, 16 April 2004
To use WSDL to describe RESTful interactions, you need some way of accommodating generative resource identifiers. In a nutshell, this means some part of the URI is dynamic. For example, with HTTP I might describe an address book where someone named “Jones” has a corresponding entry URI; http://www.example.org/people/Jones When describing the overall interface on offer, it’s not that interesting to talk about this particular resource; rather, you want to describe the interface provided for all people. To do this, you...
Sunday, 28 March 2004
Ian Hickson is thinking about client-side technologies (scroll down a bit). Some of his ideas resonated; Sortable tree views and list views with rich formatting. Do a search on eBay, and you’ll find you have to hit the server whenever you want to change the sort order. That shouldn’t be necessary. +1; after reading Adam’s thoughts on client-side stuff, I played around with the idea of annotating HTML tables to allow them to be sorted by a particular column or...
Wednesday, 17 March 2004
In an otherwise excellent article, Jon Udell blames the lack of one-click subscribe in syndication formats on lack of vision; How users will interact with the formats and APIs is left as an exercise for the implementer. But of course that’s where the rubber meets the road. So syndication still lacks a well-known mechanism for one-click subscribe. In fact, there’s a clear path to one-click subscribe in syndication. Here’s how: If every syndication format were served with a well-known media...
Sunday, 15 February 2004
I’ve published a revision of the Caching Tutorial for Web Authors and Webmasters, the first non-trivial edit in some time almost since I wrote it in 1998. That said, there aren’t any substantial changes; this is mostly tweaking and incorporation of new information. The bigger modifications include: - Replacing “Object” with “Representation” as appropriate - New information on Gateway Caches and Content Delivery Networks - Incorporation of various corrections and additional information received - Creative Commons licensing I’m sure there...
Tuesday, 3 February 2004
Just got some mail regarding the Cacheability Engine which led me to NetKernel; NetKernel is a Java-based virtual REST operating system for internet applications. NetKernel is a scalable microkernel which implements a modular REST abstraction in which all software components are URI addressable REST-services. Very interesting stuff; it’ll be instructive for many to be able to compare REST and SOA for the purposes of integration, etc. at something other than a theoretical level; while it’s been possible for quite some...
Monday, 8 December 2003
HTTP provides considerable benefits to Web applications that take advantage of it; everything from scalability (through caching), client-integrated authentication, automated redirection, multiple format support and lots more. I’ve been drafting some entries about how cool all of these things are; I’m going to try to get a few up in the coming weeks. As I’ve been writing, however, I’ve noticed a common thread — just about every one of them is difficult to realise using existing server-side technology. Web Metadata...
Sunday, 7 December 2003
Adam asks if there’s a description format for REST. I don’t know of any that have wide acceptance (and I think the hard-core RESTafarians will answer “REST is self-describing, that’s the point” ;) but I have been noodling on something for my own purposes. So far, I have a sample XML file that describes the RESTLog API (with some modifications to make it more interesting), as well as a stylesheet to do code generation for Tarawa (which is also still...
Saturday, 13 September 2003
I shudder when I see these words. Everyone I’ve asked has, at least once, gotten two orders of something online (personally, I’ve had the SonyEricsson store ship *three* duplicate orders); “Click Submit Only Once” is intended to stop that. The problem is, it puts me and every other shopper between a rock and a hard place. That’s because if there’s any problem with my browser, my computer’s network connection, the Internet itself or the server, I don’t know the status...
Thursday, 21 August 2003
I've heard several people in the industry assert that HTTP fundamentally limits the performance of applications that use it; in other words, there's a considerable disadvantage to using it, and that therefore other protocols (usually proprietary or platform-specific systems that those same people happen to sell) are needed to "unleash the power of Web services." Specifically, it's common wisdom that HTTP maxes out at a few hundred requests per second per CPU, give or take, while the requirements for "enterprise-class"...
Monday, 11 August 2003
I just found a draft finding that the W3C TAG published about a month ago, regarding the use of metadata in URIs. This is very cool, and I especially like the emphasis on authorities’ ability to embed metadata in URIs. To me, though, it begs a big question. Right now, there isn’t any normal way for an authority to describe how that metadata is structured; one has to resort to more prosaic descriptions, which doesn’t really scale. What I’ve wanted...
Monday, 4 August 2003
Marc Hadley points out that the version of iDisk in OSX Panther looks like it will enable offline functionality with caching; it also looks to do some synchronization. I'd really like to have a look at how they've done it. Hey Apple, how about opening up the specs and the client so that we can do it with any site?...
Tuesday, 29 July 2003
Ted Leung points out that caching PUT (and other WebDAV methods) would suit Subversion - probably the most interesting WebDAV application under open development - quite well. The only thing he says that I disagree with (and it might just be a misunderstanding) is in regard to a need for a Subversion-specific client cache; the whole point of doing this with Web protocols it to avoid application-specific infrastructure. A well-designed WebDAV cache should work equally well for any application, not...
Tuesday, 29 July 2003
Mark Baker is the latest in a series to weigh in on the TAG issue regarding what a HTTP URI can identify. I haven't followed the debate closely, but it appears that the arguments haven't changed substantively. Another way of stating this issue is asking whether a URI scheme fundamentally constrains the types of resources it identifies. If you look at it that way, the scheme component of a URI is fairly special; it's a declaration of support for a...
Saturday, 26 July 2003
If we WebDAV-enable Web applications, people will be able to interact with them like filesystems. To blog something, you'd be able to write an entry in the text editor of your choice, and then drag-and-drop them into what MSFT has called "Web folders." What happens, however, when you're offline? Right now, you get an error. Sure, you could write that article, save it on your local filesystem, and then drag-and-post it later when you connect, but who wants the hassle?...
Saturday, 26 July 2003
One of my personal background tasks in the last couple of months has been finding sample applications to excercise Tarawa with. Although my load is high and I've only got a single processor - me - I'm still trying to push this. One of those that I've been considering has been a blog engine, where each post has a URI and therefore a corresponding Resource instance, and so forth. One of my other goals has been to WebDAV enable Tarawa,...
Friday, 25 July 2003
Mark Pilgrim is starting to think about issues surrounding the transport, transfer and general moving around of the Format Formerly Known as Echo (nee Pie). This feels suspiciously like a profile of HTTP. We tried that a little bit in the Basic Profile, and I'm still undecided as to whether it was worthwhile or not. That having been said, some thoughts and suggestions: * Move away from RFC2119 language (MUST, SHOULD, MAY and friends) and make it more advisory. HTTP...
Saturday, 28 June 2003
I feel compelled to respond to Norm Walsh's thoughts on caching. It's important to distinguish between the capabilities of a specific product (such as WWWoffle) and the technology that it implements (caching). I would agree that the general state of cache implementation leaves much to be desired, both in clients and in proxies. However, I would not write off the technology wholesale. For example, Norm claims that you can't populate a cache. Untrue; it's simple to populate any cache, simply...
Tuesday, 17 June 2003
What does this interesting new, ad hoc work have to do with this interesting , new standards work and this interesting, new-ish effort by GK?...
Thursday, 12 June 2003
Having a network-enabled (even if only through BlueTooth and infrared) is a heady experience; the ability to access the Web and sync applications from anywhere - really anywhere - is quite liberating. However, after playing with the new toy for a few days, I'm left wanting more. I want to sync my addresses and todo lists with online versions that now live in a LDAP directory and a Wiki, not Palm Desktop or Outlook. I want to edit my Weblog...
Thursday, 29 May 2003
Jonathan Rosenberg published a new Internet-Draft, XCAP, to the SIMPLE Working Group in the IETF. Here's the skinny: Abstract This specification defines the Extensible Markup Language (XML) Configuration Access Protocol (XCAP). XCAP allows a client to read, write and modify application configuration data, stored in XML format on a server. XCAP is not a new protocol. XCAP maps XML document sub-trees and element attributes to HTTP URIs, so that these components can be directly accessed by HTTP. [...] With a...
Thursday, 8 May 2003
I know at least one person who will think that this is a good idea. Anybody else? I'd looove to do this work......
Monday, 5 May 2003
I've finally gotten sick enough of a project that I've been working on for waaaay too long to release it to the unsuspecting^H^H^H general public. Tarawa is, in short, a HTTP server API that tries to apply some of the lessons we've learned recently in discussions regarding the Web architecture and REST. This release is for Python; the API is currently very Python-specific, but I expect that to change. It's also very alpha, but should be functional for playing with....
Saturday, 26 April 2003
For those who have been helping, it's alive, has been for almost a week, but I still want to do a bit more documentation, hunt down a few bugs, and get some more unit tests down. Unfortunately, I need more !@#$ memory for the iMac to do that (long story), and !@#$ Fry's sold me a dodgy 512M module. On top of that, Charlie was sick Thursday night, I was sick last night and still feel crap, and Anitra has...
Thursday, 24 April 2003
It's not necessary to lament the lack of ETags on generated Web pages; cgi_buffer automagically generates and validates them for Perl, Python and PHP scripts. Yes, I keep on shamelessly plugging this one, but the question keeps on coming up......
Tuesday, 8 April 2003
LiveHTTPHeaders for Mozilla is the best HTTP header sniffer I've seen yet; up till now, I've been using WebTee, but for *most* purposes, this is much better. Enjoy....
Thursday, 27 March 2003
Dave seems excited by Macromedia's announcement. I'm less enthusiastic. Not only does it lock you into one vendor, but their product, well, sucks. Anitra loaded up Flash MX on the trusty eMac the other week, to refamiliarize herself before teaching a lesson on it. I'd been saying "Don't worry, Flash is easy!" for weeks; then I looked at the UI. Needless to say, Flash is no longer easy; it's about the most counterintuitive interface I've ever seen. If this is...
Tuesday, 25 March 2003
Jorgen hits a subject that's of great interest to me; RSS standardization. I originally started the Syndication list to get RSS moving towards some sort of recognized standard; more recently, my effort to register an RSS media type was stalled by the lack of a stable spec published by a recognized group. More recently, there was an informal meeting of the minds in San Francisco last (northern) autumn regarding this topic. There was good participation from the blogging, RSS weenie...
Sunday, 26 January 2003
I always wondered why so many people had their blogs' comments and even trackback indicators turned off. Go ahead and surf around; it's a rare blog indeed, at least in my experience, that has these features visible for the world to see. It's obvious, now that i've actually had a blog (OK, it's about as healthy as the plant in the living room that I Really Should Water Soon, but I think this goes even for healthy blogs). It's that...
Wednesday, 27 November 2002
I'm extremely wary about the new prefetching feature in Mozilla. The Web caching community has tried this from about every angle, but the general consensus of professionals (with one notable exception) is that prefetching is a bad approach. For one thing, it assumes free bandwidth; not such a hot idea in a lot of places (e.g., Australia, where you pay per Mb). I've also had network and server administrators calling me in a panic because they're being flooded with...
Thursday, 12 September 2002
Jeremy Allaire talks about establishing a "rich client" platform because HTML is "stagnant." Two questions; will it be standards-based, and what about SVG? I'd really rather not have Macromedia succeed in its ambitions to be the next Microsoft......
