mark nottingham

How Web-Ready is XMLHttpRequest?

Monday, 23 January 2006

HTTP

I’ve been playing around with some ideas that use XMLHttpRequest recently, but I keep on bumping up against implementation inconsistencies on IE vs. Safari vs. Opera vs. Mozilla. Although the interface exposed is pretty much the same, what it does in the background is very different, especially with regards to HTTP.

For example, some implementations will handle redirects and cache validation for you, while others will pass through the HTTP status codes, expecting you to pick up the pieces. This gets really annoying when you’re trying to use the Web to your advantage, rather that just make HTTP into another RPC protocol (which so many AJAX apps do… but I digress).

Frustrated by this and with a bit of spare time last week, I put together a set of tests for HTTP functionality in XMLHttpRequest (and friends). It’s fairly rough right now, but it’s already been useful to me. The fun part is it’s implemented on the server side with plain files and .htaccess magic (mostly mod_rewrite and mod_asis).

So far, the most bizarre thing I’ve come across is Safari’s inability to pop up an authentication dialog if you’re using XMLHttpRequest synchronously, even though it’s happy to do so if you’re working async. Basic redirect support is pretty universal, but things quickly fall apart on most browsers when you do tricky things like use non-GET/POST methods on redirecting resources. IE 6 is actually one of the better implementations I’ve tried so far (maybe not so surprising, considering they came up with it).

Anyway, give it a try; I need to work on the documentation and some of the tests, especially around caching. Ideally, I’d like to get some sort of result collection / summary thing going (and I’ve left a hook for it), but for now, it’s interesting to just look at (and maybe print) the per-browser results.

Feedback and suggestions appreciated below.


29 Comments

M. David Peterson said:

Hi Mark,

Very cool!

One thing that might be worth looking into is using the benefit of browser-based XSLT, now available in all major browsers, including Opera in the 9.0 release (currently preview). The bottom of this [http://www.xsltblog.com/archives/2005/12/finally_someone_1.html] article/post goes into greater detail of how you can use this method to point to separate configuration files for each browser that allow you to take a browser specific debugging/functionality development approach instead of being forced to deal with creating a one browser fits all solution.

A working sample found at the bottom of the above link that simply points out the browser type (and “brand” in the case of Mozilla) if it properly passes the “non-spoofing” test (not a necessary piece, but was built in to showcase the fact that the browser can be expected to do things like self validation, removing any type of extended work from the server and freeing up resources in doing so) is here [http://channelxml.com/explorations/SessionConfig.aspx] This, of course, is easy enough to extend to implement for browser specific config files, etc…

Tuesday, January 24 2006 at 2:03 AM

Joe Gregorio said:

Hey Mark, I’ve been doing some HTTP client testing myself recently:

http://bitworking.org/projects/httplib2/

in particular: http://bitworking.org/projects/httplib2/httplib2test.py.txt

One of the things this begs for is some sort of common markup for writing these kinds of tests in a language independent manner.

One of your tests is incorrect I believe. “GET to a 301 Permanent Redirect” does not require user prompting for GET or HEAD. Same thing for a 302 and 307.

Tuesday, January 24 2006 at 7:45 AM

Joe Gregorio said:

Mark,

D’oh, I missed that.

Tuesday, January 24 2006 at 8:53 AM

M. David Peterson said:

Why should thousands (if not millions) of Web developers be required to reverse-engineer what a handful of software vendors do, and then hope those vendors won’t change what they’ve done in the next release?

If they change what they do in the next release you would have to rewrite no matter what. Developing one core solution doesn’t protect you from future releases of ANY vendor.

My suggestion is not to write extensive deviations from the core solution, and instead a core solution that can allow for each browser’s differences without forcing ALL browsers to work to the lowest common denominator, or writing extensive work arounds for one platform and because of this cause performance to degrade due to additional processing and/or over the wire bandwidth.

No matter which way your argue this point, you will spend more time attempting to solve this with one core solution, that allowing for the problem to be broken out into smaller pieces.

My return question to you: Why should these same supposed millions of developers be forced into a position of beating their head against a wall attempting to create a one size fits all solution when one size doesn’t fit all?

The devil is in the details. Solve as much of the problem as you can with one core code set, and then let each platform deal with its own details instead of making all of them deal with each others details as well.

Tuesday, January 24 2006 at 8:56 AM

M. David Peterson said:

I agree with all of what you’ve stated. I wonder if we are simply coming at this from two different directions, but have the same ideals in mind?

I agree 100% that having a base of standards that we can rely on to work across all platforms is something that we should continue to strive for. The work that you and the rest of the folks you are involved with in regards to building a base of standards that the rest of us can build from is something I have EXTREME admiration for, and plan to continue my own development efforts with a focus smack dab in the middle of all of them.

Unfortunately, as you have uncovered, the inconsistencies with the various implmentations of XMLHTTP across browsers/platforms is something that creates a need to implement various hacks to bring at least some level of consistancy and expectation, no matter which browser a site visitor might choose to use.

I can see where my original point left a big wide gaping area of interpretation that pointed heavily towards a suggestion that deviation from the core solution should be the primary focus, instead of the last resort. That’s my bad. I should have been quite a bit more clear.

Ultimately thats really the point I was/am trying to make. Simple things like differences in CSS support, or better said, interpretation of the CSS spec lead to ‘!important’ hacks, and various other ways to tell IE to do one thing and the other browsers another. The way the XMLHTTP object is instantiated, or the various little intricacies that cause a need to implement try/catch object creation (althouh some smart folks as of late have helped reduce such tactics to a minimum) to determine the platform and therefore how to go about implementing a call to the server via XMLHTTP or XmlHttpRequest(), etc… things like this that increase the over-the-wire and client-side processing bandwidth can be helped along by implementing simple extensions to the core code base that are specific to each platform.

Ultimatelly the focus should be to bring a level of cross-browser/platform consistancy such that no matter what, we can always count on fairly simple things like this to just work. It’s good to see that the right people such as yourself are spearheading this movement, as I believe its a REALLY important piece that needs to be handled by the folks who know who to handle things of this nature.

Cheers :)

Tuesday, January 24 2006 at 10:34 AM

Julian Reschke said:

What a great idea, thanks.

Here’s one thing I know Microsoft is doing incorrectly and which ma be worth testing.

When submitting PROPFIND to a URI and getting a 3xx, the object will (rightfully) try to follow the redirect. However, it will fail to resend the request body, and thus cause a hang if it wasn’t empty in the first place (lesson: only follow redirects if the original request didn’t have a request body, or if the library keeps it for the retry).

Another thing I noticed missing from Firefox’ implementation is a way to read the content of a response body binary (Microsoft allows getting a stream object).

Best regards, Julian

Tuesday, January 24 2006 at 11:47 AM

ingo said:

I’m not sure if that can be fixed on your side, but I found the request dialogs difficult to interpret. “Yes”/”No” instead of “Cancel”/”OK” would have helped me, at least.

Also, do I read one of your comments correctly, that an “allowed to redirect” dialog is supposed to pop up several times? I only got it once and then several questions, where I assumed that I should answer yes, because there had been a request the first time…

Tuesday, January 24 2006 at 12:08 PM

Stefan Eissing said:

Neat idea and implementation.

I wonder however, if Content-Encoding can be handled transparently to the caller. In my own HTTP Java lib I started with this feature and removed it later on. Applications using my lib got confused about the headers. If one passes them up unchanged and transforms the content, they do not fit together. Content-Length is the first which comes to mind and could be fixed by the lib, ETag however is more tricky.

In my experience ETags for “plain” content and content-encoded versions need to differ (if they are handed out). Otherwise things like If-Range headers become ambigous, e.g. the server cannot be certain if you are talking about the encoded or plain content. (I will happily hear that I am wrong on this, btw.)

I think Transfer-Encoding really does what people want to achieve with Content-Encoding. It’s hop by hop, so it works most beautiful with caching and ranges.

To summariye: content-encoding should not be handled transparently on a request/response API level.

Tuesday, January 24 2006 at 12:23 PM

Yves said:

I also have a list of HTTP related tests, at http://jigsaw.w3.org/HTTP/ some of them dedicated to clients, like redirect and Content-Location tests.

Thursday, January 26 2006 at 12:51 PM

Steven Williams said:

Another thing you may want to test is the various browsers over HTTPS. On my current project we have run into a problem with IE6/HTTPS/POST, which unfortunately is one of our most common usage scenarios. I would be interested if other people have experienced this and the workarounds (besides using GET).

Thursday, February 9 2006 at 2:18 AM

Steven Williams said:

Another thing you may want to test is the various browsers over HTTPS. On my current project we have run into a problem with IE6/HTTPS/POST, which unfortunately is one of our most common usage scenarios. I would be interested if other people have experienced this and the workarounds (besides using GET).

Thursday, February 9 2006 at 2:19 AM

Johan Sundström said:

I’d absolutely love a table with the outcome of your tests on one axis and a busload of browsers and browser versions on the other. Great work!

Friday, February 10 2006 at 1:05 AM

Ben said:

Neat experiment. I also did an experiment to see if I could send the right Authentication headers manually. The experiment is located at http://www.dichotomize.com/czmap/protect.html .

Friday, February 10 2006 at 8:55 AM

Troels said:

A results table would be so useful - Can’t you log the results on the serverside ?

Wednesday, March 22 2006 at 12:06 PM

good said:

Tests don’t even run on Netscape 6.2.3-7.2.

Tuesday, April 25 2006 at 1:56 AM

Yuri said:

What about the case where the server called using XMLHTTPRequest returns a redirect response, but I want it to be followed by the browser and not by XMLHTTPRequest? Is that even possible with XMLHTTPRequest? If yes, would that be a another test case for your XMLHTTPRequest test page?

In my context, I have a page that sends lots of AJAX requests to update state on the server. However, if the session on the server expired or the user logged out in concurrently, I would like to simply redirect the browser to the login page in the same way the browser would redirect in a non-Ajax scenario. The last Ajax update that is lost is not a concern at this point.

Later, I am considering rendering the login page redirected and retuned by XMLHTTPRequest as a lightbox, or something like that, so that the re-authentication happens in the same page and the last update can be appropriately re-submitted.

Forgive me for digressing to a related but off topic point.

Friday, July 7 2006 at 10:08 AM

Marcus Granado said:

@ Julian, 24 January 2006

There is a way to download a binary stream in Mozilla/Firefox using XMLHttpRequest: have a look at mgran.blogspot. com/2006/08/downloading-binary-streams-with.html on details about it. []s, Marcus

Monday, August 21 2006 at 5:54 AM

lemmett said:

The best I’ve been able to come up with for the redirect-to-login-page problem is as follows.

I tweaked the login page we redirect to so that it returns an ‘X-session-timeout: true’ header. Then I extended the class we use for XMLHttpRequest()’s so that if the final resolved result has that header, it sets window.location to the URL of the login page.

The big disadvantage is that I have to hardcode the URL to redirect to, so if the URL changes I’ve got to tweak the library and having multiple different pages that get redirected to isn’t very practical.

I’ve considered adding a second custom header, say X-current-url on the page redirected to. That could be used to determine what the value of window.location should be set to so we could have multiple targets without having to hard code it on the client side, but so far we haven’t needed more than the one and I haven’t gotten around to making it more generic on principle.

Friday, October 20 2006 at 4:09 AM

Jon Hanna said:

Sweet. For a start it answered all the questions I had come up with last night about XMLHttpRequest implementations and saved me all the tests I was planning on writing.

Great to have this stuff published. Nice work from Mark as usual.

Thursday, January 18 2007 at 8:35 AM

Jon Hanna said:

One note.

“Get: fail (Method sent: GET)”

The Working Draft currently indicates that methods that match GET, POST, HEAD, PUT or DELETE in a case-insensitive comparison should be adjusted to upper case.

So this would be correct behaviour if that draft is made a Recommendation. Arguably this is a bug in the spec, but it wouldn’t be a bug in the XMLHttpRequest implementation.

Adjusting Foo to FOO is still a bug though.

Tuesday, January 23 2007 at 8:07 AM

Jonathan Arkell said:

Hey Mark

Good Job. Your test turns out to be an excellent resource if you are building any HTTP client library.

How do you feel about 3rd party tools connecting it to run a suite of tests? For instance: I am hacking at snows http library so it will properly handle put, delete, etc, and I want to use your library to test some of its functionality. Would it be alright if I utilized your tests in an automated suite?

Cheers!

Sunday, June 24 2007 at 11:08 AM

ilango said:

Hello

I made a REST web service.When send request from client to server using xmlHttpRequest(“get”,”https://192.168.1.2”,”false”) the following error will occur.

Exception… “Access to restricted URI denied” code: “1012” nsresult: “0x805303f4 (NS_ERROR_DOM_BAD_URI)”.

It run fine in same machine.But different machine the error made on.pls give solution.

Thursday, August 7 2008 at 11:34 AM