Re: [syndication] SOAP meets RSS

["Dave Winer" <dave@userland.com>]

Thanks Dan..

Hammering-avoidance is very much on my mind as our servers are getting
pounded by search engine crawlers in vast numbers and always at the worst
possible time. At the same time we're experiencing substantial growth in our
free Manila hosting service, and money is a big issue (as it is everywhere
these days it seems).

In that context, we designed this system so that:

1. Most bits are served statically.

2. The reliance on SOAP/XML-RPC is limited to smallish messages, usually
with just a few scalars and possibly an array. These travel over the wire,
even for people with relatively slow lines (like me), very quickly.

3. As much as possible use the CPU cycles on the recipient side, conserve
CPU cycles in the cloud so that it scales. That's the price you pay for good
current content, but it's an easy price for most to pay because the cycles
on the recipient end usually are being wasted (this is the P2P proposition).

I want other content types to be able to hitch a ride in a RSS channel. We
have a design for this, but it isn't written up yet. The goal is to make
exactly the types you mention, audio and video, flow around in higher
fidelity, optimizing the user experience and using the Internet when it
isn't so busy. It's largely a UI problem, the technology is brain-dead
simple. I'll post a note here for sure when it's written up.

Basically it flips around the Akamai equation. My goal isn't to get the bits
to you as fast as possible while you wait for them, but to have the bits
arrive before you even know they're there. ;->

Dave


> Interesting...
>
> How would you distinguish the RSS case from the general problem of
> caching and change notification for Web content? Would you propose a
> SOAP-based solution to the broader case of wanting to know straight away
> about changes to arbitrary Web-accessible chunks of data (images, audio,
> markup etc). Normally one would try to retrieve all this via HTTP caches
> to avoid hammering the original server; do you reckon it would it make
> sense for Web caches to use SOAP or similar to stay more up to date?