Re: [syndication] Scraper code?

["Bill Kearney" <wkearney99@hotmail.com>]

> > A generic dispatcher would be interesting.  The complexity required of if,
> > however, would be quite daunting.

> What did you have in mind for a generic dispatcher? Browsers already do
> generic dispatching - so I assume you mean something else.
> What parts would be daunting? What issues are complex?
> If you provide a list of features, I can probably build a simple app pretty
> quickly. It is fairly easy on Windows for a client aggregator to register to
> handle a particular Content-Type.

I'm with you, on Windows there are some very cool things possible.  But it seems
many of the developers out there in RSS/RDF land have some strange phobia
against making use of it.  Thus they're all tormenting themselves trying to make
Mozilla jump through it's hoops.  <sigh>

As for a dispatcher's complexities, consider that 'catching' the transport of a
content type and 'routing' things based on it's contents could be rather
interesting.  Watch for certain structures within the XML and take actions
accordingly.  It'd be interesting to see something catching info from user
indentity (e-mail, vcard, foaf, etc), events, timestamped data, and the like.
But structuring the mechanism that did this in such a was as to not cause more
trouble than it's worth is a problem.  Considering that many tools assume
they're going to have to do all the 'heavy lifting' the dispatcher would never
see the data.  Abstracting the retrieval would help but it'd take considerable
efforts on the part of developers to wrestle with using the abstracted
interfaces.

Even simple stuff like detecting resource availability problems would benefit
here.  Trying to load a feed and it keeps timing out?  That'd be something a
user might want to know.  Or something an intermediary program could learn about
and seek alternative delivery mechanisms.  But alas, the level of integration
with the operating system is just much higher than most developers really want
to tangle with.

-Bill Kearney