Re: [syndication] Aggregating your global content output into your blog

["Dave Winer" <dave@userland.com>]

The first two are pretty onerous assumptions. This is exactly what we were
trying to avoid, expressing compatibility in terms of products, instead of
formats and protocols. What is it about Movable Type that you depend on?
Maybe you should hook this into your aggregator -- that's how we did it in
Radio. The tool I posted a pointer to adds callbacks to the aggregator. The
usual disclaimers apply. Dave

----- Original Message -----
From: "Morbus Iff" <morbus@disobey.com>
To: <syndication@yahoogroups.com>
Sent: Tuesday, June 03, 2003 7:12 AM
Subject: Re: [syndication] Aggregating your global content output into your
blog


> >I routinely post to many different content systems on the web. I'd like
> >to aggregate all this into my blog. I suspect that I'm not the only
> >person who would like to do this.[1]
>
> I've done this with RSS feeds, but under the assumption that:
>
>  * you're using the MT metaWeblog API.
>  * all feeds have <dc:date> values.
>  * you run the script once a night.
>  * it's in Perl and requires modules.
>
> The script is at http://disobey.com/d/code/myrssmerger.pl. You
> can see an example of it's output with the two items on May 27th:
>
>   http://www.disobey.com/dnn/2003/05/index.shtml#001498
>
> Here's a quick run-through of it's use:
>
>  ./myrssmerger.pl --server http://disobey.com/cgi-bin/mt/mt-xmlrpc.cgi
>   --username morbus --password HAAHAHAH -blogid 1
>                   --showcategories
>
> The output is:
>
>  ----------------------------------------------------------------------
>   The following blog categories are available:
>
>   1: Disobey Stuff
>   2: The Idiot Box
>   3: CHIApet
>   4: Friends O' Disobey
>   5: Stalkers O' Morbus
>   6: Morbus Shoots, Jesus Saves
>   7: El Casho Disappearo
>   8: TechnOccult
>   9: Potpourri
>   10: Collected Nonsensicals
>
>  Category ID's can be used for --catid or -c.
>  ----------------------------------------------------------------------
>
> If you have no categories, you'll be told as such. Now, when you're
> actually posting to the blog, you can choose to post into a category or
not
> - if I wanted to post into "Disobey Stuff", I'd use either -c 1 or --catid
> 1. If I wanted no category, I'd specify no category.
>
> Here's one way I use this script. What I want to do is take all these
> different RSS feeds that have my data in it (from Gamegrene, from ORA,
> etc.) and have them all synchronized into my primary blog, DNN. So, I run
> this through cron every night:
>
>  ./myrssmerger.pl --server http://disobey.com/cgi-bin/mt/mt-xmlrpc.cgi
>   --username morbus --password HAAHAHAH -blogid 1
>                   --catid 1 http://gamegrene.com/index.xml
>
> In this case, I'm saying "ok, every night, check that URL for entries
> posted today, and if you see some, post to 'Disobey Stuff', referenced as
> category ID 1." I do something similar to my O'Reilly blog, only using
> --catid 8. An example of the above output looks like:
>
>  ----------------------------------------------------------------------
>  Downloading RSS feed at http://gamegrene.com/index.xml...
>   Publishing item: 'RPG, For Me'.
>   Skipping (failed date check): 'Just Say No To Powergamers'.
>   Skipping (failed date check): 'Every Story Needs A Soundtrack'.
>   Skipping (failed date check): 'The Demise of Local Game Shops'.
>   Skipping (failed date check): 'Death Of A Gaming System'.
>   Skipping (failed date check): 'What Do You Do With Six Million Elves?'.
>  ----------------------------------------------------------------------
>
> You can pass multiple URLs too. Say I've got thirty friends, and I want to
> check all their RSS feeds for new entries posted today. Any new entries
> should be posted to my blog with an id of 4 ('Friends of Disobey'):
>
>  ./myrssmerger.pl --server http://disobey.com/cgi-bin/mt/mt-xmlrpc.cgi
>   --username morbus --password HAAHAHAH -blogid 1
>                   --catid 4 http://gamegrene.com/index.xml
>                   http://researchbuzz.com/researchbuzz.rss
>   http://camworld.com/index.rdf
>
> The shortened output looks like:
>
>  ----------------------------------------------------------------------
>  Downloading RSS feed at http://gamegrene.com/index.xml...
>   Skipping (failed date check): 'RPG, For Me'.
>   Skipping (failed date check): 'Just Say No To Powergamers'.
>   Skipping (failed date check): 'Every Story Needs A Soundtrack'.
>  ----------------------------------------------------------------------
>  Downloading RSS feed at http://camworld.com/index.rdf...
>   Publishing item: 'Trinity's Hack from Matrix Reloaded'.
>   Skipping (failed date check): 'Siberian Desktop'.
>   Skipping (failed date check): 'The Sweet Hereafter'.
>  ----------------------------------------------------------------------
>  Downloading RSS feed at http://researchbuzz.com/researchbuzz.rss...
>   Skipping (no description/date): 'Northern Light Coming Back?'.
>   Skipping (no description/date): 'This Week in LLRX'.
>  ----------------------------------------------------------------------
>
> Finally, you add a --filter "" to the command line. The
> following will only post entries that match "perl":
>
>  ./myrssmerger.pl --server http://disobey.com/cgi-bin/mt/mt-xmlrpc.cgi
>   --username morbus --password HAAHAHAH -blogid 1
>                   --catid 4 --filter "perl" http://camworld.com/index.rdf
>
> As such, it skips over Cam's latest entry:
>
>   Skipping (failed filter): 'Trinity's Hack from Matrix Reloaded'.
>
> So, in this regard, you could pass 30 URLs on the command line, all
> filtered according to the word "perl" and all stuck in a category called
> "Perl", for instance. Assuming the RSS feeds had a <dc:date> entry, new
> items matching those requirements would be posted.
>
> --
> Morbus Iff ( softcore vulcan pr0n rulezzzzz )
> Culture: http://www.disobey.com/ and http://www.gamegrene.com/
> Tech: http://www.oreillynet.com/pub/au/779 - articles and weblog
> icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>