[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [syndication] Re: robots.txt and rss

To: syndication@yahoogroups.com
Subject: Re: [syndication] Re: robots.txt and rss
From: Ben Hammersley <ben@benhammersley.com>
Date: Fri, 8 Nov 2002 16:51:25 +0000
In-reply-to: <aqgn76+7n37@eGroups.com>

On Friday, November 8, 2002, at 04:02  pm, Dave Winer wrote:

opinion:  RSS aggregators probably *should* respect robots.txt.


I agree. If someone were to post a BDG, like the one [1] Simon Fell
posted for Etags, I would put together support for it in Radio
(assuming it was easy, which I assume it is).

Why would you have an rss feed, and an robots.txt that restricts usingit? Just don't publish an RSS feed.

Follow-Ups:
- Re: [syndication] Re: robots.txt and rss
  - From: Dan Brickley <daniel.brickley@bristol.ac.uk>
- Re: [syndication] Re: robots.txt and rss
  - From: Ian Davis <iand@internetalchemy.org>

References:
- Re: robots.txt and rss
  - From: "Dave Winer" <dave@userland.com>

Prev by Date: Re: [syndication] robots.txt and rss
Next by Date: Re: [syndication] Re: robots.txt and rss
Previous by thread: Re: [syndication] Re: robots.txt and rss
Next by thread: Re: [syndication] Re: robots.txt and rss
Index(es):
- Date
- Thread