Dear robot authors

Gigabot is an example of a crawler which is trawling my site but doesn’t bother to put a URL with information about it in the user agent string. Dear robot authors — please be polite about what you’re doing on my site.

[tags: internet crawl]