# Robots.txt file
# Info about robots.txt: http://www.robotstxt.org/wc/robots.html
# $Id: $

# Right now we tell all cralwer to hold for 2 seconds between each request
# This is to avoid being hogged down as our requests are heavy
# At least Google and MSN bots adhere to this.
# -- nicolasm 2007-08-30

User-agent: *
Crawl-delay: 2