# Tries to stop all robots # derived from Nexor's robots.txt # See http://web.nexor.co.uk/users/mak/doc/robots/norobots.html # Original by Martijn Koster , # User-agent: spider name (Ultraseek, * for all) # Disallow: /directory/ (e.g. /icons/) User-Agent: msnbot Crawl-Delay: 10 User-agent: * Disallow: /images/ Disallow: /photos/ Disallow: /updates.htm