# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-agent: * Crawl-delay: 10 Disallow: /shop/admin/ Disallow: /shop/cart/ Disallow: /shop/orders/ Disallow: /shop/user/ Disallow: /shop/422.html Disallow: /shop/500.html Disallow: /shop/maintenance.html