# This file will be ignored unless it is at the root of the host. # # After changing, perform a syntax check: # http://www.frobee.com/robots-txt-check # Each rule record has to start with at least one User-agent statement. Blank # lines delimit rule records and may not be used between User-agent and # Disallow statements. User-agent: * Crawl-delay: 10 Disallow: /