# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-Agent: * Disallow: /id Disallow: /jobs/new Disallow: /jobs/preview Disallow: /jobs/stage3 Disallow: /jobs/activate Disallow: /jobs/confirm