# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-Agent: * Disallow: /area/8/search Disallow: /area/3/jobs/comp Disallow: /area/7/jobs/comp Disallow: /area/2/jobs/comp Disallow: /area/8/jobs/comp Allow: / Disallow: /area/2/search Disallow: /area/7/search Disallow: /area/1/jobs/comp Disallow: /area/6/jobs/comp Disallow: /area/3/search Disallow: /area/5/jobs/comp Disallow: /area/5/search Disallow: /area/1/search Disallow: /area/6/search User-Agent: Googlebot Disallow: /m/ Allow: / User-Agent: Googlebot-Mobile Disallow: / Allow: /m/