# Blocks robots from specific folders / directories User-agent: * Disallow: /cgi-bin/ Disallow: /archive/ Disallow: /_dev/ Disallow: /services/ Disallow: /_showcase/ Disallow: /_resources/ Sitemap: https://www.slu.edu/sitemap.xml User-agent: bingbot Crawl-delay: 9 User-agent: MJ12bot Crawl-Delay: 9