User-agent: * Allow: Disallow: *?s=* Disallow: *mature* User-agent: bingbot Allow: Sitemap: https://crawlinfo.com/sitemap_index.xml