User-agent: Twitterbot Disallow: User-agent: * Disallow: /App_Config/ Disallow: /App_Data/ Disallow: /App_Browsers/ Disallow: /bin/ Disallow: /attachments/ Disallow: /data/ Disallow: /img/ Disallow: /layouts/ Disallow: /sitecore/ Disallow: /sitecore modules/ Disallow: /media/ Disallow: /temp/ Disallow: /upload/ Disallow: /xsl/ Disallow: /Design.config/ Disallow: /uploadedFiles/ crawl-delay: 10