User-agent: Fasterfox Disallow: / User-agent: * Disallow: *.js Disallow: /forum/clientscript/ Disallow: /forum/cpstyles/ Disallow: /forum/customavatars/ Disallow: /forum/customprofilepics/ Disallow: /forum/images/ Disallow: /forum/ajax.php Disallow: /forum/attachment.php Disallow: /forum/calendar.php Disallow: /forum/cron.php Disallow: /forum/editpost.php Disallow: /forum/global.php Disallow: /forum/image.php Disallow: /forum/inlinemod.php Disallow: /forum/joinrequests.php Disallow: /forum/login.php Disallow: /forum/misc.php Disallow: /forum/moderator.php Disallow: /forum/newattachment.php Disallow: /forum/newreply.php Disallow: /forum/newthread.php Disallow: /forum/online.php Disallow: /forum/poll.php Disallow: /forum/postings.php Disallow: /forum/printthread.php Disallow: /forum/private.php Disallow: /forum/profile.php Disallow: /forum/register.php Disallow: /forum/report.php Disallow: /forum/reputation.php Disallow: /forum/sendmessage.php Disallow: /forum/subscription.php Disallow: /forum/threadrate.php Disallow: /forum/usercp.php Disallow: /forum/usernote.php Disallow: /forum/arcade.php # Allow Archive.org to save snapshots of everything User-agent: ia_archiver Allow: / # Tame yahoo... it tends to eat a ton of resources without a delay User-agent: Slurp Crawl-delay: 60 #list individual pages and files here that all bots should ignore, as well as group extentions. #If you re-write everything to .html, you can disallow *.php, but note that if you don't have a CRR for custom pages, those will be blocked. User-agent: * Disallow: *.js Disallow: search.php Disallow: /includes/ Disallow: /install/ Disallow: /customavatars/ #Finally, list the path to your sitemap: Sitemap: http://florida3s.com/forum/sitemap_index.xml.gz