User-agent: * Disallow: /admin/ Disallow: /admin_pages/ Disallow: /administration/ Disallow: /webservices/ Disallow: plughole.aspx # robots we're not interested in User-agent: InnovantageBot Disallow: / User-agent: Intelliseek Disallow: / User-agent: LMQueueBot Disallow: / User-agent: ContactBot Disallow: / User-agent: BaiDuSpider Disallow: / User-agent: DFusionBot Disallow: / User-agent: Gaisbot Disallow: / User-agent: Girafabot Disallow: / User-agent: polybot Disallow: / User-agent: Szukacz Disallow: / User-agent: NaverRobot Disallow: / User-agent: vspider Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-agent: Microsoft Scheduled Cache Content Download Service Disallow: / # robots we really don't want User-agent: ia_archiver Disallow: / User-agent: SlySearch Disallow: / User-agent: turnitinbot Disallow: / User-agent: BlackWidow Disallow: / User-agent: ChinaClaw Disallow: / User-agent: Custo Disallow: / User-agent: DISCo Disallow: / User-agent: Download Demon Disallow: / User-agent: eCatch Disallow: / User-agent: EirGrabber Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: Express WebPictures Disallow: / User-agent: ExtractorPro Disallow: / User-agent: EyeNetIE Disallow: / User-agent: FlashGet Disallow: / User-agent: GetRight Disallow: / User-agent: GetWeb! Disallow: / User-agent: Go!Zilla Disallow: / User-agent: Go-Ahead-Got-It Disallow: / User-agent: GrabNet Disallow: / User-agent: Grafula Disallow: / User-agent: HMView Disallow: / User-agent: HTTrack Disallow: / User-agent: Image Stripper Disallow: / User-agent: Image Sucker Disallow: / User-agent: Indy Library Disallow: / User-agent: InterGET Disallow: / User-agent: Internet Ninja Disallow: / User-agent: JetCar Disallow: / User-agent: JOC Web Spider Disallow: / User-agent: larbin Disallow: / User-agent: LeechFTP Disallow: / User-agent: Mass Downloader Disallow: / User-agent: MIDown tool Disallow: / User-agent: Mister PiX Disallow: / User-agent: Navroad Disallow: / User-agent: NearSite Disallow: / User-agent: NetAnts Disallow: / User-agent: NetSpider Disallow: / User-agent: Net Vampire Disallow: / User-agent: NetZIP Disallow: / User-agent: Octopus Disallow: / User-agent: PageGrabber Disallow: / User-agent: Papa Foto Disallow: / User-agent: pavuk Disallow: / User-agent: pcBrowser Disallow: / User-agent: RealDownload Disallow: / User-agent: ReGet Disallow: / User-agent: SiteSnagger Disallow: / User-agent: SmartDownload Disallow: / User-agent: SuperBot Disallow: / User-agent: SuperHTTP Disallow: / User-agent: Surfbot Disallow: / User-agent: tAkeOut Disallow: / User-agent: Teleport Pro Disallow: / User-agent: VoidEYE Disallow: / User-agent: Web Image Collector Disallow: / User-agent: Web Sucker Disallow: / User-agent: WebAuto Disallow: / User-agent: WebCopier Disallow: / User-agent: WebFetch Disallow: / User-agent: WebGo IS Disallow: / User-agent: WebLeacher Disallow: / User-agent: WebReaper Disallow: / User-agent: WebSauger Disallow: / User-agent: Website eXtractor Disallow: / User-agent: Website Quester Disallow: / User-agent: WebStripper Disallow: / User-agent: WebWhacker Disallow: / User-agent: WebZIP Disallow: / User-agent: Wget Disallow: / User-agent: Widow Disallow: / User-agent: WWWOFFLE Disallow: / User-agent: Xaldon WebSpider Disallow: / User-agent: Zeus Disallow: / User-agent: NPBot Disallow: /