User-agent: !Susie Disallow: / User-agent: !Susie (http://www.sync2it.com/susie) Disallow: / User-agent: 2Bone_LinkChecker Disallow: / User-agent: Accoona AI Agent Disallow: / User-agent: AdoptionMedia Disallow: / User-agent: aipbot Disallow: / User-agent: Aipbot Disallow: / User-agent: aipbot/1.0 Disallow: / User-agent: aipbot/2-beta Disallow: / User-agent: aipbot/2-beta (aipbot dev; http://aipbot.com; aipbot@aipbot.com) Disallow: / User-agent: aipbot/1.0 (aipbot; http://www.aipbot.com; aipbot@aipbot.com) Disallow: / User-agent: Alexibot Disallow: / User-agent: ALJ Disallow: / User-agent: ALJ Feb 13 2005 alexf2000@yandex.ru Disallow: / User-agent: Alpha Search Agent Disallow: / User-agent: AnswerBus Disallow: / User-agent: AnswerBus (http://www.answerbus.com/) Disallow: / User-agent: appie 1.1 Disallow: / User-agent: appie 1.1 (www.walhello.com) Disallow: / User-agent: Aqua_Products Disallow: / User-agent: b2w/0.1 Disallow: / User-agent: BackDoorBot/1.0 Disallow: / User-agent: Baiduspider Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: Baiduspider+(+http://www.baidu.com/search/spider.htm) Disallow: / User-agent: BDFetch Disallow: / User-agent: BecomeBot Disallow: / User-agent: Bilbo Disallow: / User-agent: Blockerades av Norton Disallow: / User-agent: BlockNote.net Disallow: / User-agent: BlowFish/1.0 Disallow: / User-agent: boitho.com dc Disallow: / User-agent: BotALot Disallow: / User-agent: BuiltBotTough Disallow: / User-agent: Bullseye/1.0 Disallow: / User-agent: BunnySlippers Disallow: / User-agent: ccubee Disallow: / User-agent: cfetch Disallow: / User-agent: Checkbot Disallow: / User-agent: CheeseBot Disallow: / User-agent: CherryPicker Disallow: / User-agent: CherryPickerElite/1.0 Disallow: / User-agent: CherryPickerSE/1.0 Disallow: / User-agent: citenikbot Disallow: / User-agent: CFNetwork Disallow: / User-agent: combine Disallow: / User-agent: Combine Disallow: / User-agent: COMBINE Disallow: / User-agent: COMBINE/0.0 Disallow: / User-agent: Combine/2.0 Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: cosmos Disallow: / User-agent: cosmos/0.9_(robot@xyleme.com) Disallow: / User-agent: cosmos/0.8_(robot@xyleme.com) Disallow: / User-agent: Crescent Disallow: / User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Disallow: / User-agent: Crescent Internet ToolPak HTTPOLE Control v.1.0 Disallow: / User-agent: csci_b659 Disallow: / User-agent: CSHttpClient Disallow: / User-agent: curl Disallow: / User-agent: Curl Disallow: / User-agent: curl/7.10.x Disallow: / User-agent: curl/7.10.x (i386-redhat-linux-gnu) libcurl/7.10.x OpenSSL/0.9.7a ipv6 zlib/1.1.4 Disallow: / User-agent: curl/7.7.x Disallow: / User-agent: curl/7.7.x (i386--freebsd4.3) libcurl 7.7.x (SSL 0.9.6) (ipv6 enabled) Disallow: / User-agent: curl/7.8 Disallow: / User-agent: curl/7.8 (i686-pc-linux-gnu) libcurl 7.8 (OpenSSL 0.9.6) Disallow: / User-agent: curl/7.9.x Disallow: / User-agent: curl/7.9.x (win32) libcurl 7.9.x Disallow: / User-agent: DA 7.0 Disallow: / User-agent: DataCha0s Disallow: / User-agent: DataparkSearch Disallow: / User-agent: DataparkSearch/4.xx Disallow: / User-agent: DataparkSearch/4.xx (http://www.dataparksearch.org/) Disallow: / User-agent: DittoSpyder Disallow: / User-agent: DoCoMo Disallow: / User-agent: Doris Disallow: / User-agent: dumbot Disallow: / User-agent: eagle Disallow: / User-agent: EasyDL Disallow: / User-agent: EbiNess 0.1a Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: Enterprise_Search Disallow: / User-agent: Enterprise_Search/1.0 Disallow: / User-agent: envolk Disallow: / User-agent: envolk[ITS]spider Disallow: / User-agent: envolk[ITS]spider/1.6 Disallow: / User-agent: EroCrawler Disallow: / User-agent: es Disallow: / User-agent: eStyleSearch 4 Disallow: / User-agent: Exabot Disallow: / User-agent: Expired Domain Sleuth Disallow: / User-agent: ExtractorPro Disallow: / User-agent: Extreme Picture Finder Disallow: / User-agent: FairAd Client Disallow: / User-agent: FAST Enterprise Crawler Disallow: / User-agent: FAST Enterprise Crawler 6 Disallow: / User-agent: FAST Enterprise Crawler 6 used by Schibsted Sok (webcrawl@schibstedsok.no) Disallow: / User-agent: FAST MetaWeb Crawler Disallow: / User-agent: FAST MetaWeb Crawler (helpdesk at fastsearch dot com) Disallow: / User-agent: FAST WebCrawler Disallow: / User-agent: FavOrg Disallow: / User-agent: Findexa Crawler Disallow: / User-agent: Findexa Crawler (http://www.findexa.no/gulesider/article26548.ece) Disallow: / User-agent: findlinks Disallow: / User-agent: Flaming AttackBot Disallow: / User-agent: Foobot Disallow: / User-agent: full_breadth_crawler Disallow: / User-agent: full_breadth_crawler (zoidberg.ucr.edu; Linux i686; http://ivia.ucr.edu/user_agents.html) Disallow: / User-agent: g2Crawler Disallow: / User-agent: g2Crawler (nobody@airmail.net) Disallow: / User-agent: g3.pl Disallow: / User-agent: Gaisbot Disallow: / User-agent: Gaisbot/3.0 Disallow: / User-agent: Gaisbot/3.0 (indexer@gais.cs.ccu.edu.tw; http://gais.cs.ccu.edu.tw/robot.php) Disallow: / User-agent: genieBot Disallow: / User-agent: genieBot (http://64.5.245.11/faq/faq.html) Disallow: / User-agent: geniebot wgao@genieknows.com Disallow: / User-agent: GetRight/4.2 Disallow: / User-agent: grub Disallow: / User-agent: grub-client Disallow: / User-agent: GT::WWW Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: Hatena Antenna Disallow: / User-agent: head scan.pl Disallow: / User-agent: hloader Disallow: / User-agent: HooWWWer Disallow: / User-agent: ht: Disallow: / User-agent: HTMLParser Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-agent: httplib Disallow: / User-agent: humanlinks Disallow: / User-agent: Hypatia LDB Disallow: / User-agent: iaskspider Disallow: / User-agent: IEFav172Free Disallow: / User-agent: Incutio HttpClient v0.9 Disallow: / User-agent: InetURL Disallow: / User-agent: InetURL: Disallow: / User-agent: InfoNaviRobot Disallow: / User-agent: INGRID Disallow: / User-agent: internet=money Disallow: / User-agent: IP*Works! Disallow: / User-agent: IP*Works! HTTP Disallow: / User-agent: IP*Works! V5 HTTP Disallow: / User-agent: IRLbot Disallow: / User-agent: Iron33/1.0.2 Disallow: / User-agent: ISC Systems iRc Search 2.1 Disallow: / User-agent: Java Disallow: / User-agent: JennyBot Disallow: / User-agent: Jetbot Disallow: / User-agent: Jetbot/1.0 Disallow: / User-agent: JobSpider_BA Disallow: / User-agent: Kenjin Spider Disallow: / User-agent: Keyword Density/0.9 Disallow: / User-agent: LapozzBot Disallow: / User-agent: larbin Disallow: / User-agent: larbin (pradipjadav@gmail) Disallow: / User-agent: larbin_2.6.3 Disallow: / User-agent: larbin_2.6.3 (larbin2.6.3@unspecified.mail) Disallow: / User-agent: larbin_2.6.3 (ltaa_web_crawler@groupes.epfl.ch) Disallow: / User-agent: larbin_test Disallow: / User-agent: larbin_test (nobody@airmail.etn) Disallow: / User-agent: LBot Disallow: / User-agent: lc Disallow: / User-agent: LG C1100 MIC Disallow: / User-agent: Libby_1.1 Disallow: / User-agent: libWeb/clsHTTP Disallow: / User-agent: libwww perl Disallow: / User-agent: libwww-perl/5.65 Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: LinkLint Disallow: / User-agent: LinkLint checkonly Disallow: / User-agent: Links Disallow: / User-agent: Links (2.1pre15; NetBSD 2.99.11 amd64; x) Disallow: / User-agent: Links 2.0 Disallow: / User-agent: Links SQL Disallow: / User-agent: LinkScan Disallow: / User-agent: Links SQL (http://gossamer-threads.com/scripts/links-sql/) Disallow: / User-agent: LinkWalker Disallow: / User-agent: LNSpiderguy Disallow: / User-agent: LWP::Simple Disallow: / User-agent: lwp trivial Disallow: / User-agent: MapoftheInternet.com Disallow: / User-agent: Mata Hari Disallow: / User-agent: Microsoft URL Control Disallow: / User-agent: Microsoft URL Control - 5.01.4511 Disallow: / User-agent: Microsoft URL Control - 6.00.8169 Disallow: / User-agent: Microsoft URL Control 6.00.8862 Disallow: / User-agent: Microsoft URL Control 6.01.9782 Disallow: / User-agent: Microsoft WebDAV MiniRedir Disallow: / User-agent: MIIxpc Disallow: / User-agent: MIIxpc/4.2 Disallow: / User-agent: Missigua Locator 1.9 Disallow: / User-agent: Mister PiX Disallow: / User-agent: Mizzu Labs 2.2 Disallow: / User-agent: MJ12bot Disallow: / User-agent: MnoGoSearch Disallow: / User-agent: Mnogosearch 3.1.21 Disallow: / User-agent: moget Disallow: / User-agent: moget/2.1 Disallow: / User-agent: MojeekBot Disallow: / User-agent: Moozilla Disallow: / User-agent: MOT A 1F Disallow: / User-agent: MOT V26X_ Disallow: / User-agent: MSIECrawler Disallow: / User-agent: MSNPTC Disallow: / User-agent: MSProxy Disallow: / User-agent: MVAClient Disallow: / User-agent: My Session 1 Disallow: / User-agent: My WinHTTP Connection Disallow: / User-agent: naver Disallow: / User-agent: Naver Disallow: / User-agent: NaverBot Disallow: / User-agent: NaverBot 1.0 Disallow: / User-agent: NaverBot-1.0 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) Disallow: / User-agent: NetAnts Disallow: / User-agent: NetSprint 2.0 Disallow: / User-agent: NewsTroveBot Disallow: / User-agent: NewsTroveBot (http://NewsTrove.com) Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: NextGenSearchBot 1 Disallow: / User-agent: NextGenSearchBot 1 (for information visit http://www.eliyon.com/nextgensearchbot) Disallow: / User-agent: NG Disallow: / User-agent: nicebot Disallow: / User-agent: NICErsPRO Disallow: / User-agent: Nogate Disallow: / User-agent: noxtrumbot Disallow: / User-agent: NuSearch Spider Disallow: / User-agent: NuSearch Spider (compatible; MSIE 6.0) Disallow: / User-agent: ObjectsSearch Disallow: / User-agent: oBot Disallow: / User-agent: OCP HRS 1.0 Disallow: / User-agent: OmniExplorer_Bot Disallow: / User-agent: onCHECK Linkchecker von www.scientec.de fuer www.onsinn.de Disallow: / User-agent: Openbot Disallow: / User-agent: Openfind Disallow: / User-agent: Openfind data gatherer Disallow: / User-agent: Openfind data gatherer, Openbot Disallow: / User-agent: PageBitesHyperBot Disallow: / User-agent: PEAR HTTP_Request class Disallow: / User-agent: PEAR HTTP_Request class ( http://pear.php.net/ ) Disallow: / User-agent: PeerFactor 404 crawler Disallow: / User-agent: PerMan Disallow: / User-agent: PHP version tracker Disallow: / User-agent: PHP version tracker (http://www.nexen.net/phpversion/bot.php) Disallow: / User-agent: PlantyNet_WebRobot_V1.9 Disallow: / User-agent: PlantyNet_WebRobot_V1.9 (dhkang@plantynet.com) Disallow: / User-agent: PMAFind Disallow: / User-agent: POE Component Client HTTP Disallow: / User-agent: Pompos Disallow: / User-agent: PostFavorites Disallow: / User-agent: PrivacyFinder Disallow: / User-agent: Program Shareware 1.0.0 Disallow: / User-agent: ProPowerBot/2.14 Disallow: / User-agent: ProWebWalker Disallow: / User-agent: psbot Disallow: / User-agent: PureSight Disallow: / User-agent: PyQuery Disallow: / User-agent: Python-urllib Disallow: / User-agent: QueryN Metasearch Disallow: / User-agent: Radiation Retriever 1.1 Disallow: / User-agent: RepoMonkey Disallow: / User-agent: RepoMonkey Bait & Tackle/v1.01 Disallow: / User-agent: research spider Disallow: / User-agent: RMA Disallow: / User-agent: RobotAgent Disallow: / User-agent: RSS_READER Disallow: / User-agent: RSS_READER (mctwist@mail.dr k.info) Disallow: / User-agent: RufusBot Disallow: / User-agent: RufusBot (Rufus Web Miner; http://64.124.122.252/feedback.html) Disallow: / User-agent: RX Bar Disallow: / User-agent: SBIder Disallow: / User-agent: SBIder/0.8-dev Disallow: / User-agent: Screw Ball Disallow: / User-agent: searchbot Disallow: / User-agent: searchpreview Disallow: / User-agent: Shim-Crawler Disallow: / User-agent: Shim-Crawler(Mozilla-compatible;+http://www.logos.ic.iu-tokyo.ac.jp/crawler/;+crawl@logos.ic.iu-tokyo.ac.jp) Disallow: / User-agent: ShopWiki Disallow: / User-agent: ShowTags Disallow: / User-agent: SiteSnagger Disallow: / User-agent: snap.com beta crawler v0 Disallow: / User-agent: Snoopy v1.2 Disallow: / User-agent: sohu agent Disallow: / User-agent: sohu search Disallow: / User-agent: sootle Disallow: / User-agent: SonyEricssonW800i Disallow: / User-agent: SpankBot Disallow: / User-agent: spanner Disallow: / User-agent: SpellWonder Crawler Disallow: / User-agent: Stanford Disallow: / User-agent: Stanford Comp Sci Disallow: / User-agent: SuperBot Disallow: / User-agent: SURF Disallow: / User-agent: SurveyBot Disallow: / User-agent: suzuran Disallow: / User-agent: SygolBot Disallow: / User-agent: SygolBot http://www.sygol.net Disallow: / User-agent: SynooBot Disallow: / User-agent: Szukacz/1.4 Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: Telesoft Disallow: / User-agent: The Intraformant Disallow: / User-agent: TheNomad Disallow: / User-agent: Thumbnail.CZ robot 1.1 Disallow: / User-agent: thumbshots de Bot Disallow: / User-agent: thumbshots de Bot (Version: 1.02, powered by www.thumbshots.de) Disallow: / User-agent: toCrawl/UrlDispatcher Disallow: / User-agent: True_Robot Disallow: / User-agent: True_Robot/1.0 Disallow: / User-agent: turingos Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Twiceler www.cuill.com Disallow: / User-agent: UdmSearch Disallow: / User-agent: UdmSearch/ Disallow: / User-agent: UdmSearch/3.1.21 Disallow: / User-agent: UP.Browser Disallow: / User-agent: updated Disallow: / User-agent: URL Control Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: URLy Warning Disallow: / User-agent: uö¨v Disallow: / User-agent: Vagabondo Disallow: / User-agent: Vagabondo/2.0 MT Disallow: / User-agent: VCI Disallow: / User-agent: VCI WebViewer VCI WebViewer Win32 Disallow: / User-agent: virus_detector Disallow: / User-agent: virus_detector (virus_harvester@securecomputing.com) Disallow: / User-agent: Visicom Toolbar Disallow: / User-agent: VoilaBot BETA 1.2 Disallow: / User-agent: Vortex Disallow: / User-agent: voyager Disallow: / User-agent: voyager/1.0 Disallow: / User-agent: VPF 5.0 Disallow: / User-agent: VSE Disallow: / User-agent: walnut Disallow: / User-agent: Watchfire WebXM Disallow: / User-agent: WDG_Validator Disallow: / User-agent: WebAlta Crawler Disallow: / User-agent: Web Downloader Disallow: / User-agent: Web Image Collector Disallow: / User-agent: WebAuto Disallow: / User-agent: WebBandit Disallow: / User-agent: WebBandit/3.50 Disallow: / User-agent: webcollage Disallow: / User-agent: webcollage.perl Disallow: / User-agent: WebCopier Disallow: / User-agent: WebEnhancer Disallow: / User-agent: WebForm 1 Disallow: / User-agent: webGobbler Disallow: / User-agent: WebmasterWorld Extractor Disallow: / User-agent: WebmasterWorldForumBot Disallow: / User-agent: WebPix 1.0 Disallow: / User-agent: WebPix 1.0 (www.netwu.com) Disallow: / User-agent: WebSauger Disallow: / User-agent: Website Quester Disallow: / User-agent: Webster Pro Disallow: / User-agent: WebStripper Disallow: / User-agent: WebVac Disallow: / User-agent: WebZip Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: WEP Search 00 Disallow: / User-agent: Wget Disallow: / User-agent: Wget/1.5.3 Disallow: / User-agent: Wget/1.6 Disallow: / User-agent: Wir sind die Borg Disallow: / User-agent: WordPress Disallow: / User-agent: wume_crawler Disallow: / User-agent: WWW-Collector-E Disallow: / User-agent: www.petitsage.fr site detector 0.4 Disallow: / User-agent: wwwster Disallow: / User-agent: Xenu Link Sleuth 1.2d Disallow: / User-agent: Xenu Link Sleuth 1.2e Disallow: / User-agent: Xenu Link Sleuth 1.2f Disallow: / User-agent: Xenu Link Sleuth 1.2g Disallow: / User-agent: Xenu's Disallow: / User-agent: Xenu's Link Sleuth 1.1c Disallow: / User-agent: Xerka MetaBot v1.0.0 [Gaia_sportic_20060118.1_EN] Disallow: / User-agent: xirq Disallow: / User-agent: Y!J BSC Disallow: / User-agent: Y!OASIS Disallow: / User-agent: yacy Disallow: / User-agent: yacy (www.yacy.net; v20040602; i386 Linux 2.4.31; java 1.5.0_05; Europe/de) yacy.net Disallow: / User-agent: yacy (x86 Windows 2000 5.0; java 1.5.0_04; Europe/de) yacy.net Disallow: / User-agent: yacy (www.yacy.net; v20040602; i386 Linux 2.4.31; java 1.4.2 03; Europe/en) yacy.net Disallow: / User-agent: yacy (i386 Linux 2.6.11-6mdk; java 1.4.2-rc1; America/en) yacy.net Disallow: / User-agent: yacy (ppc Linux 2.6.13-rc3; java 1.4.2; Europe/en) yacy.net Disallow: / User-agent: Z Add Link Checker Disallow: / User-agent: zerxbot Disallow: / User-agent: zerxspid Disallow: / User-agent: Zeus Disallow: / User-agent: Zeus 32297 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 66657 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 91033 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus Link Scout Disallow: / User-agent: Zeusbot Disallow: / User-agent: zspider Disallow: / User-agent: * Disallow: /cgi Disallow: /cgi-bin Disallow: /cgipro Disallow: /form.cgi Disallow: /counter Disallow: /Flash Disallow: /images Disallow: /imanager Disallow: /112750 Disallow: /land Disallow: /logwebsdata Disallow: /Miller-deCoux-Art Disallow: /Journal Disallow: /look Disallow: /sandrabrown Disallow: /test Disallow: /urchin Disallow: /farm.htm Disallow: /search Disallow: /search.htm Disallow: /messages Disallow: /dbasics/wwwboard/board/messages Disallow: /dbasics/wwwboard/general/messages Disallow: /dbasics/wwwboard/ADHD/messages Disallow: /dbasics/wwwboard/cp/messages Disallow: /dbasics/wwwboard/spiritual/messages Disallow: /dbasics/wwwboard/error.txt