# CommerceXchange robots file User-agent: * Allow: /admin/images/ Disallow: /CGI-BIN/ Disallow: /admin/ Disallow: /ajax/ Disallow: /ajaxCFC/ Disallow: /CDS/ Disallow: /administration/ Disallow: /admin/ Disallow: /adminnew/ Disallow: /bannerAdvert/ Disallow: /common/ Disallow: /components/ Disallow: /cachedpages/ Disallow: /clic2pay/ Disallow: /forwarding/ Disallow: /forwardingBuy/ Disallow: /guides/ Disallow: /messages/ Disallow: /program/ Disallow: /scripts/ Disallow: /SearchResults/ Disallow: /SQL/ Disallow: /Store/ Disallow: /Templates/ Disallow: /bot_trap/ Disallow: /tasks/ Sitemap: http://www.cxcommerce.com/googlesitemaps/sitemap_M_1.xml.gz User-agent: crawler4j Disallow: / User-agent: 007ac9 disallow: / User-agent: 007ac9 Crawler disallow: / User-agent: 2ADAMbot disallow: / User-agent: 2ADAMbot/1.0 disallow: / User-agent: 200PleaseBot disallow: / User-agent: 360Spider disallow: / User-agent: &as_qdr=all disallow: / user-agent: Abonti disallow: / user-agent: Abonti/0.92 disallow: / user-agent: abot v1.0 disallow: / user-agent: aboutthedomain disallow: / user-agent: Add Catalog disallow: / user-agent: Add Catalog/2.1 disallow: / user-agent: AdvBot disallow: / user-agent: AdvBot/2.0 disallow: / user-agent: Affectv Robot v1.0/Nutch-1.6 disallow: / user-agent: AhrefsBot disallow: / user-agent: Ahrefs-Bot disallow: / user-agent: Ahrefs-Bot/1.0 disallow: / user-agent: Aitellu disallow: / user-agent: Aitellu ForumSwarm disallow: / user-agent: Anonymous disallow: / user-agent: Anonymous/0.0 disallow: / user-agent: Apache-HttpClient/4.1.1 (java 1.5) disallow: / user-agent: Apache-HttpClient/4.1.3 (java 1.5) disallow: / user-agent: Associative Spider disallow: / User-agent: Baiduspider disallow: / User-agent: Baidu Spider disallow: / User-agent: Battleztar Bazinga disallow: / User-agent: Battleztar Bazinga/0.01 disallow: / User-agent: betaBot disallow: / User-agent: bieshu disallow: / User-agent: Bigli SEO disallow: / User-agent: Blackboard Safeassign disallow: / User-agent: Blazer 1.0 disallow: / User-agent: BLEXBot disallow: / User-agent: BLEXBot/1.0 disallow: / User-agent: BLP_bbot disallow: / User-agent: BLP_bbot/0.1 disallow: / User-agent: BOIA-Accessibility-Agent disallow: / User-agent: BOIA-Accessibility-Agent/PR 1.0 disallow: / User-agent: BOT for JCE disallow: / User-agent: BOT/0.1 (BOT for JCE) disallow: / User-agent: BUbiNG disallow: / User-agent: ca-crawler disallow: / User-agent: ca-crawler/1.0 disallow: / User-agent: CakePHP disallow: / User-agent: Calypso disallow: / User-agent: Calypso v/0.01 disallow: / User-agent: CamelHttpStream/1.0 Evolution/3.2.2 disallow: / User-agent: CB/Nutch-1.7 disallow: / User-agent: CCBot disallow: / User-agent: CCBot/2.0 disallow: / User-agent: chushou disallow: / User-agent: Cliqzbot disallow: / User-agent: Cliqzbot/0.1 disallow: / User-agent: CMS Crawler disallow: / User-agent: coccoc disallow: / User-agent: CoinCornerBot disallow: / User-agent: CoinCornerBot/1.1 disallow: / User-agent: Copyscape disallow: / User-agent: crawler4j disallow: / User-agent: CrazyWebCrawler disallow: / User-agent: CrazyWebCrawler-Spider disallow: / User-agent: curl/7.15.3 disallow: / User-agent: curl/7.26.0 disallow: / User-agent: curl/7.32.0 disallow: / User-agent: curl/7.37.0 disallow: / User-agent: curl/7.38.0 disallow: / User-agent: Curl/PHP 5.3.23 disallow: / User-agent: Curl/PHP 5.3.29 disallow: / User-agent: Curl/PHP 5.4.37 disallow: / User-agent: cuwhois disallow: / User-agent: cuwhois/1.0 disallow: / User-agent: dahoms disallow: / User-agent: datagnionbot disallow: / User-agent: Dispatch/0.11.0 disallow: / User-agent: DomainAppender disallow: / User-agent: DomainAppender /1.0 disallow: / User-agent: DomainSigmaCrawler disallow: / User-agent: DomainSigmaCrawler/0.1 disallow: / User-agent: Domnutch disallow: / User-agent: Domnutch-Bot disallow: / User-agent: Domnutch-Bot/Nutch disallow: / User-agent: Domnutch-Bot/Nutch-1.0 disallow: / User-agent: dotbot disallow: / User-agent: eCommerceBot disallow: / User-agent: EPiServer Link Checker disallow: / User-agent: EuripBot disallow: / User-agent: EuripBot/2.0 disallow: / User-agent: Exabot disallow: / User-agent: Findxbot disallow: / User-agent: Findxbot/1.0 disallow: / User-agent: Flamingo_SearchEngine disallow: / User-agent: Flipboard Robot disallow: / User-agent: getrequest disallow: / User-agent: getrequest/1.0 disallow: / User-agent: GigablastOpenSource disallow: / User-agent: GigablastOpenSource/1.0 disallow: / User-agent: Gluten Free Crawler disallow: / User-agent: Gluten Free Crawler/1.0 disallow: / User-agent: Go 1.1 package http disallow: / User-agent: Haiula disallow: / User-agent: Haiula/1.4 disallow: / User-agent: HaosouSpider disallow: / User-agent: heritrix disallow: / User-agent: heritrix/1.14.4 disallow: / User-agent: heritrix/3.1.1 disallow: / User-agent: Hivemind disallow: / User-agent: HostHarvest disallow: / User-agent: HostHarvest/0.4.28 disallow: / User-agent: http://git.io/tl_S2w disallow: / User-agent: http://www.checkprivacy.or.kr:6600/RS/PRIVACY_ENFAQ.jsp disallow: / User-agent: httpunit/1.x disallow: / User-agent: HubSpot disallow: / User-agent: HubSpot Links Crawler 1.0 disallow: / User-agent: HubSpot Webcrawler disallow: / User-agent: hunchan disallow: / User-agent: HyperCrawl disallow: / User-agent: HyperCrawl/0.2 disallow: / User-agent: Ichiro Robot disallow: / User-agent: image.coccoc/1.0 disallow: / User-agent: Indy Library disallow: / User-agent: integrity/5 disallow: / User-agent: InterNaetBoten disallow: / User-agent: InterNaetBoten/0.99 disallow: / User-agent: jakarta disallow: / User-agent: Jakarta Commons disallow: / User-agent: Jakarta Commons-HttpClient disallow: / User-agent: Jakarta Commons-HttpClient/3.0.1 disallow: / User-agent: Jakarta Commons-HttpClient/3.0-rcx disallow: / User-agent: Jakarta Commons-HttpClient/3.1 disallow: / User-agent: JamesBOT disallow: / User-agent: James BOT disallow: / User-agent: Java/1.4.1_04 disallow: / User-agent: Java/1.6.0_04 disallow: / User-agent: Java/1.6.0_10-rc2 disallow: / User-agent: Java/1.6.0_15 disallow: / User-agent: Java/1.6.0_20 disallow: / User-agent: Java/1.6.0_24 disallow: / User-agent: Java/1.6.0_26 disallow: / User-agent: Java/1.6.0_32 disallow: / User-agent: Java/1.7.0_09 disallow: / User-agent: Java/1.7.0_45 disallow: / User-agent: Java/1.7.0_55 disallow: / User-agent: Java/1.7.0_60 disallow: / User-agent: Java/1.7.0_60-ea disallow: / User-agent: Java/1.7.0_65 disallow: / User-agent: Java/1.7.0_67 disallow: / User-agent: Java/1.7.0_71 disallow: / User-agent: Java/1.7.0_80 disallow: / User-agent: Java/1.8.0_20 disallow: / User-agent: Java/1.8.0_25 disallow: / User-agent: Java/1.8.0_31 disallow: / User-agent: Java/1.8.0_40 disallow: / User-agent: Java/1.8.0_45 disallow: / User-agent: JetBrains disallow: / User-agent: JetBrains 5.0 disallow: / User-agent: Kraken disallow: / User-agent: Kraken/0.1 disallow: / User-agent: Kraken/Nutch-2.2.1 disallow: / User-agent: larbin disallow: / User-agent: lechenie disallow: / User-agent: libcurl disallow: / User-agent: libcurl/7.15.3 disallow: / User-agent: libcurl/7.19.7 disallow: / User-agent: libcurl/7.21.0 disallow: / User-agent: LibreOffice disallow: / User-agent: Link/1.0 disallow: / User-agent: linkCheck disallow: / User-agent: linkchecker disallow: / User-agent: link checker disallow: / User-agent: linkCheckV3.0 disallow: / User-agent: Linkdex disallow: / User-agent: LinkdexBot disallow: / User-agent: linkdexbot/2.0 disallow: / User-agent: linkdexbot/2.1 disallow: / User-agent: linkdex.com/v2.0 disallow: / User-agent: linkdex.com/v2.1 disallow: / User-agent: linkdexbot-mobile/2.1 disallow: / User-agent: LinkpadBot disallow: / User-agent: LinkpadBot/1.06 disallow: / User-agent: Lipperhey disallow: / User-agent: Lipperhey SEO Service disallow: / User-agent: Lipperhey-Kaus-Australis disallow: / User-agent: Lipperhey-Kaus-Australis/5.0 disallow: / User-agent: listicka disallow: / User-agent: LSSRocketCrawler disallow: / User-agent: LSSRocketCrawler/1.0 disallow: / User-agent: LSSRocketCrawler/1.0 LightspeedSystems disallow: / User-agent: ltx71 disallow: / User-agent: LWNutch/Nutch-1.4 disallow: / User-agent: magpie-crawler disallow: / User-agent: Mail.RU disallow: / User-agent: Mail.RU_Bot disallow: / User-agent: Mail.RU_Bot/2.0 disallow: / User-agent: Mail.RU_Bot/Fast/2.0 disallow: / User-agent: MaxPointCrawler disallow: / User-agent: MaxPointCrawler/Nutch-1.1 disallow: / User-agent: md5sum disallow: / User-agent: md5sum\x22 disallow: / User-agent: meanpathbot disallow: / User-agent: MegaIndex.ru disallow: / User-agent: MegaIndex.ru/2.0 disallow: / User-agent: mezhpozvonochnoi disallow: / User-agent: MixBot disallow: / User-agent: MJ12bot disallow: / User-agent: mycrowl/Nutch-1.9 disallow: / User-agent: MyIPTest disallow: / User-agent: My Nutch Spider/Nutch-1.9 disallow: / User-agent: NerdyBot disallow: / User-agent: netseer disallow: / User-agent: Netvibes disallow: / User-agent: NING/1.0 disallow: / User-agent: node.io disallow: / User-agent: node.js disallow: / User-agent: nutch-1.4 disallow: / User-agent: nutch-1.4/Nutch-1.4 disallow: / User-agent: oBot/2.3.1 disallow: / User-agent: omgilibot disallow: / User-agent: omgilibot/0.4 disallow: / User-agent: OpenHoseBot disallow: / User-agent: OpenHoseBot/2.1 disallow: / User-agent: Openstat disallow: / User-agent: Openstat/0.1 disallow: / User-agent: OptimizationCrawler disallow: / User-agent: OptimizationCrawler/0.2 disallow: / User-agent: PageAnalyzer disallow: / User-agent: PageAnalyzer/1.1 disallow: / User-agent: Page Analyzer disallow: / User-agent: Page Analyzer v4.0 disallow: / User-agent: PagesInventory disallow: / User-agent: Pagespeedbot disallow: / User-agent: Pagespeed/1.1 disallow: / User-agent: Pagespeed/1.1 Fetcher disallow: / User-agent: PathDefender disallow: / User-agent: PathDefender/1.0 disallow: / User-agent: Perl LWP disallow: / User-agent: PHP/5.2 disallow: / User-agent: PHP/5.3.28 disallow: / User-agent: PHPCrawl disallow: / User-agent: Plukkie disallow: / User-agent: Plukkie/1.5 disallow: / User-agent: POGS/2.0 disallow: / User-agent: PowerPivot disallow: / User-agent: PRIVACY_ENFAQ.jsp disallow: / User-agent: Prlog disallow: / User-agent: Prlog/1.0 disallow: / User-agent: psbot disallow: / User-agent: publiclibraryarchive.org disallow: / User-agent: publiclibraryarchive.org/1.0 disallow: / User-agent: pycurl disallow: / User-agent: PycURL/7.24.0 disallow: / User-agent: Python-urllib disallow: / User-agent: Python-urllib/1.17 disallow: / User-agent: Python-urllib/2.5 disallow: / User-agent: Python-urllib/2.6 disallow: / User-agent: Python-urllib/2.7 disallow: / User-agent: qingdao disallow: / User-agent: R6_FeedFetcher disallow: / User-agent: R6_CommentReader disallow: / User-agent: Riddler disallow: / User-agent: rogerbot disallow: / User-agent: rogerbot/1.0 disallow: / User-agent: Rome Client disallow: / User-agent: rootlink disallow: / User-agent: RU_Bot/2.0 disallow: / User-agent: Ruby disallow: / User-agent: Scrapy disallow: / User-agent: Scrapy/0.24.4 disallow: / User-agent: Scrapy/0.24.6 disallow: / User-agent: SemrushBot disallow: / User-agent: SemrushBot-SA disallow: / User-agent: SEOkicks disallow: / User-agent: SEOkicks-Robot disallow: / User-agent: SeznamBot disallow: / User-agent: ShowyouBot disallow: / User-agent: SheerBoredom.Experimental.Robot disallow: / User-agent: SheerBoredom.Experimental.Robot/0.2 disallow: / User-agent: sistrix disallow: / User-agent: SISTRIX Crawler disallow: / User-agent: SiteExplorer disallow: / User-agent: SiteExplorer/1.0b disallow: / User-agent: SkimBot disallow: / User-agent: SkimBot/1.0 disallow: / User-agent: SMTBot disallow: / User-agent: SMTBot/1.0 disallow: / User-agent: Sogou Pic Spider disallow: / User-agent: Sogou Pic Spider/3.0 disallow: / User-agent: Sogou spider disallow: / User-agent: Sogou web spider disallow: / User-agent: SpamBayes disallow: / User-agent: SpamBayes/1.1a3+ disallow: / User-agent: spbot disallow: / User-agent: spbot/4.4.2 disallow: / User-agent: Spiderbot/Nutch-1.7 disallow: / User-agent: spray-can disallow: / User-agent: spray-can/1.2.1 disallow: / User-agent: SSG/3.0 disallow: / User-agent: Statastico disallow: / User-agent: Statastico/4.0 disallow: / User-agent: Steeler disallow: / User-agent: Steeler/3.5 disallow: / User-agent: StudioFACA disallow: / User-agent: StudioFACA Search disallow: / User-agent: sukibot disallow: / User-agent: sukibot_heritrix disallow: / User-agent: sukibot_heritrix/3.1.1 disallow: / User-agent: SurveyBot disallow: / User-agent: Synapse disallow: / User-agent: Synthesio Crawler release MonaLisa disallow: / User-Agent: trendictionbot disallow: / User-Agent: TrueBot disallow: / User-Agent: TrueBot/1.0 disallow: / User-Agent: Typhoeus disallow: / User-Agent: uipbot/1.0 disallow: / User-Agent: uMBot-LN disallow: / User-Agent: uMBot-LN/1.0 disallow: / User-Agent: updown_tester disallow: / User-Agent: URLChecker disallow: / User-agent: WASALive disallow: / User-agent: WASALive-Bot disallow: / User-agent: WeAreNotEvil disallow: / User-agent: WebCookies disallow: / User-agent: WebCookies/1.0 disallow: / User-agent: WebCopier vx.xa disallow: / User-agent: Web corpus crawler disallow: / User-agent: WebQL disallow: / User-agent: Web-sniffer disallow: / User-agent: Web-sniffer/1.1.0 disallow: / User-agent: Webster Pro V3.4 disallow: / User-agent: WeCrawlForThePeace disallow: / User-agent: Wegtam disallow: / User-agent: Wegtam Crawler disallow: / User-agent: WeLikeLinks disallow: / User-agent: WhatWeb disallow: / User-agent: WhatWeb/0.4.8-dev disallow: / User-agent: woobot disallow: / User-agent: woobot/1.1 disallow: / User-agent: woobot/2.0 disallow: / User-agent: WorldBrewBot disallow: / User-agent: WorldBrewBot/2.1 disallow: / User-agent: worldwebheritage.org disallow: / User-agent: worldwebheritage.org/1.0 disallow: / User-agent: www.deadlinkchecker.com disallow: / User-agent: www.petitsage.fr site detector 0.4 disallow: / User-agent: V1.0/1.2 disallow: / User-agent: vBSEO disallow: / User-agent: VeriCiteCrawler disallow: / User-agent: VeriCiteCrawler/Nutch-1.9 disallow: / User-agent: Visited by http://tools.geek-tools.org disallow: / User-agent: Voila Robot disallow: / User-agent: voltron disallow: / User-agent: Vorboss Web Crawler disallow: / User-agent: Vorboss Web Crawler/Nutch-2.3 disallow: / User-agent: Xenu Link Sleuth disallow: / User-agent: Xenu's Link Sleuth disallow: / User-agent: XoviBot disallow: / User-agent: XoviBot/2.0 disallow: / User-agent: Yandex disallow: / User-agent: Yandex Robot disallow: / User-agent: Yetibot disallow: / User-agent: YisouSpider disallow: / User-agent: yoozBot disallow: / User-agent: yoozBot-2.2 disallow: / User-agent: zzabmbot disallow: / User-agent: zzabmbot/1.0 disallow: / User-agent: Spiderbot/Nutch-1.7 disallow: / User-agent: GeedoBot disallow: / User-agent: GPTBot disallow: /