# robots.txt for http://www.carphunter.net/ # Zugriff auf Dateien erlauben User-agent: Googlebot User-agent: Abacho User-agent: Sharelook User-agent: Witch User-agent: Allesklar User-agent: Lycos_Spider_(T-Rex) User-agent: KIT-Fireball User-agent: MetaGer-LinkChecker User-agent: ArchitextSpider # Excite User-agent: Scooter # Altavista User-agent: Slurp # Yahoo Disallow: /downloads/ Disallow: /suche/ Disallow: /login/ Disallow: /cgi-data/ Disallow: /cgi-bin/ Disallow: /images/ Disallow: User-agent: EmailCollector Disallow: / User-agent: * Disallow: /