|
|||||||||||||||||||
Snipsnap Config Robots
|
# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://www.cassiel.com/space/SnipSnap/config/robots# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://www.zilverline.org/zilverlineweb/space/SnipSnap/config/robots# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://wiki.xdroop.com/space/SnipSnap/config/robots# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://www.cassiel.com/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... https://ulc-community.canoo.com/snipsnap/space/SnipSnap/config/robots.txtCreated by Administrator. Last edited by Administrator, 4 years and 175 days ago. Viewed 13,691 times. #1 http://louie.cs.nott.ac.uk/pmm/comments/SnipSnap/config/robots# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that ... http://snipsnap.org/comments/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://wiki.xdroop.com/space/SnipSnap/config/robots.txt# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://www.quartetproject.net/space/SnipSnap/config/robots# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html IGNORE FAST-WebCrawler http://fast.no/support/crawler.asp ... http://webtest-community.canoo.com/wiki/space/SnipSnap/config/robots# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://snipsnap.org/comments/SnipSnap/config/robots# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://www.quartetproject.net/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://www.kartingpt.net/racing/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://webtest-community.canoo.com/wiki/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://snippy.ceejbot.com/wiki/show/snipsnap/config/robotstxt# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://tom.conjective.ch/tomtom/space/SnipSnap/config/robots# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://www.tecweb.inf.puc-rio.br/semweb/space/SnipSnap/config/robots# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://wever.org/java/space/SnipSnap/config/robots# This file contains User-Agent mappings for robots # # WWW Search Engines Googlebot http://www.googlebot.com/bot.html FAST-WebCrawler http://fast.no/support/crawler.asp Slurp http ... http://www.loadbang.net/space/SnipSnap/config/robots# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://www.mfenner.de/wiki/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://wever.org/java/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://www.loadbang.net/space/SnipSnap/config/robots.txtUser-agent: * Crawl-Delay: 2 Disallow: /space/SnipSnap/ Disallow: /blog/exec/ Disallow: /blog/rdf/ http://www.pabrantes.net/blog/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://www.tecweb.inf.puc-rio.br/semweb/space/SnipSnap/config/robots.txt# # ROBOTS.TXT -- this file is sent to robots accessing the URL /robots.txt # # *** This only works if SnipSnap is running at the top level of the server *** # Keep in mind that robots ... http://www.bscweasel.de/snipsnap_en/space/SnipSnap/config/robots.txt |
|