Robots.txt

From Cyborg Anthropology
Revision as of 20:36, 25 June 2011 by And.awarner (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Definition

"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web.

External Links

http://www.javascriptkit.com/howto/robots.shtml