Robots.txt is a plain text file that Search Engines use
to see if there are areas within your Website which should
not be indexed.
This file must be placed in your 'www' directory in order
for a search engine to see it.
The sample below tells search engines not to index pages
in the specified folders.
---- robots.txt (contents follow)
|