A robots.txt file is a little text file that lives in your site's root folder. It tells the search engine which part of the website to crawl and index...