Friday, April 04, 2008

Google's robots.txt generator

eMarketing System recommend... Business & Marketing Books | Business & Finance Magazine Subscriptions

I've found very interesting article from Google Webmaster Central Blog it's about Speaking the language of robots written by Sean Harding the Webmaster Tools Team on Thursday, March 27, 2008

They release and introduce you to your personal robot translator: the Robots.txt Generator in Webmaster Tools. It's designed to give you an easy and interactive way to build a robots.txt file. It can be as simple as entering the files and directories so you don't want crawled by any robots and you can create fine-grained rules for specific robots and areas of your site as well if you need.

You can test your new robots.txt file with their robots.txt analysis tool after finished it with the generator to see its effect. After you're done, just save your file to the top level (root) directory of your site, and you're good to go.

However, they informed that there are a couple of important things to keep in mind about the robots.txt files. First, not every search engine will support every extension to robots.txt files. Next, robots.txt is simply a request.

For more information just visit the article "Speaking the language of robots".

This new tool will helps you inform what you want to Googlebot and all other robots whose visit your site. In case that you want to learn more about robots.txt files, you can check out Google's Help Center. And if you'd like to discuss robots.txt and robots with other webmasters, visit their Google Webmaster Help Group.

Hope this useful.

eMarketing System Team

Tag: | | | |

No comments: