How do I block or allow Googlebot

[mhadaily]

How do I block or allow Googlebot?
[ltr]
Blocking Googlebot
Google uses several user-agents. You can block access to any of them by including the bot name on the User-agent line of an entry. Blocking Googlebot blocks all bots that begin with "Googlebot".

Googlebot: crawl pages from our web index and our news index
Googlebot-Mobile: crawls pages for our mobile index
Googlebot-Image: crawls pages for our image index
Mediapartners-Google: crawls pages to determine AdSense content. We only use this bot to crawl your site if AdSense ads are displayed on your site.
Adsbot-Google: crawls pages to measure AdWords landing page quality. We only use this bot if you use Google AdWords to advertise your site.
For instance, to block Googlebot entirely, you can use the following syntax:

User-agent: Googlebot

Disallow: /
Allowing Googlebot
If you want to block access to all bots other than Googlebot, you can use the following syntax:

User-agent: *

Disallow: /


User-agent: Googlebot

Disallow:
Googlebot follows the line directed at it, rather than the line directed at everyone.

The Allow extension
Googlebot recognises an extension to the robots.txt standard called Allow. This extension may not be recognised by all other search engine bots, so check with other search engines in which you are interested to find out. The Allow line works exactly like the Disallow line. Simply list a directory or page that you want to allow.

You may want to use Disallow and Allow together. For instance, to block access to all pages except one in a subdirectory, you could use the following entries:

User-agent: Googlebot

Disallow: /folder1/

Allow: /folder1/myfile.html
Those entries would block all pages inside the folder1 directory except for myfile.html.

If you block Googlebot and want to allow another of Google's bots (such as Googlebot-Mobile), you can allow access to that bot using the Allow rule. For instance:

User-agent: Googlebot

Disallow: /




User-agent: Googlebot-Mobile
[/ltr]
Allow: /