Google Adwords services provides a wide range of advertisment options. Currently Google is using a spider specially for their Google Adwords program that automatically crawls and analyzes the content of advertising that is displayed on the Google Network (other websites). The name of this spider is AdsBot-Google. If you see such an entry in your web stats or logs then you know, your website or pages has been visited by this bot.

Why does Google’s AdsBot visit pages where advertisement is placed?

Google uses AdsBot to determine the quality of the pages where advertisement is displayed. The content on these landing pages will be used for the quality score that Google assigns to your Google Adwords ads.

Google uses the quality score and the amount you are willing to pay, to determine the position of your ads. Advertisement with a high quality score can rank higher and get better positions, even if you pay less than others for the advertisement.

If you do not want that Google spiders these landing pages you can add the following robots.txt on your server:

User-agent: adsbot-google

Disallow: /

The tag line above forbid Adsbot-Google to spider and analyze your web pages that display advertisement from Google Adwords. The bad thing about this however is that your Adwords quality score will drop and you certainly have to pay for it. So you better forget to deny Adsbot-Google crawler access to your pages. Google AdWords landing pages and site quality guidelines are published on the Adwords pages of Google.