AdsBot-Google
AdsBot-Google is Google's crawler for checking landing pages and ad content for its advertising products.
Should you let it in?
Allow or block AdsBot-Google
Yes, if you advertise with Google: it checks your landing pages for its ad products.
To ask it to stay out, add this to robots.txt. Whether it follows robots.txt is Google's call; its documentation says.
User-agent: AdsBot-Google
Disallow: /Is it really?
How to tell a real AdsBot-Google from an impostor
Anyone can put “AdsBot-Google” in a user agent. What can't be faked:
- Check the address. Google publishes the IP ranges AdsBot-Google uses: developers.google.com/static/crawling/ipranges/special-crawlers.json. A request whose user agent says AdsBot-Google from an address outside them is an impostor.
- Check reverse DNS. Look up the requesting IP's host name; it should end in
googlebot.comorgoogle.com. Then look that name up and make sure it resolves back to the same IP, since anyone can set a reverse DNS name on their own addresses.
In Punchline
How Punchline counts AdsBot-Google
Punchline labels a request that names AdsBot-Google with how sure it is: verified or declared when the evidence checks out, a lower tier when it can't be checked, and impostor when the evidence contradicts it. Each visit is counted under Google, with its purpose, so you can see whether AdsBot-Google is reading your site and which pages it reads. How it works · Known agents