Blog · November 16, 2026 · Punchline Analytics
Is that really Googlebot? How to spot an impostor
Anyone can call their scraper "Googlebot" or "GPTBot". Here's how to check a crawler is who it says it is, with the reverse DNS and IP-range methods Google, OpenAI and others document.
Short answer: never trust the user agent alone. A real Googlebot comes from an IP address whose reverse DNS name ends in googlebot.com, google.com or googleusercontent.com, and that name resolves back to the same address; or from an address in the IP ranges Google publishes. Other crawlers (GPTBot, ClaudeBot, PerplexityBot and more) can be checked against their operators' published IP ranges. A request that claims a famous name from anywhere else is an impostor.
Why it matters
Scrapers borrow well-known crawler names because sites tend to let them in. If you allow "Googlebot" past a paywall, a rate limit or a bot filter on the strength of its user agent, you've let in anyone who types that string. And if your analytics counts impostors as Google, your crawl numbers are wrong.
Method 1: reverse DNS, then forward DNS
This is Google's documented method for checking a single request:
- Reverse lookup the requesting IP address:
host 66.249.66.1. - Check the name ends in
googlebot.com,google.comorgoogleusercontent.com. - Forward lookup that name:
host crawl-66-249-66-1.googlebot.com. - Check it resolves to the same IP you started with.
Step 3 is the one people skip, and the one that matters: anyone who controls an IP address can set its reverse DNS name to anything, including something ending in googlebot.com. Google made the same point back in 2006. Only the forward lookup proves Google controls the name.
Method 2: published IP ranges
For checking lots of requests, look the address up in the lists operators publish instead of doing DNS lookups each time:
- Google publishes its crawlers' IP ranges as JSON, now at developers.google.com/crawling/ipranges/ (the old location will redirect).
- OpenAI publishes ranges per crawler, such as
openai.com/gptbot.jsonandopenai.com/searchbot.json(docs). - Perplexity publishes JSON ranges for each of its crawlers (docs).
Refresh the lists regularly; they change. The AI agent directory links each crawler's list (also as JSON).
Method 3: signatures
Some agents now sign their requests (Web Bot Auth: an HTTP message signature checked against keys the operator publishes). A valid signature is the strongest evidence of all, because it can't be borrowed by copying a string. Where an agent signs, check that first.
What to do with impostors
- Don't give them a crawler's privileges: no paywall bypass, no relaxed rate limits.
- Count them separately. They're scrapers, often aggressive ones, and lumping them in with the real crawler hides both.
- Don't panic about them in analytics, as long as your analytics labels them. Punchline checks every request that claims a known crawler's name against its operator's published addresses (and signatures where there are any) and labels the ones that fail as impostors, with the evidence. See who's borrowing names on your site.