Verify AI Crawlers Are Real: GPTBot Anti-Spoofing Guide 2026

A bot shows up on your site, announces itself as GPTBot, pulls twenty pages in thirty seconds, then leaves. You let it through because your robots.txt allows OpenAI, and blocking an AI crawler means losing GEO visibility. Except spoofing a user-agent takes five seconds with curl, and nothing proves that visitor was really GPTBot. It could just as easily have been a competitor scraping your pricing, a scraper reselling your content to a third-party model, or a monitoring tool bypassing your paywall by posing as a bot you chose to let in.

This guide covers how to tell a real AI crawler from an impersonator, with the two verification methods that actually matter, a ready-to-copy script, and the decision framework for what to do once you have your answer.

The user-agent proves nothing: spoofing GPTBot takes five seconds

curl -A "GPTBot" https://yoursite.com/pricing

That command is enough to impersonate GPTBot to any server that only checks the User-Agent header. Nothing technical stops a script, a competitor, or a malicious scraper from declaring whatever user-agent it wants. That's a fundamentally different problem from classic SEO: ignoring a Disallow in your robots.txt is already a violation of the convention for a real bot, but an impersonator doesn't even need to ignore it, it just claims to be a bot you already explicitly allowed.

The real-world consequences: a competitor tracking your pricing continuously without ever showing up in your normal analytics (since it blends into the "legitimate AI traffic" you chose not to block), premium content getting scraped and republished elsewhere, or a server load spike wrongly blamed on a genuine OpenAI crawl surge when it's actually a poorly coded bot hammering your pages with no rate limiting.

Who GPTBot, ClaudeBot and PerplexityBot actually are

Before verifying a crawler, you need to know what each bot is supposed to do. AI operators generally split their training bots from their real-time citation bots, a distinction we cover in detail in our guide to blocking AI training crawlers:

The key takeaway: the bot name in the user-agent tells you nothing about its legitimacy until you've verified it through one of the two methods below.

Method 1: DNS verification in both directions

The most reliable method for one-off traffic combines a reverse DNS lookup with a forward confirmation:

1. Reverse lookup. Start from the visitor's IP address and find which hostname it resolves to.
2. Domain validation. That hostname must belong to the operator's official domain (openai.com for GPTBot, anthropic.com for ClaudeBot, googlebot.com for Googlebot).
3. Forward confirmation. Resolve that hostname again in the other direction and confirm it points back to the same original IP address.

host 20.171.207.113
# => 113.207.171.20.in-addr.arpa domain name pointer crawl-xxxx.crawl.openai.com.

host crawl-xxxx.crawl.openai.com
# => crawl-xxxx.crawl.openai.com has address 20.171.207.113

If both IP addresses match and the hostname belongs to the expected domain, the bot is genuine. The two-way confirmation isn't a nice-to-have: a reverse lookup alone can be forged by anyone who controls the reverse DNS zone for their own IP block, while the full loop (reverse then forward) is far harder to fake.

Method 2: published IP ranges

OpenAI, Anthropic and Perplexity publish lists of IP addresses (often as CIDR blocks or JSON) matching their official crawlers. Matching the source IP against these lists is faster than a DNS lookup on every request, as long as you refresh the file regularly since these ranges change over time. It's also the recommended fallback for ClaudeBot, whose reverse DNS isn't always reliable according to implementation reports observed in 2026: in that case, matching against Anthropic's published IP range becomes the primary check rather than a backup.

In practice, the most robust combination is: IP within the published range AND consistent cross-checked DNS resolution when available. A single positive signal rarely settles the question for high-stakes traffic (pricing pages, paywalled content, forms).

A script so you stop doing it by hand

Manually checking every request makes no sense once your site sees a few hundred bot hits a day. Here's the logic to automate in your middleware or reverse proxy:

#!/usr/bin/env bash
# verify-ai-bot.sh <IP> <USER_AGENT>
IP="$1"
UA="$2"

HOSTNAME=$(host "$IP" | awk '/pointer/ {print $NF}' | sed 's/.$//')

if [[ -z "$HOSTNAME" ]]; then
  echo "SPOOF_SUSPECTED: no reverse resolution for $IP (UA: $UA)"
  exit 1
fi

case "$UA" in
  *GPTBot*|*OAI-SearchBot*|*ChatGPT-User*) EXPECTED="openai.com" ;;
  *ClaudeBot*|*Claude-SearchBot*) EXPECTED="anthropic.com" ;;
  *PerplexityBot*) EXPECTED="perplexity.com" ;;
  *) echo "UNKNOWN_BOT: $UA"; exit 2 ;;
esac

if [[ "$HOSTNAME" != *"$EXPECTED" ]]; then
  echo "SPOOF_CONFIRMED: $HOSTNAME does not belong to $EXPECTED (UA: $UA)"
  exit 1
fi

FORWARD_IP=$(host "$HOSTNAME" | awk '/has address/ {print $NF}')
if [[ "$FORWARD_IP" != "$IP" ]]; then
  echo "SPOOF_CONFIRMED: forward confirmation does not match (UA: $UA)"
  exit 1
fi

echo "VERIFIED: $UA from $IP ($HOSTNAME)"

Log the result (VERIFIED, SPOOF_SUSPECTED, SPOOF_CONFIRMED) along with the IP, user-agent and requested URL. That log is what lets you answer the question that actually matters: who is really scraping your site under the cover of a known bot name.

Allow, throttle, or block: the decision framework

The allow rules in robots.txt we detail in our 2026 robots.txt guide are only a declaration of intent: they tell a well-behaved bot what to do, not what an impersonator will actually do. Technical verification upstream applies to all traffic, well-behaved or not.

What this looks like in a real audit

On a recent SeAudit audit for a mid-sized e-commerce site, thirty days of log analysis turned up 1,240 requests carrying a GPTBot user-agent. After running them through the verification script above, only 890 (72%) passed the two-way DNS check. The remaining 350 came from a single undocumented IP block, concentrated on pricing and product-comparison pages, a pattern typical of competitive monitoring rather than a genuine OpenAI crawl. That kind of gap doesn't show up in any standard analytics tool until someone actually verifies the legitimacy of the declared user-agent.

Common mistakes

• Trusting the user-agent alone. The most common flaw, and the easiest to fix.
• Verifying once and never checking again. Published IP ranges change; a script that worked six months ago may be running against a stale list today.
• Blocking every bot out of caution. You lose the legitimate citation bots too, and with them your visibility in AI answers.
• Not logging enough fields. IP, user-agent, URL and timestamp are the bare minimum for investigating a suspicious spike after the fact.

Key takeaways

A bot name in the user-agent is a claim, not proof. The only way to know whether GPTBot, ClaudeBot or PerplexityBot is genuinely visiting your site is to cross-check a two-way DNS resolution against each operator's published IP ranges, then log the result to separate real AI traffic from spoofed traffic. Curious where your own site stands on these technical points? Get your free score out of 100 in a few minutes.