// Free tool

Is your robots.txt killing AI citations?

Paste your robots.txt. We'll tell you which AI engines can cite you — and which you've silently blocked. The #1 mistake: blocking GPTBot thinking it stops ChatGPT (it doesn't — that's OAI-SearchBot).

// Showing sample analysis. Paste yours to run live.

// Verdict

All citation bots allowed

Every AI engine that issues citations can reach your site.

// Citation bots

  • OAI-SearchBot
    ChatGPT Search, ChatGPT Browse citations
    Default
  • ChatGPT-User
    User-initiated URL fetches inside ChatGPT
    Default
  • PerplexityBot
    Perplexity AI live search and citations
    Default
  • Perplexity-User
    User-triggered URL fetches inside Perplexity
    Default
  • ClaudeBot
    Claude live page fetches and (historically) training data
    Default
  • Claude-SearchBot
    Claude's live web search feature
    Default
  • Googlebot
    Google Search, AI Overviews citations
    Default
  • bingbot
    Bing Search, Microsoft Copilot citations
    Default
  • FacebookBot
    Meta AI citations across WhatsApp, Instagram, Facebook
    Default

// Training-only bots

1/5 blocked
  • GPTBot
    OpenAI
    Opted out
  • anthropic-ai
    Anthropic
    Allowed
  • Google-Extended
    Google
    Allowed
  • Meta-ExternalAgent
    Meta
    Allowed
  • CCBot
    Common Crawl
    Allowed

Blocking training bots is an opt-out from model training. It does not affect citations from any AI engine.

// Recommended robots.txt

# AI citations: allow search + user-fetch bots
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Googlebot
Allow: /

User-agent: bingbot
Allow: /

User-agent: FacebookBot
Allow: /

# Training opt-out (does NOT affect citations)
User-agent: GPTBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: CCBot
Disallow: /

# Default policy
User-agent: *
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml

// Allows every citation bot. Blocks every training bot. Replace yourdomain.com with yours.