AI visibility checker for ChatGPT, Claude and Perplexity
Check whether AI search tools and assistants can reach your page, which bots your robots.txt blocks, and what text they actually read when they do.
Bots checked
| robots.txt token | Owner | Used for | Type |
|---|---|---|---|
OAI-SearchBot | OpenAI | ChatGPT search | Search |
ChatGPT-User | OpenAI | ChatGPT browsing | On request |
GPTBot | OpenAI | Model training | Training |
Claude-SearchBot | Anthropic | Claude search | Search |
Claude-User | Anthropic | Claude web fetch | On request |
ClaudeBot | Anthropic | Model training | Training |
PerplexityBot | Perplexity | Perplexity search | Search |
Perplexity-User | Perplexity | Perplexity live fetch | On request |
Google-Extended | Gemini training and grounding | Training | |
Applebot-Extended | Apple | Apple Intelligence training | Training |
meta-externalagent | Meta | Meta AI training | Training |
CCBot | Common Crawl | Open web crawl used by many models | Training |
What the check covers
- robots.txt rules for each AI crawler, with the exact rule that allows or blocks it.
- A live request with each bot’s User-Agent, to catch servers and CDNs that block AI bots.
- noindex and other robots directives in meta tags and the X-Robots-Tag header.
- The text in your raw HTML. AI crawlers do not run JavaScript, so a page rendered in the browser can look empty to them.
- Headings, meta description and JSON-LD structured data.
- Whether /llms.txt exists and follows the proposed format.
Search, on-request and training bots
AI companies run separate bots for separate jobs. Search crawlers index pages so an assistant can find and cite them. On-request fetchers read a page live when a person asks the assistant to. Training crawlers collect pages for future models.
Blocking a training crawler keeps your content out of model training, and does not remove you from AI answers. Blocking a search crawler does remove you from that assistant’s answers. The score counts search and on-request access and ignores training crawlers, which are a publisher’s choice.
Common problems
- The page is built by JavaScript: AI crawlers see an empty shell. Render or pre-render the content on the server.
- A CDN bot setting blocks AI crawlers even though robots.txt allows them. On Cloudflare, check Security > Bots and AI Crawl Control.
- A broad "Disallow: /" in the User-agent: * group blocks every AI bot that has no group of its own.
- robots.txt returns a server error. Crawlers then treat the whole site as off limits.
- A noindex directive left over from staging.
What it does not do
It does not predict what a model will say about your page. Answers change between runs and models, so they cannot be tested reliably. The check covers what is deterministic: whether each bot can reach the page, and what it reads.
Live requests come from Cloudflare’s network with each bot’s User-Agent. A site that verifies bots by IP address may treat the real bot differently.