Block Index · live
Can AI see your business?
Block Index checks whether the crawlers behind AI answers — GPTBot, ClaudeBot and the rest — are allowed to read your site's robots.txt and content. Run a one-off check for free, or monitor continuously with Pro.
What it does
Capabilities
Per-crawler verdicts
Reads your robots.txt the way each named AI crawler would and tells you which ones are allowed to read your content and which are blocked.
A 14,077-domain panel for context
Your result sits against a panel of 4,077 top domains and 10,000 long-tail ones, so you can see how your choices compare to the wider web rather than in isolation.
Free, no login
The one-off check needs nothing from you but a domain. No account, no email, no trial clock.
Pro Monitor
Keep watching. Pro Monitor checks your domains on an ongoing basis so a changed robots.txt does not go unnoticed.
How it works
Three steps, no setup.
Enter a domain
Any public site. No sign-up for the one-off check.
We fetch and parse
Block Index reads the live robots.txt and evaluates it for each known AI crawler user-agent.
Read the verdict
Allowed or blocked, per crawler, alongside the panel. Upgrade to Pro Monitor to keep checking over time.
Pricing
What it costs
Free — one-off check, no login.
Pro Monitor — included in Geppetto Pro, $29 per month, or $290 per year (two months free): every tool, one account. Billed by Polar, our merchant of record, by card. Cancel any time; access runs to the end of the period you paid for.
Questions
FAQ
Which crawlers does it check?
The named AI crawlers that fetch content for AI products — GPTBot, ClaudeBot and others. The list checked is shown on each result.
Does it change anything on my site?
No. It only reads what is publicly served. Changing your robots.txt is up to you.
Do I need an account for the free check?
No. Pro Monitor needs a sign-in so we can keep results and re-check on your behalf.
Is a robots.txt verdict a guarantee that a crawler behaves?
No. robots.txt is a request, not an enforcement mechanism. Block Index tells you what you have asked crawlers to do; it cannot promise they comply. See the monitoring terms.
Governing documents: Terms of Service · Monitoring terms · Refund policy.