GEO Analyzer (Generative Engine Optimization)
Paste a page's HTML and its robots.txt to check whether known AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, and others) can access it, whether noai/noimageai directives are blocking indexing, and how citable the content is: the length of its opening paragraph, average paragraph length, and whether FAQ/HowTo/Article structured data is present. An optional URL-fetch mode, through a public CORS proxy and clearly labeled as such, can pull the page and its robots.txt and check for an llms.txt file automatically. Pasting content directly keeps everything local.
Reviewed by Riccardo Dal Cero, Economist & Consultant · Last updated September 27, 2026
How it works
- 1
Paste the page's HTML and its site's robots.txt content (or use the optional URL-fetch mode).
- 2
The tool checks whether each known AI crawler is allowed, blocked, or unmentioned in robots.txt.
- 3
It scans the HTML for noai/noimageai directives and FAQ/HowTo/Article structured data.
- 4
Citability signals (opening paragraph length and average paragraph length) are measured against the pattern AI systems tend to extract cleanly.
A different crawler, a different checklist
Classic technical SEO audits robots.txt for Googlebot and traditional search crawlers, but AI crawlers are often configured separately. A site can be wide open to Googlebot while quietly blocking GPTBot or ClaudeBot, sometimes on purpose and sometimes because nobody revisited robots.txt after AI crawlers showed up. This tool checks that specific gap.
Citability is a content property, not just a technical one
Even a fully crawlable page can be hard for an AI system to cite cleanly if its key point is buried in a long, meandering paragraph. Short direct openings, digestible paragraph lengths and explicit FAQ/HowTo markup tend to help a human skimmer too, not just an AI system.
Frequently asked questions
What is GEO / Generative Engine Optimization?
It's the practice of making content easier for AI systems to crawl, understand and cite, covering chatbots, AI-generated search overviews and answer engines. It's similar in spirit to traditional SEO but aimed at AI consumption rather than classic ranked search results.
Does blocking an AI crawler always hurt me?
No. Some publishers block AI crawlers on purpose, to stop their content from training AI models or showing up in AI-generated answers without sending any traffic back. This tool just reports which bots are blocked, so you can confirm that matches what you actually intended.
Is llms.txt something I need?
It's an emerging, unofficial convention some sites use to give AI systems a curated summary of their content. Google has said it doesn't use llms.txt for Search, and adoption elsewhere is patchy, so treat it as an optional, low-cost signal rather than a requirement.
How is 'citability' measured?
With structural heuristics that track how easily an AI system can lift a clean, self-contained passage: a short, direct opening paragraph instead of throat-clearing before the point, paragraph lengths that stay moderate throughout, and structured data types like FAQPage or HowTo that mark up question-answer content explicitly.
Why does the fetch mode need a proxy, and is it safe?
Browsers block cross-origin reads of another site's raw content unless that site opts in with CORS headers. A public proxy (corsproxy.io) works around that, so the URL and fetched content pass through that third-party service. That's the one case where this tool sends data beyond what you paste; it's opt-in and labeled, and the paste-based mode never leaves your browser.
Related tools
Embeddable Importer Generator
Generate a copy-paste CSV import widget for your own website.
FatturaPA Validator
Check an Italian e-invoice (FatturaPA) XML for structure, ID checksums and math errors.
EU VAT Number Checker (VIES-style)
Batch-validate EU VAT number checksums and check the OSS €10,000 threshold.