AI Web Readiness
Free · No sign-up required

Is your site ready for
AI agents?

AI agents don't browse, they parse. Score any website across 20 machine-readability signals and see exactly what AI systems can and can't read.

https://
example.com74 / 100
● Good12 of 20 checks passed
✓
AI agent manifest10/10
✓
Structured data9/9
✕
Full-content AI feed0/3
✕
Q&A schema0/2
✓
Attribution schema5/5
Sample report · real results in ~5 seconds
Community data

How the web scores

Aggregated across every site scanned. Updated in real time.

Unique sites
-
distinct domains scanned
Total scans run
-
including rescans
Average score
- / 100
across all scanned sites
Avg rescan improvement
- pts
avg score gain on rescan
The 20 checks

Every signal that matters to AI agents

Each check maps to something an AI agent looks for when it reads, classifies, or cites your site. Pass them all and you're invisible to nothing.

01AI agent manifest+10 pts
What we assess. We request /llms.txt from the root of your site and confirm that it is a real text file with meaningful content, not an empty file and not your homepage or an error page returned for a missing path.
Why it matters. An llms.txt file is a short, plain-text briefing written for AI agents. It states what your site is, what you offer and which pages matter most, so an agent can understand you in seconds instead of crawling dozens of pages and guessing. Without it, an assistant has to infer your purpose from whatever it happens to find, and it may describe you inaccurately or skip you.
02llms.txt quality+6 pts
What we assess. We read the file itself and look for the structure the llmstxt.org format asks for: a title line, a one-sentence summary, grouped sections of links, links that stay on your own domain, and links that actually load. We test a sample of the links to see whether they work.
Why it matters. Agents rely on that structure to decide where to go next. A file that is just a paragraph of prose, or one full of broken or off-site links, gives them little to act on and can send them to dead ends. A well-formed file works like a trustworthy table of contents: it tells an assistant which pages answer which questions, which makes it more likely your best pages are the ones that get read and cited.
03Full-content AI feed+3 pts
What we assess. We check whether /llms-full.txt exists and returns real text content rather than an error or a substitute page.
Why it matters. Where llms.txt is a map, llms-full.txt is the whole territory: the full text of your key pages in one place. An agent with limited time or a limited context window can read your content in a single request, with no page-by-page crawling and no risk of missing something behind navigation or scripts. It is a small file to publish, and it lets AI tools represent your content accurately and completely.
04Indexability+10 pts
What we assess. We check three things that can quietly keep a page out of search results: a noindex directive in the robots meta tag, a noindex or similar value in the X-Robots-Tag HTTP header, and robots.txt rules that shut search crawlers out of the homepage.
Why it matters. These are the most damaging problems a site can have, because they work silently. A leftover noindex from a staging site, or a blanket Disallow in robots.txt, removes your pages from search engines, and the AI assistants that rely on search results cannot find or cite what is not indexed. A site can score well everywhere else and still be invisible if this check fails, which is why it carries the highest weight.
05Structured data+9 pts
What we assess. We read the JSON-LD blocks in your page source, including nested and @graph structures, and check which entity types they declare (such as Organization, WebSite, Article, Product, LocalBusiness or FAQPage). Schema found on any sampled page counts toward your score.
Why it matters. JSON-LD states plainly, in a format machines read without guessing, who you are, what you offer and how your pages relate to each other. Search engines and AI systems use it to build their understanding of an entity and to decide what to show or quote. Without it, they must infer meaning from layout and wording, and the inference is often wrong or shallow. Richer, more specific types give agents more facts they can use with confidence.
06Attribution schema+5 pts
What we assess. We look in your structured data for an Organization or Person entity with a name and a url, supporting identity signals such as a logo or sameAs links to your official profiles, and author or publisher properties on your content.
Why it matters. When an AI assistant quotes or recommends a source, it needs to know who stands behind the content. Clear attribution data lets it credit you by name, connect your site to your profiles and reputation, and trust that the content comes from a real, identifiable author or organization. Anonymous content is easier to ignore, and harder to cite with confidence.
07Readable content ratio+7 pts
What we assess. We strip out scripts, styles and hidden elements, then measure how many characters of readable text remain and what share of the page's total HTML they make up. We run this on your homepage and a sample of inner pages, and the best result counts.
Why it matters. AI crawlers read pages under strict size and time limits. A page that is mostly code, trackers and layout markup gives them very little real content for the effort and tends to be deprioritised or truncated. A page with a healthy amount of readable text is quick to process and easy to summarise, which makes it more likely to be understood correctly and used as a source.
08Crawler-visible content+8 pts
What we assess. We fetch your pages the way a crawler does, with no JavaScript execution, and check whether the meaningful content is already present in the HTML: the amount of text, a main H1 heading, subheadings and substantial paragraphs. We run this on your homepage and a sample of inner pages, and the best result counts.
Why it matters. Many AI crawlers do not run JavaScript. If your site builds its content in the browser, those crawlers receive an almost empty page and conclude that there is nothing here, even though a human visitor sees a full site. Delivering content in the initial HTML, through server rendering or static generation, is what makes it visible to the widest range of search engines and AI tools.
09AI crawler permissions+8 pts
What we assess. We read your robots.txt and work out, for each of 14 well-known AI agents (such as GPTBot, ClaudeBot, PerplexityBot and Google-Extended), whether that agent is allowed to crawl your site. A site that is open to everyone, including through a wildcard User-agent: * with Allow: /, is treated as open to all of them. We measure access, not whether you name each agent.
Why it matters. Robots.txt is the first thing a well-behaved crawler reads, and a block there means the agent never sees your content at all. Some sites block AI agents on purpose, which is a legitimate choice, but many do it by accident through an overly broad rule or a security plugin. This check shows the real consequence: every agent that is shut out cannot read, learn from or cite your site.
10Page index+6 pts
What we assess. We look for a valid XML sitemap at /sitemap.xml, or one declared in your robots.txt, and check that it parses correctly, lists a sensible number of URLs, includes last-modified dates and is announced in robots.txt.
Why it matters. A sitemap is the list of pages you want found. It lets crawlers discover content that internal links might not surface and tells them which pages have changed recently, so updates are picked up faster. Without one, discovery depends on crawlers following links, and pages that are deep in your site, new or poorly linked can be missed for a long time.
11Social and citation metadata+4 pts
What we assess. We read the OpenGraph meta tags in your page head and check for the four that matter most: og:title, og:description, og:image and og:site_name. We run this on your homepage and a sample of inner pages, and the best result counts.
Why it matters. OpenGraph tags are the headline, summary and picture that others see when your page is shared, previewed or cited. AI assistants and link-preview systems use them to describe your page quickly and consistently. When they are missing, the tool has to improvise from the page body, and the result is often an awkward title, a wrong summary or no image at all.
12Page identity signals+7 pts
What we assess. We read each page's title tag and meta description and score their presence and length. Titles in the range of roughly 40 to 70 characters and descriptions of roughly 50 to 160 characters score full marks, and values that are missing, too thin or long enough to be cut off lose points. We run this on your homepage and a sample of inner pages, and the best result counts.
Why it matters. The title and description are the first things a search engine or an AI agent reads to decide what a page is about and whether it is relevant to a question. Missing or vague ones make the page harder to classify, and overlong ones get truncated in search results and citations. A clear, specific title and description improve how often your page is chosen and how accurately it is described.
13Canonical authority+4 pts
What we assess. We read the canonical link tag on your pages and check that it exists, is an absolute URL, points at your own domain, and leads to a page that actually loads rather than at a different, old or dead address. We run this on your homepage and a sample of inner pages, and the best result counts.
Why it matters. The same page is often reachable at several addresses, such as with and without www, with tracking parameters, or through old links. The canonical tag tells search engines and agents which address is the real one so that your authority is gathered in one place. A missing tag splits that authority, and a wrong one can tell search engines to drop your pages in favour of an address that does not exist.
14Redirect and host health+4 pts
What we assess. We follow the redirects from the different ways a visitor can reach your site: plain http, with www and without www. We check that http leads to https, that only one hostname actually serves the site, and that no entry point needs a long chain of hops to arrive.
Why it matters. Every extra version of your address splits your search presence and wastes crawler time. When both www and non-www serve the site, or http does not redirect to https, search engines can treat them as separate sites and share your authority between them. Long redirect chains slow crawlers down and some give up before the end. One clean permanent redirect to a single main address keeps everything together.
15Accessibility and speed+3 pts
What we assess. We confirm that your site is served over HTTPS and measure how long your homepage takes to download from our server. A download of about 1.5 seconds or faster earns full marks, and slower responses lose points progressively.
Why it matters. Browsers, search engines and AI crawlers all treat HTTPS as a baseline for trust, and plain HTTP sites are often skipped or flagged. Speed matters because crawlers work to a time budget: a slow site gets fewer pages read per visit, and a response that is too slow may be abandoned entirely. Fast, secure delivery means your content is reliably fetched and fully read.
16Semantic structure+6 pts
What we assess. We inspect the structure of your HTML: whether there is exactly one H1, whether heading levels follow in order without skipping, whether the main content sits inside a main landmark, whether the page declares its language, whether images have alt text and whether a mobile viewport tag is present. We run this on your homepage and a sample of inner pages, and the best result counts.
Why it matters. Agents cannot see your design. They read the structure of the markup to work out what the page is about, which part is the main content and how it is organised. Clean headings and landmarks let them extract the right text and skip navigation and footers. A page built from anonymous containers is much harder to interpret, so its content is more likely to be misread or left out.
17Q&A schemabonus+2 pts
What we assess. Optional bonus. We look for FAQPage, QAPage or HowTo structured data on your sampled pages, including the questions and answers inside it.
Why it matters. Assistants answer people's questions, and question-and-answer markup is already in the shape they need: a clear question with a clear answer. Pages that provide it hand agents ready-made, quotable answers with your name attached, and they tend to be cited more often. If your site has no questions to answer or steps to explain, you can safely skip this.
18Content feedbonus+1 pts
What we assess. Optional bonus. We check whether your site advertises an RSS or Atom feed through a link tag in the page head, or serves one at a common location such as /feed.xml or /rss.xml. A feed that is advertised scores higher than one that has to be found by guessing.
Why it matters. A feed is a machine-readable stream of your latest content. It tells agents and aggregators that you publish regularly and gives them a simple, reliable way to see what is new without crawling the whole site. If you publish articles, news or updates, it is one of the quickest ways to get new content noticed. A site that rarely changes can skip it.
19Agent discovery endpointsbonus+2 pts
What we assess. Optional bonus. We look for machine-readable discovery documents at their well-known locations: an MCP server card, an A2A agent card, an API catalog, an agents.json file, an AI plugin manifest or an OpenAPI description. One found earns most of the credit and two or more earn full marks.
Why it matters. A growing share of AI traffic comes from agents that act on a service, not only read it, for example by calling an API or starting a task for a user. These files are how such agents discover what your service can do and how to use it. If you run an API or an agent, publishing them makes you usable by other AI systems. A purely informational site does not need them.
20Content freshnessbonus+2 pts
What we assess. Optional bonus. We look for machine-readable dates on your pages: datePublished and dateModified in structured data, article date meta tags, time elements with a datetime value, and lastmod dates in your sitemap. We also note how recent the newest date is. Server headers are not used for this check.
Why it matters. AI systems prefer to quote information that is current, and they cannot tell the age of a page unless you say so. Honest, machine-readable dates let an agent judge whether your content is up to date and safe to cite, and they help search engines know when to come back. Stale or undated pages are easier to skip in favour of sources that show they are maintained.
How it works

Three steps to a shareable report

No account needed. Enter a URL, wait a few seconds, and get a full breakdown with a ready-to-deploy /llms.txt file.

01

Fetch

We request your homepage, /robots.txt, /sitemap.xml, /llms.txt, /llms-full.txt, agent discovery files, and up to three inner pages. Requests run in parallel so results come back fast.

02

Analyze

20 checks run against the raw HTML and file responses with no JavaScript execution. This mirrors exactly how AI crawlers like GPTBot and ClaudeBot see your site, giving you an accurate picture of your real AI visibility.

03

Report

You get a score out of 100, a grade, and a full per-check breakdown with specific fixes for each failure. Every report has a unique shareable URL so you can send it to your team or track changes over time.

FAQ

Common questions

What is AI web readiness?

AI web readiness measures how well a website can be discovered, parsed, and cited by AI agents and large language models. It covers signals like /llms.txt, structured data, robots.txt AI crawler permissions, server-side rendering, and OpenGraph metadata. A site with high AI readiness is more likely to be understood, referenced, and surfaced by AI-powered tools and assistants.

What is /llms.txt and why does it matter?

/llms.txt is a plain text file placed at the root of a website that tells AI agents what the site is about, what it offers, and which pages to prioritize. It follows a simple convention proposed by the developer community to give large language models a structured map of site content. Without it, AI agents have to infer your site's structure from raw HTML, which is less reliable. Adding /llms.txt is the single highest-impact change most sites can make for AI visibility.

How is the AI readiness score calculated?

The score is calculated across 20 weighted checks, each representing a signal that AI agents use when reading, classifying, or citing a website. Each check is a pass, partial, or fail, and the points add up to a score out of 100. The checks cover indexability (10 pts), /llms.txt (10 pts), structured data (9 pts), robots.txt AI permissions (8 pts), and 12 further core signals that together make up exactly 100 points. Four optional bonus checks (Q&A schema, RSS feed, agent discovery files and content freshness) are reported separately as up to 7 bonus points. They never change the 0-100 score, so a site that is not an API or a publisher can still score a perfect 100. Page-level checks also look at a sample of inner pages, which can raise your score but never lower it.

Is AI readiness the same as SEO?

They overlap but are distinct. Traditional SEO optimizes for search engine crawlers and ranking algorithms. AI readiness optimizes for large language models and AI agents that read, summarize, and cite content directly. Many SEO best practices such as structured data, fast load times, and clear content also help AI readiness, but AI-specific signals like /llms.txt have no SEO equivalent. A site can rank well in Google while being nearly invisible to AI systems, and vice versa. The most future-proof approach is to optimize for both.

Which AI crawlers does the tool check for?

The tool reads robots.txt and checks access for GPTBot (OpenAI), ClaudeBot and anthropic-ai (Anthropic), PerplexityBot (Perplexity), Google-Extended and GoogleOther (Google), CCBot (Common Crawl), cohere-ai (Cohere), OAI-SearchBot (OpenAI search), and Bytespider (ByteDance). The tool reads both Allow and Disallow rules. A site that is open to them, even through a general User-agent: * with Allow: / rule, scores full marks, and each crawler you block costs points because it cannot read the site.

Is this tool free to use?

Yes, AI Web Readiness is completely free to use. No account or sign-up is required. Simply enter a URL and get a full report in seconds. All scans are saved and shareable via a unique report link, and you can rescan any URL at any time to track improvements over time. The tool runs 20 checks and returns results in under 10 seconds for most sites.

How often should I rescan my site?

Rescan whenever you make changes that could affect machine readability: publishing a new /llms.txt, adding structured data, updating robots.txt, or significantly changing your page content. The tool stores your previous score so you can track improvement over time. Many users run a scan before and after a launch to confirm their changes had the intended effect on AI visibility. A good habit is a monthly scan even without changes, since third-party scripts and CMS updates can quietly reduce your content ratio or break structured data.

What does a Poor or Fair score mean for my site?

A Poor score (below 40) means most AI systems will struggle to parse, classify, or cite your content reliably. A Fair score (40 to 64) means the basics are in place but key signals are missing. In both cases, the check breakdown on your report shows exactly which fixes will move the needle most. The highest-weight checks, indexability and /llms.txt at 10 points each and structured data at 9 points, are usually the fastest path to a meaningful score improvement.