What is AI web readiness?

AI web readiness measures how well a website can be discovered, parsed, and cited by AI agents and large language models. It covers signals like /llms.txt, structured data, robots.txt AI crawler permissions, server-side rendering, and OpenGraph metadata.

What is /llms.txt and why does it matter?

/llms.txt is a plain text file placed at the root of a website that tells AI agents what the site is about, what it offers, and which pages to prioritize. Without it, AI agents have to guess your site's structure and purpose. It is the most impactful single change you can make for AI visibility.

How is the AI readiness score calculated?

The score is calculated across 20 weighted checks, each representing a signal that AI agents use when reading, classifying, or citing a website. Each check is a pass, partial, or fail, and the points add up to a score out of 100. The checks cover indexability (10 pts), /llms.txt (10 pts), structured data (9 pts), robots.txt AI permissions (8 pts), and further signals totalling 100 core points, plus four optional bonus checks.

Is AI readiness the same as SEO?

They overlap but are distinct. Traditional SEO optimizes for search engine crawlers and ranking algorithms. AI readiness optimizes for large language models and AI agents that read, summarize, and cite content directly. Many SEO best practices (structured data, fast load times, clear content) also help AI readiness, but AI-specific signals like /llms.txt have no SEO equivalent.

Which AI crawlers does the tool check for?

The tool checks for explicit permissions in robots.txt for GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended, anthropic-ai, cohere-ai, and meta-externalagent, among others.

Is this tool free to use?

Yes, AI Web Readiness is completely free to use. No account or sign-up is required. Simply enter a URL and get a full report in seconds.

credence.today - 14/100 Poor - AI Web Readiness

expand

Grade

Poor

Most AI systems struggle to read this site. Start with the highest-weight failures below.

Score

+0 / 7 bonus

Passed

Partial

Failed

Not sure where to start?

Get a free 15-minute session with an AI visibility expert. We'll walk through your results and prioritise the fixes that move the needle fastest.

Book a free call →

No obligation · 15 min · Free

Check breakdown1 passed · 5 partial · 14 failed

AI agent manifest

No /llms.txt found (status 404).

0 / 10

⌄

llms.txt quality

There is no usable /llms.txt to evaluate. Add one first.

0 / 6

⌄

Full-content AI feed

/llms-full.txt returned HTTP 200, but the response body is an HTML page, not the expected file. This site answers 200 for URLs that do not exist, so a 200 alone does not prove the file is there.

0 / 3

⌄

Indexability

The homepage is marked noindex in the robots meta tag, which tells search engines and AI search tools to leave it out.

0 / 10

⌄

Structured data

No JSON-LD structured data found on the homepage.

0 / 9

⌄

Attribution schema

No Organization or Person entity, and no author or publisher property, found in JSON-LD structured data.

0 / 5

⌄

Readable content ratio

78 chars of text, 2.6% text-to-HTML ratio. Improving content density would raise this score.

1.5 / 7

⌄

Crawler-visible content

78 chars server-rendered, 1 H1, 0 H2, 1 substantial paragraph(s). Add more structured content to improve this score.

2.5 / 8

⌄

AI crawler permissions

Blocks 14 of 14 tracked AI agents: gptbot, claudebot, claude-web, anthropic-ai, perplexitybot.... Blocking is a legitimate choice, but a blocked agent cannot read the site, so it earns no credit.

0 / 8

⌄

Page index

No /sitemap.xml (status 401).

0 / 6

⌄

Social and citation metadata

No OpenGraph tags found.

0 / 4

⌄

Page identity signals

title: 11 chars, description: missing. Titles of 40-70 and descriptions of 50-160 characters score full marks.

2 / 7

⌄

Canonical authority

No <link rel="canonical"> tag found.

0 / 4

⌄

Redirect and host health

Good: http redirects to https, no redirect needed. To fix: www.credence.today serves the site too instead of redirecting, which splits your authority across two hostnames.

2.5 / 4

⌄

✓

Accessibility and speed

HTTPS confirmed, homepage downloaded in 135ms.

3 / 3

⌄

Semantic structure

Issues: no <main> landmark; no lang attribute on <html>; only 1 of 2 images have alt text.

3 / 6

⌄

Q&A schemabonus

No FAQPage or HowTo schema found. Adding structured Q&A can significantly increase AI citations.

0 / 2

⌄

Content feedbonus

No RSS or Atom feed detected. A content feed signals an actively updated site that agents should re-index.

0 / 1

⌄

Agent discovery endpointsbonus

No agent discovery documents found (MCP server card, A2A agent card, API catalog, agents.json, or OpenAPI description).

0 / 2

⌄

Content freshnessbonus

No machine-readable publish or modified dates found (JSON-LD, meta tags, <time> elements, or sitemap <lastmod>).

0 / 2

⌄

Fix this with AI

A complete brief covering all 19 outstanding issues, with the reason each one matters and the exact fix. Copy it into Claude, ChatGPT, Cursor, or any coding agent working on credence.today.

# AI readiness fixes for credence.today

Audit by AI Web Readiness (https://aiwebreadiness.com) on 2026-11-09. Full report: https://aiwebreadiness.com/report/Oe5mzlKwN3
Current score: 14/100 (Poor). 1 of 20 checks at full marks.

## Your task

You are working on the website https://credence.today. An automated audit measured how well this site can be discovered, parsed, and cited by AI agents and LLM-powered search engines. The issues it found are listed below, ordered by how many points each is worth.

Work through them in order, highest value first. For each issue, make the change in this site's codebase or CMS using the stack and conventions already in use. Where a fix needs copy, facts, or URLs you cannot verify from the existing site, ask me rather than inventing them.

There are 85.5 points on the table across these 15 issues. Fixing all of them reaches 100/100. If you only have time for the top 3, that alone is worth 29 points and takes the score to 43/100.

---

## Issues to fix (15)

### 1. AI agent manifest (10 points available)

**What the scan found:** No /llms.txt found (status 404).

**Why it matters:** An llms.txt file is a short, plain-text briefing written for AI agents. It states what your site is, what you offer and which pages matter most, so an agent can understand you in seconds instead of crawling dozens of pages and guessing. Without it, an assistant has to infer your purpose from whatever it happens to find, and it may describe you inaccurately or skip you.

**How to fix it:** Create a plain text file at /llms.txt in your site root. It should start with a # Title line, a short > description, and a list of your key pages as markdown links. Use the draft generated below as your starting point. It is ready to deploy as-is.

### 2. Indexability (10 points available)

**What the scan found:** The homepage is marked noindex in the robots meta tag, which tells search engines and AI search tools to leave it out.

**Why it matters:** These are the most damaging problems a site can have, because they work silently. A leftover noindex from a staging site, or a blanket Disallow in robots.txt, removes your pages from search engines, and the AI assistants that rely on search results cannot find or cite what is not indexed. A site can score well everywhere else and still be invisible if this check fails, which is why it carries the highest weight.

**How to fix it:** Remove any noindex or none value from your robots meta tag and X-Robots-Tag header on pages you want found. Do not use nofollow or nosnippet on pages you want cited. Make sure robots.txt does not Disallow: / for Googlebot or for User-agent: *.

### 3. Structured data (9 points available)

**What the scan found:** No JSON-LD structured data found on the homepage.

**Why it matters:** JSON-LD states plainly, in a format machines read without guessing, who you are, what you offer and how your pages relate to each other. Search engines and AI systems use it to build their understanding of an entity and to decide what to show or quote. Without it, they must infer meaning from layout and wording, and the inference is often wrong or shallow. Richer, more specific types give agents more facts they can use with confidence.

**How to fix it:** Add a <script type="application/ld+json"> block to your homepage <head>. At minimum include @context, @type (e.g. Organization, WebSite, or Article), name, url, and description. For higher scores add richer types like BreadcrumbList, Article, or Product on the relevant pages. Google's Structured Data Markup Helper is a good starting point.

### 4. AI crawler permissions (8 points available)

**What the scan found:** Blocks 14 of 14 tracked AI agents: gptbot, claudebot, claude-web, anthropic-ai, perplexitybot.... Blocking is a legitimate choice, but a blocked agent cannot read the site, so it earns no credit.

**Why it matters:** Robots.txt is the first thing a well-behaved crawler reads, and a block there means the agent never sees your content at all. Some sites block AI agents on purpose, which is a legitimate choice, but many do it by accident through an overly broad rule or a security plugin. This check shows the real consequence: every agent that is shut out cannot read, learn from or cite your site.

**How to fix it:** Do not Disallow: / for the AI crawlers you want reading your site, and do not block them with a blanket User-agent: * rule. Optionally name them explicitly with Allow: / rules (GPTBot, ClaudeBot, PerplexityBot, Google-Extended). Blocking an agent is a legitimate choice, but it lowers this score. Publish a /robots.txt even if it only allows everything and points at your sitemap.

### 5. llms.txt quality (6 points available)

**What the scan found:** There is no usable /llms.txt to evaluate. Add one first.

**Why it matters:** Agents rely on that structure to decide where to go next. A file that is just a paragraph of prose, or one full of broken or off-site links, gives them little to act on and can send them to dead ends. A well-formed file works like a trustworthy table of contents: it tells an assistant which pages answer which questions, which makes it more likely your best pages are the ones that get read and cited.

**How to fix it:** Follow the llmstxt.org format. Start with a single "# Site name" line, add a "> one sentence summary" blockquote, then group at least three markdown links under "## Section" headings, written as "- [Page name](https://yoursite.com/page): what it covers". Keep links on your own domain and make sure each one loads.

### 6. Page index (6 points available)

**What the scan found:** No /sitemap.xml (status 401).

**Why it matters:** A sitemap is the list of pages you want found. It lets crawlers discover content that internal links might not surface and tells them which pages have changed recently, so updates are picked up faster. Without one, discovery depends on crawlers following links, and pages that are deep in your site, new or poorly linked can be missed for a long time.

**How to fix it:** Create a valid XML sitemap listing your important pages with <loc> and <lastmod> entries, serve it at /sitemap.xml, and add a line 'Sitemap: https://yourdomain.com/sitemap.xml' to your robots.txt. Most CMS platforms generate this automatically.

### 7. Readable content ratio (5.5 points available, currently scoring 1.5 of 7)

**What the scan found:** 78 chars of text, 2.6% text-to-HTML ratio. Improving content density would raise this score.

**Why it matters:** AI crawlers read pages under strict size and time limits. A page that is mostly code, trackers and layout markup gives them very little real content for the effort and tends to be deprioritised or truncated. A page with a healthy amount of readable text is quick to process and easy to summarise, which makes it more likely to be understood correctly and used as a source.

**How to fix it:** Reduce inline scripts and styles, remove unused third-party widgets, and ensure your main content is substantial. Aim for at least 1,000 characters of visible body text and a text-to-HTML ratio above 10%. Moving scripts to external files and deferring non-critical CSS both help.

### 8. Crawler-visible content (5.5 points available, currently scoring 2.5 of 8)

**What the scan found:** 78 chars server-rendered, 1 H1, 0 H2, 1 substantial paragraph(s). Add more structured content to improve this score.

**Why it matters:** Many AI crawlers do not run JavaScript. If your site builds its content in the browser, those crawlers receive an almost empty page and conclude that there is nothing here, even though a human visitor sees a full site. Delivering content in the initial HTML, through server rendering or static generation, is what makes it visible to the widest range of search engines and AI tools.

**How to fix it:** Ensure your pages use server-side rendering (SSR) or static generation so content appears in the raw HTML response. If you use React, Next.js, or similar frameworks, enable SSR or SSG. At minimum your H1, primary heading structure, and first few paragraphs of content should be present without JavaScript.

### 9. Attribution schema (5 points available)

**What the scan found:** No Organization or Person entity, and no author or publisher property, found in JSON-LD structured data.

**Why it matters:** When an AI assistant quotes or recommends a source, it needs to know who stands behind the content. Clear attribution data lets it credit you by name, connect your site to your profiles and reputation, and trust that the content comes from a real, identifiable author or organization. Anonymous content is easier to ignore, and harder to cite with confidence.

**How to fix it:** Add an Organization (or Person) entity to your JSON-LD with a name, a url, a logo, and sameAs links to your official profiles (LinkedIn, Wikipedia, social accounts). On articles and content pages, add author and publisher properties that reference those entities.

### 10. Page identity signals (5 points available, currently scoring 2 of 7)

**What the scan found:** title: 11 chars, description: missing. Titles of 40-70 and descriptions of 50-160 characters score full marks.

**Why it matters:** The title and description are the first things a search engine or an AI agent reads to decide what a page is about and whether it is relevant to a question. Missing or vague ones make the page harder to classify, and overlong ones get truncated in search results and citations. A clear, specific title and description improve how often your page is chosen and how accurately it is described.

**How to fix it:** Write a descriptive <title> tag of 40-70 characters that clearly names the page and your brand. Add a <meta name="description"> of 50-160 characters that summarises what the page offers. Longer values get truncated. Both should be unique per page, specific rather than generic, and written for humans.

### 11. Social and citation metadata (4 points available)

**What the scan found:** No OpenGraph tags found.

**Why it matters:** OpenGraph tags are the headline, summary and picture that others see when your page is shared, previewed or cited. AI assistants and link-preview systems use them to describe your page quickly and consistently. When they are missing, the tool has to improvise from the page body, and the result is often an awkward title, a wrong summary or no image at all.

**How to fix it:** Add OpenGraph meta tags to your page <head>: og:title (your page title), og:description (a 1-2 sentence summary), og:image (a representative image URL), and og:site_name (your brand name). Each tag present adds to your score. These same tags are used by social platforms when your content is shared.

### 12. Canonical authority (4 points available)

**What the scan found:** No <link rel="canonical"> tag found.

**Why it matters:** The same page is often reachable at several addresses, such as with and without www, with tracking parameters, or through old links. The canonical tag tells search engines and agents which address is the real one so that your authority is gathered in one place. A missing tag splits that authority, and a wrong one can tell search engines to drop your pages in favour of an address that does not exist.

**How to fix it:** Add <link rel="canonical" href="https://yourdomain.com/page-url"> to the <head> of every page. Use an absolute URL on your own domain that returns a 200 and matches the page itself. Never point it at a staging, old, or unrelated domain.

### 13. Full-content AI feed (3 points available)

**What the scan found:** /llms-full.txt returned HTTP 200, but the response body is an HTML page, not the expected file. This site answers 200 for URLs that do not exist, so a 200 alone does not prove the file is there.

**Why it matters:** Where llms.txt is a map, llms-full.txt is the whole territory: the full text of your key pages in one place. An agent with limited time or a limited context window can read your content in a single request, with no page-by-page crawling and no risk of missing something behind navigation or scripts. It is a small file to publish, and it lets AI tools represent your content accurately and completely.

**How to fix it:** Create /llms-full.txt at your site root containing the full text of your most important pages, concatenated in order. Each page should be separated by a clear heading. Many static site generators and CMS platforms have plugins that generate this automatically.

### 14. Semantic structure (3 points available, currently scoring 3 of 6)

**What the scan found:** Issues: no <main> landmark; no lang attribute on <html>; only 1 of 2 images have alt text.

**Why it matters:** Agents cannot see your design. They read the structure of the markup to work out what the page is about, which part is the main content and how it is organised. Clean headings and landmarks let them extract the right text and skip navigation and footers. A page built from anonymous containers is much harder to interpret, so its content is more likely to be misread or left out.

**How to fix it:** Use exactly one <h1> per page and do not skip heading levels (h1, h2, h3 in order). Wrap primary content in <main>. Set <html lang='en'> (or your language). Add <meta name='viewport' content='width=device-width, initial-scale=1'>. Give meaningful images an alt attribute.

### 15. Redirect and host health (1.5 points available, currently scoring 2.5 of 4)

**What the scan found:** Good: http redirects to https, no redirect needed. To fix: www.credence.today serves the site too instead of redirecting, which splits your authority across two hostnames.

**Why it matters:** Every extra version of your address splits your search presence and wastes crawler time. When both www and non-www serve the site, or http does not redirect to https, search engines can treat them as separate sites and share your authority between them. Long redirect chains slow crawlers down and some give up before the end. One clean permanent redirect to a single main address keeps everything together.

**How to fix it:** Add a single permanent (301) redirect from http to https, and from the version of your hostname you do not use (www or non-www) to the one you do. Aim for one hop or none between any entry URL and the final page.

---

## Optional bonus checks

These are scored separately from the 100-point core and do not change the headline score. They only apply if they fit the site. Skip any that do not make sense for what this site is.

- **Q&A schema** (up to 2 more points): No FAQPage or HowTo schema found. Adding structured Q&A can significantly increase AI citations. How: On any page that contains questions and answers, add a FAQPage JSON-LD block with Question and Answer entities. Each Question needs a name (the question text) and an acceptedAnswer with a text property. For step-by-step content, use HowTo schema instead. Google's Rich Results Test can validate your implementation.
- **Agent discovery endpoints** (up to 2 more points): No agent discovery documents found (MCP server card, A2A agent card, API catalog, agents.json, or OpenAPI description). How: Publish at least one machine-readable discovery document that applies to you, for example /.well-known/mcp.json for an MCP server, /.well-known/agent-card.json for an A2A agent, /.well-known/api-catalog for your APIs, or /openapi.json for a REST API. Two or more earn full marks.
- **Content freshness** (up to 2 more points): No machine-readable publish or modified dates found (JSON-LD, meta tags, <time> elements, or sitemap <lastmod>). How: Expose dates in machine-readable form: datePublished and dateModified in your JSON-LD, article:modified_time meta tags, <time datetime> elements, and <lastmod> in your sitemap. Keep them truthful and update them when content changes.
- **Content feed** (up to 1 more points): No RSS or Atom feed detected. A content feed signals an actively updated site that agents should re-index. How: Create an RSS or Atom feed at /feed.xml or /rss.xml listing your most recent content with title, link, description, and pubDate per item. Add a <link rel="alternate" type="application/rss+xml" href="/feed.xml"> tag to your <head> so agents can discover it automatically. A feed that exists but is not advertised earns partial credit.

---

## Ready-to-deploy /llms.txt

This draft was generated from the site's own metadata and links. Review the wording, correct anything inaccurate, then save it as a plain text file at the site root so it resolves at https://credence.today/llms.txt

```
# Coming Soon

> Pages and resources from credence.today.

## Key Pages
- [Home](https://credence.today/)

## About
This file follows the /llms.txt convention, which provides AI agents and language models with a structured map of this site. Learn more at https://llmstxt.org.
```

---

## Already passing, do not regress

These signals are already at full marks. Preserve them while making the changes above:

- **Accessibility and speed** (3 pts): HTTPS confirmed, homepage downloaded in 135ms.

---

## When you are done

Deploy the changes, then rescan credence.today at https://aiwebreadiness.com to confirm the score moved. The report tracks the delta between scans automatically.

Click the box to select all · 17,327 characters

/llms.txt draft

Ready-to-deploy file. Drop it in your site root so AI agents can find a structured map of what you offer.

⌄