llms.txt explained: what it is and whether you need one

Updated 18 October 2026 · AI Fix Shop

Check your site free: Run the free AI readiness checker (no sign-up, about 20 seconds).

llms.txt is a proposed standard: a plain Markdown file at the root of your site (yoursite.com/llms.txt) that gives AI tools a short, curated guide to your most important content. It was proposed in September 2024 by Jeremy Howard of Answer.AI, and it has spread quickly among documentation sites, developer tools and SaaS products.

What problem it tries to solve

Web pages are full of navigation, scripts, cookie banners and layout code. A language model reading your site has limited space and has to dig the useful parts out. llms.txt offers a shortcut: a concise summary and a list of links to clean, readable pages, ideally with Markdown versions.

How it differs from robots.txt and sitemap.xml

• robots.txt tells crawlers what they may and may not fetch. It's about permission.
• sitemap.xml lists every URL you want indexed. It's about completeness.
• llms.txt highlights what matters most and explains it in plain language. It's about understanding.

llms.txt does not block or allow anything. If you want to control AI crawlers, that's robots.txt; see GPTBot vs OAI-SearchBot.

The format

The proposal defines a simple structure, in this order:

1. An H1 with the name of the site or project (the only required part).
2. A blockquote with a short summary.
3. Optional paragraphs or lists with more context.
4. H2 sections containing Markdown link lists, each link optionally followed by a colon and a description.
5. An optional section titled "Optional" for secondary links that can be skipped when space is short.

Template for a small business

# Rossi Bakery

> Family bakery in Bologna, Italy, since 1987. Fresh bread, pastries
> and custom cakes. Open Tuesday to Sunday, 7:00-19:00.

We take custom cake orders with 48 hours' notice and deliver within Bologna.

## Key pages
- [Menu and prices](https://rossibakery.example/menu): full product list with prices
- [Custom cakes](https://rossibakery.example/cakes): how to order, lead times, pricing
- [Visit us](https://rossibakery.example/contact): address, hours, phone, map

## Policies
- [Delivery](https://rossibakery.example/delivery): areas, fees and times
- [Allergens](https://rossibakery.example/allergens): ingredients and allergen information

## Optional
- [Our story](https://rossibakery.example/about)

Save it as plain text, UTF-8, at the root of your domain, served with a text content type.

An honest view: does anyone read it?

This is where many articles overpromise. As of this writing, the major AI search providers haven't publicly committed to using llms.txt as a ranking or retrieval signal, and Google representatives have said Google Search doesn't use it. Some AI coding tools and agents do read llms.txt files, especially for software documentation, and some site owners report AI crawlers fetching the file in their logs. Fetching isn't the same as relying on it.

So treat llms.txt as a cheap, low-risk extra, not a fix. It won't help if the AI search bots are blocked, if your content only appears after JavaScript runs, or if your pages don't explain clearly what you do. Those basics matter far more. The upside: writing one forces you to state, in a few lines, who you are and which pages matter, which often improves the site itself.

Do you need one?

• Developer tools, APIs, documentation: yes, it's becoming expected and AI coding tools use it.
• SaaS and content sites: worth the fifteen minutes.
• Local businesses and small shops: optional. Do robots.txt, Organization schema and clear page text first.

Common mistakes

• Listing every page (that's what the sitemap is for).
• Marketing language instead of facts.
• Broken or redirected links.
• Putting it in a subfolder instead of the root.
• Letting it go stale when prices or hours change.

The free checker tells you whether your site has an llms.txt, alongside the crawler access and readability checks that matter more. You can see ours at /llms.txt.

FAQ

Is llms.txt an official standard?

No. It's a community proposal from September 2024. It's widely discussed and adopted by many documentation sites, but no standards body has approved it.

Does llms.txt help me rank in ChatGPT or Google?

There's no public confirmation that major AI search providers use it as a signal, and Google has said its Search doesn't. Treat it as a low-cost extra.

Does llms.txt replace robots.txt?

No. robots.txt controls crawler access; llms.txt only summarizes your content. You need robots.txt rules regardless.

Check your site free: Run the free AI readiness checker (no sign-up, about 20 seconds).