What is llms.txt? Format, examples and whether you need one

What llms.txt is

llms.txt was proposed by Jeremy Howard of Answer.AI in September 2024, and the specification lives at llmstxt.org. The idea is simple: a web page is full of navigation, scripts and layout that a language model has to wade through, and a model's context window is limited. A short, hand-picked list of a site's key pages, each with a one-line description, gives an AI tool a clean starting point.

The file lives at https://example.com/llms.txt, next to robots.txt. The specification also allows one in a subfolder, such as /docs/llms.txt, covering only that section.

The format, with an example

llms.txt is plain Markdown with a fixed order:

1. An H1 with the site or project name. This is the only required part.
2. A blockquote with a one or two sentence summary.
3. Optional paragraphs or lists with any context that helps, but no headings.
4. H2 sections of links, each link written as - [Name](url): note, where the note is optional.
5. A section called Optional, by convention, for links a tool can skip when it's short of space.

# Example Co

> Example Co makes accounting software for small businesses in the UK.

Prices are in GBP and include VAT. Support is available on weekdays.

## Product
- [Features](https://example.com/features/): What the software does, by plan
- [Pricing](https://example.com/pricing/): Plans, prices and what each includes

## Docs
- [Getting started](https://example.com/docs/start/): Set up an account and connect a bank feed
- [Importing data](https://example.com/docs/import/): Move from spreadsheets or another tool

## Optional
- [Company history](https://example.com/about/)
- [Privacy policy](https://example.com/privacy/)

Keep it short. The point is to pick the pages that answer most questions, not to copy your sitemap. The proposal also suggests publishing a clean Markdown version of each page at the same URL with .md added, and some sites publish an llms-full.txt with the full text of their documentation in one file. Both are optional extras.

llms.txt vs robots.txt vs sitemap.xml

The important difference: llms.txt doesn't allow or block anything. If you want to control which AI crawlers can fetch your pages, that is robots.txt's job, covered in how to block GPTBot and other AI crawlers.

Who actually reads it

This is where most articles about llms.txt overstate things. As of October 2026:

• Google says in its documentation on AI features in Search that "you don't need to create new machine readable files, AI text files, or markup" to appear in AI Overviews or AI Mode. Pages qualify the same way they qualify for normal results: by being indexed and eligible for a snippet. Google's John Mueller has said that no AI system currently uses llms.txt.
• OpenAI, Anthropic and Perplexity document their crawlers and how robots.txt controls them, but none has said its search or training crawlers use llms.txt to choose or rank pages.
• Developer tools are the real audience. Many documentation sites publish llms.txt so developers can paste it, or point a coding assistant at it, to give the model accurate, current docs. Here it is genuinely useful.

Server logs are the honest test. If you publish one, check how often /llms.txt is requested and by which user agents.

Should you add one?

• Yes, if you publish documentation, an API or a developer product. It helps people who use AI assistants with your product, and it's quick to maintain.
• It's optional for everyone else. It costs little and can't hurt, but don't expect it to change rankings, AI Overview appearances or citations in ChatGPT.
• What does affect AI search: pages that are crawlable by the bots you want, indexed, and that answer questions clearly in their HTML. A technical SEO checklist covers those basics.

How to create one

1. List the 10 to 50 pages that answer most questions about your business or product.
2. Write a one-line description of each, saying what it answers, not what it's called.
3. Group them under a few H2 headings, and put legal and background pages under Optional.
4. Save it as UTF-8 plain text named llms.txt and upload it to your site's root.
5. Check that https://yoursite.com/llms.txt loads and that every link in it works.

Or start from a draft built from your own pages:

The generator reads your homepage and the pages it links to, skips redirects, errors, noindex pages and non-canonical URLs, and groups the rest into sections. Edit the result before uploading; the summary and order matter more than completeness. Its check mode tests an existing llms.txt for format and broken links.

Questions