Guide
llms.txt, explained
llms.txt is a plain-text file at the root of a site that gives AI systems a curated summary of what the site offers and where its key pages are. It is a proposed convention rather than a standard, it costs very little to publish, and it complements robots.txt rather than replacing it.
Last updated
What it is for
A crawler can read your navigation, but navigation is designed for people and often buries what matters. llms.txt states it directly: here are the pages that explain this product, here is the pricing, here is the documentation.
Some sites also publish a longer llms-full.txt carrying complete product facts, pricing and FAQs, so an assistant answering a question about you has accurate material to quote rather than assembling it from fragments.
What belongs in it
Keep it short and link-led. A heading, a one-line description of the site, then sections of links with a line of description each.
- The pages that explain what you do, one line each
- Pricing, stated plainly, with the actual numbers
- Documentation or reference material, if you have it
- What you are not — a surprisingly useful line, because it heads off wrong associations
- A link to the longer llms-full.txt if you publish one
How it differs from robots.txt
robots.txt controls access: which agents may fetch which paths. llms.txt offers orientation: given that you can read this site, here is what is worth reading.
They are not alternatives. A site with a perfect llms.txt that blocks the retrieval crawlers is still invisible, and access without orientation still works — llms.txt simply makes the job easier.
An honest note on adoption
No major assistant has committed publicly to reading llms.txt, and you should treat anyone claiming guaranteed effects with caution. It is a low-cost bet on a convention that may become useful.
The stronger reason to write one is the exercise itself: deciding which fifteen pages actually explain your product usually improves the site regardless of who reads the file.
Questions
Do AI assistants actually read llms.txt?
There is no public commitment from the major assistants that they do. It is cheap to publish and harmless, but it should sit well below crawler access and source strategy on any priority list.
Where does the file go?
At the root of the domain, served as text/plain — the same placement as robots.txt. A longer llms-full.txt sits alongside it if you publish one.
Should it be generated or hand-written?
Generated from the same route list that feeds your sitemap, so it cannot drift. A hand-written file goes stale the first time someone adds a section and forgets.
Terms used here
Read next
AI crawlers, explained
Every major AI crawler, what each one actually does, and how to decide which to allow without accidentally removing yourself from answers.
Structured data for AI search
Which schema types earn their place, what they actually do for AI answers, and the one rule that keeps you out of trouble.
Answer engine optimization, explained
What AEO is, how it differs from SEO, and the three kinds of work that actually move whether an AI assistant recommends your brand.
Ready to Dominate AI Search?
Get started now
Allswap