Article
Aug 17, 2026
llms.txt: make your site readable by AI
LLMs crawl your site but keep the wrong content. llms.txt is the map you give them to guide what they read and cite. Here's what I learned building mine.
AI reads your site, but not the way you think
When a model like ChatGPT, Claude or Gemini explores the web, it doesn't read your site page by page like a curious visitor. It swallows text in bulk, pulls out whatever looks like facts, and keeps what it judges reliable. The result: it often keeps the wrong passage, an old footer sentence or an empty marketing headline, instead of the clean description of what you do.
The llms.txt file exists to reduce that randomness. It's a map you hand the models to steer what they read and, above all, what they cite. Here's what I understood building mine.
What the llms.txt file is
It's a simple text file, written in Markdown, that you place at the root of your site. Its job is to present your product to language models in a clean format, with no menu, no cookie banner, no script. You put the essentials in it: what you do, who it's for, and the pages that truly deserve to be read.
The idea is close to what a sitemap is for Google, but the intent differs. A sitemap lists URLs for indexing bots. An llms.txt file summarizes meaning for models that generate answers. The first says "here are my pages," the second says "here's what to understand about me."
What an llms.txt file looks like
The structure fits in a few blocks: a title, a one-sentence summary as a quote, then sections of annotated links. Here's an example close to the one I use for Shiiift.
Each link is followed by a short description. That annotation matters more than the URL itself: it's what tells the model why the page exists and when to cite it. A list of bare URLs adds almost nothing.
What llms.txt is not
People often confuse it with robots.txt. Both are files at the root of the site, but their roles are opposite. robots.txt is there to block or allow bots from accessing parts of the site. llms.txt blocks nothing: it guides understanding. One is a barrier, the other is a signpost.
Another misunderstanding: it's not a guarantee of citation. No file forces a model to talk about you. llms.txt increases your chances of being read correctly, but it replaces neither good content nor a real presence on the web. It's a lever, not a switch.
Build yours, step by step
Start with an honest summary sentence: what your product does, who for, in plain language, no superlatives.
List your key pages: product page, documentation, pricing, in-depth articles. For each, write a one-line annotation that explains its content.
Cut the noise: no legal pages, no checkout funnels, no duplicate content. Keep only what you'd want to see cited.
Stay concise: a short, clean file beats a full inventory. You're steering, not copying your whole site.
Where to put it and how to check it
The file goes at the root of the domain, at a simple, predictable address.
Once it's live, open the URL in a browser to confirm it returns plain text, with no redirect or error page. Then reread each line and ask yourself: if a model read only this file, would it honestly understand what I do? If the answer is no, the problem isn't the format, it's the content.
llms.txt doesn't replace a readable page
The file points the model to your best pages, but those pages have to keep their promise. If your product page stays vague, llms.txt won't change a thing. That's why it goes hand in hand with the deeper work on how to optimize your product page to get cited by ChatGPT.
In the same way, some content formats get cited far more often than others. Before writing, it's worth looking at which content formats LLMs cite most, then structuring your pages accordingly.
Where to start
Create a first short llms.txt: an honest summary, three or four annotated links, nothing more. Put it online, check the URL, and let it evolve as your pages improve. It's one brick among others in the broader effort to appear in the answers of ChatGPT, Claude and Gemini.
This is exactly the kind of work Shiiift aims to make simple: spotting what AI understands about your brand and fixing what gets cited badly. The product is still in early access, and this file remains one of the first useful moves you can make today, without waiting.