If you've read anything about getting cited by AI in the last year, you've been told to add an
The idea is reasonable on its face. Crawlers waste effort parsing cluttered HTML; a single
Here's what the advice usually leaves out: no major AI company has confirmed it uses llms.txt, and there's no measured ranking or citation lift from adding one. As practitioners tracking it through 2025 have documented, it remains a community proposal, not an adopted standard — no formal commitment from OpenAI, Anthropic, Google, or Perplexity, and no controlled study showing the file changed who got cited. Google's own search team has publicly said it isn't something they use.
So treat it as cheap insurance: harmless, possibly useful later if adoption comes, trivial to add. But the file is a table of contents. A table of contents to pages a crawler can't read does nothing — and if your pages are client-rendered, that's exactly the situation you're in. The file is not the work.
The levers with real evidence behind them are all properties of how the site is built and what's on the page:
Notice the shape of that list. Every real lever is structural — a property of your rendering, your content model, and your publishing discipline. None of them is a text file you paste in. Which is the recurring theme: AI visibility is decided by the architecture of your CMS, and a one-file shortcut can't substitute for it any more than a meta description ever substituted for a good page.
This is exactly why agntcms is built the way it is — server-rendered public pages, content stored as structured JSON, frozen dated snapshots for stable URLs. Add the llms.txt too, by all means. But the citations come from the page being readable, structured, and stable underneath it, not from the file pointing at it.
llms.txt is cheap to add and unproven to matter — a community proposal no major model has confirmed it uses, with no measured citation lift. Add it, then ignore it. The levers with evidence are all structural: server-rendered HTML, structured content, citable substance, stable URLs. A table of contents to pages a crawler can't read is a map to nowhere.
Start with what's measurable — test whether your site is machine-readable, fix the rendering wall, and understand why the CMS is the bottleneck. agntcms is open source on GitHub.
github.com/agntcms
llms.txt file. Drop a curated, plain-text map of your site at the root, the pitch goes, and the language models will read it and prefer you. It's become the default first move — the AEO equivalent of "add a meta description." It is also, on the current evidence, mostly a placebo. Worth doing in five minutes, not worth mistaking for the work. Here's the honest status, and what actually moves the needle.The llms.txt pitch
The idea is reasonable on its face. Crawlers waste effort parsing cluttered HTML; a single
llms.txt file would hand them a clean, curated index of your most important pages in Markdown, the format models digest most easily. It rhymes with robots.txt and sitemap.xml, so it feels like an established standard. The how-to posts followed in force, and now it's near-universal advice.The honest status
Here's what the advice usually leaves out: no major AI company has confirmed it uses llms.txt, and there's no measured ranking or citation lift from adding one. As practitioners tracking it through 2025 have documented, it remains a community proposal, not an adopted standard — no formal commitment from OpenAI, Anthropic, Google, or Perplexity, and no controlled study showing the file changed who got cited. Google's own search team has publicly said it isn't something they use.
So treat it as cheap insurance: harmless, possibly useful later if adoption comes, trivial to add. But the file is a table of contents. A table of contents to pages a crawler can't read does nothing — and if your pages are client-rendered, that's exactly the situation you're in. The file is not the work.
What actually moves the needle
The levers with real evidence behind them are all properties of how the site is built and what's on the page:
- The page renders on the server. The non-negotiable floor: AI crawlers don't run JavaScript, so the content has to be in the raw HTML. Test yours in 60 seconds. An llms.txt pointing at unreadable pages is a map to nowhere.
- The content is structured. Clean headings, lists, and schema make facts extractable — the decoupled, content-as-data shape engines parse best.
- The page earns the quote. Statistics, quotations, and citations lift generative-engine visibility 30–40% in the Princeton GEO study. Substance gets lifted; filler gets skipped.
- The site is stable. Citation engines swap 40–60% of their cited domains month to month; fixed URLs and consistent entities make you the source they keep, not the one they drop.
The architecture point
Notice the shape of that list. Every real lever is structural — a property of your rendering, your content model, and your publishing discipline. None of them is a text file you paste in. Which is the recurring theme: AI visibility is decided by the architecture of your CMS, and a one-file shortcut can't substitute for it any more than a meta description ever substituted for a good page.
This is exactly why agntcms is built the way it is — server-rendered public pages, content stored as structured JSON, frozen dated snapshots for stable URLs. Add the llms.txt too, by all means. But the citations come from the page being readable, structured, and stable underneath it, not from the file pointing at it.
The short version
llms.txt is cheap to add and unproven to matter — a community proposal no major model has confirmed it uses, with no measured citation lift. Add it, then ignore it. The levers with evidence are all structural: server-rendered HTML, structured content, citable substance, stable URLs. A table of contents to pages a crawler can't read is a map to nowhere.
Start with what's measurable — test whether your site is machine-readable, fix the rendering wall, and understand why the CMS is the bottleneck. agntcms is open source on GitHub.
github.com/agntcms