What the file is and where it lives
llms.txt sits at your domain root, alongside robots.txt and sitemap.xml, at https://example.com/llms.txt. Unlike robots.txt it issues no instructions and grants no permissions — it's a curated index in Markdown, meant to let a model find your best pages without crawling and parsing your whole site.
Jeremy Howard, co-founder of Answer.AI, proposed it in September 2024. The stated problem was real: HTML pages are full of navigation, cookie banners and scripts, and a model with a limited context window wastes most of it on furniture rather than content.
A companion convention, llms-full.txt, concatenates the actual content of those pages into one long Markdown file so a model can ingest everything in a single fetch.
A working sample file
The format is deliberately simple — an H1 with the site name, a blockquote summary, then H2 sections containing Markdown links with short descriptions. An ## Optional section marks pages a model may skip when it's short on context.
# Meridian Ceramics> Meridian Ceramics is a Jaipur-based manufacturer of hand-glazed tableware, shipping to 14 countries.## Products- [Dinner sets](https://example.com/dinner-sets): Sizes, glazes, lead times and minimum order quantities.- [Custom orders](https://example.com/custom): The process, tooling costs and typical eight-week timeline.## Company- [About](https://example.com/about): Founded 2009, 40-person studio, kiln capacity and certifications.## Optional- [Press coverage](https://example.com/press): Media mentions since 2019.
What Google and the model providers have actually said
Google's John Mueller has publicly compared llms.txt to the old keywords meta tag — a file a site owner writes about itself, which a search system therefore has little reason to trust — and has said he isn't aware of any AI system currently using it. Google has been consistent that it has no bearing on ranking.
On the provider side, none of the major labs have documented llms.txt support in their crawler documentation. Where the file does get read today, it's mostly by developer tools, coding assistants and documentation platforms fetching a specific site on request, rather than by large-scale training or retrieval crawls.
That's the honest state of it. Not a scam, not a standard, and nowhere near the thing being sold in 2026 as a shortcut to AI visibility.
When it's worth it, and when to skip
Two clear cases, and a large middle where the answer is genuinely no.
| Your situation | Verdict |
|---|---|
| Your docs platform generates it automatically | Keep it. Zero cost, small upside. |
| You publish technical documentation people query in coding assistants | Worth doing. This is the case the proposal was designed for. |
| Small business site, 20 to 200 pages, no developer on call | Skip it, or spend the hour writing it by hand and expect nothing. |
| An agency wants a paid sprint for it | No. Spend it on internal links or page speed. |
| You have no XML sitemap yet | Fix that first. Sitemaps are read by systems that demonstrably exist. |
What to do instead, if AI visibility is the goal
If the real question is being cited by assistants, the answers sit elsewhere and they're not new. Make sure your XML sitemap is accurate and your robots.txt doesn't accidentally block the AI crawlers you want to be read by — a genuinely common own-goal, and a decision worth making deliberately rather than inheriting from a template.
Then do the work that actually moves citation rates: rank for the query, write extractable answers, and build third-party presence. That's covered properly in generative engine optimization, and none of it fits in a text file at your domain root.