The llms.txt question: should you add one, and does it do anything?

Every few months a new technical “must-do” sweeps SEO Twitter, and right now it is llms.txt — a proposed plain-text file you place at the root of your site to tell AI models what matters on it. The pitch is tidy: just as robots.txt guides crawlers and sitemap.xml guides indexers, llms.txt would guide language models to your most important content. Dozens of “add this today” threads followed. So should you add one? The honest answer is more interesting than either the hype or the backlash, and it is worth understanding properly before you spend a single hour on it.

Here is the short version, so you are not left guessing: adding llms.txt is cheap, harmless, and currently does little to nothing that we can measure, because no major AI engine has committed to reading it. It is a reasonable low-effort bet on a possible future, not a lever that moves your AI visibility today. If someone is selling it to you as the key to getting cited by ChatGPT, they are overselling. If someone tells you it is pure snake oil, they are overclaiming in the other direction. The truth sits in the unglamorous middle, and the middle is where good technical SEO decisions usually live.

What llms.txt is supposed to do

The idea borrows directly from two files every site already has. robots.txt tells crawlers where they may and may not go; sitemap.xml hands indexers a tidy list of your URLs. llms.txt proposes a third file in the same spirit, aimed at language models: a Markdown document at your root that lists and briefly describes your most important pages, so that a model trying to understand your site has a curated, clean summary to work from instead of crawling everything and guessing. Some versions of the proposal go further and suggest companion files containing the full clean text of key pages, so a model can ingest your best content without wading through navigation and markup. As a piece of design thinking it is genuinely sensible — models do prefer clean, structured input, and a curated map of your site is exactly the kind of thing that would help one understand you faster.

The problem: nobody has agreed to read it

A standard is only as useful as its adoption, and this is where the enthusiasm gets ahead of reality. For robots.txt and sitemap.xml to matter, the major search engines had to agree to honour them — and they did, explicitly, decades ago. For llms.txt to matter in the same way, the companies behind ChatGPT, Gemini, Perplexity and the rest would need to commit to fetching and respecting it. As of now, none of the major engines has publicly confirmed that it reads llms.txt as part of how it retrieves or cites sources. That does not mean no model ever touches it, and it does not mean it will stay that way — standards sometimes get adopted after a grassroots push. But it does mean that today you cannot point to a mechanism by which the file changes whether you get cited, and you should be sceptical of anyone who claims otherwise without evidence.

Why the hype runs ahead of the evidence

There is a predictable pattern to these moments, and it helps to recognise it. A plausible-sounding technical tweak appears; it is easy to implement, which makes it easy to write about and easy to sell; and because AI visibility feels mysterious and urgent, people grab for anything concrete they can control. A root-level file you can add in twenty minutes is irresistibly tangible compared to the slow, hard work of earning authority. So it spreads, not because anyone has measured it working, but because it scratches the itch of wanting a lever. The same thing happened with countless meta tags and schema types over the years, most of which turned out to be marginal or inert. Scepticism here is not cynicism; it is pattern recognition.

So should you add one? A straight answer

Yes, if — and only if — it is genuinely a quick job and you treat it as a cheap option on a possible future rather than a priority. The downside is essentially nil: a well-formed llms.txt cannot hurt you, and if the standard does get adopted, you are already set up. So if you can write an accurate one in half an hour, do it and move on. What you should not do is spend real time crafting and maintaining elaborate companion files, or let it displace work that provably matters, or believe it will move your AI visibility in the near term. Treat it the way you would treat any speculative micro-optimisation: low cost, low priority, no expectations. The moment it starts eating hours that your authority-building or content work needs, you have mispriced it.

What actually moves AI visibility — and provably does

If you want to spend an hour improving how AI engines see you, here is where that hour returns far more than llms.txt will. Make sure your important content is in clean, crawlable HTML that a model can actually read — content buried in heavy JavaScript or locked behind interactions is invisible to many retrieval systems, and fixing that is a real, measurable win. Add the structured data that engines demonstrably use: Organization schema so your identity is clear, Article and FAQPage schema so your content and its answers are machine-readable. Write pages that answer specific questions directly, because that is what gets extracted into answers. And build the genuine authority — relevant links, consistent mentions, a clear entity — that determines whether a model trusts you enough to cite you at all. Every one of these has an understood mechanism and observable effect. llms.txt has neither yet.

The useful way to think about emerging standards

There is a healthy posture for all of this, because llms.txt will not be the last proposed AI-SEO standard to go viral. Adopt cheap, harmless, forward-looking things when they cost you almost nothing — there is real option value in being early if a standard takes off. But never let a speculative file crowd out the fundamentals, and never pay for a tool or a service whose entire pitch is a standard no engine has committed to. The brands that win the AI-visibility game are not the ones who chased every file and tag the moment it trended; they are the ones who got genuinely useful, genuinely trusted, and genuinely readable, and who added the speculative extras as a free afterthought. Add your llms.txt if it is quick. Then go do the work that actually gets you named.

How to write one properly, if you are going to

If you have decided it passes the twenty-minute test, do it well, because a sloppy file is worse than none. Keep it to the content that genuinely represents you: your core pages, your best resources, and a one-line description of each in plain language — the same clear, repeatable descriptions that help your entity everywhere else. Resist the urge to dump your entire sitemap into it; the whole point of the proposal is curation, so a file that lists two thousand URLs defeats its own purpose and signals nothing. Keep it accurate and keep it current, because a file that describes pages you have since deleted or rewritten is a small liability if any system does start reading it. In other words, treat it with the same honesty you would treat your homepage: it is a public statement of what you are and what matters on your site, and the discipline of writing it clearly is worth a little even if no model ever reads it, because it forces you to decide what your most important content actually is.

The quiet upside nobody mentions

There is one benefit of the llms.txt exercise that is real regardless of adoption, and it is the reason we do not dismiss the file entirely. Sitting down to write an honest, curated list of your most important pages, each with a crisp one-line description, is a genuinely clarifying act. Most teams discover in the process that they cannot cleanly describe half their pages, or that their “most important” content is a muddle of overlapping posts that should be consolidated, or that the single sentence describing their business is different on every page. Those discoveries are worth far more than the file itself. If writing your llms.txt prompts you to tighten your entity description, prune some thin pages, and decide what your site is really about, you will have improved your actual AI visibility — not because a model read the file, but because the thinking behind it made your whole site clearer. That is the paradox worth ending on: the most valuable thing about llms.txt today is not the file. It is the clarity the exercise forces, and that clarity is something the engines genuinely do reward.

How it differs from robots.txt and sitemaps

It helps to place llms.txt against the files people already know. Robots.txt tells crawlers where they may and may not go; an XML sitemap tells them what exists and where to find it. Llms.txt is a different idea again: a curated, human-readable guide that points language models to the content you most want them to understand, often with brief context about what each resource is. Where a sitemap is exhaustive and mechanical, llms.txt is selective and editorial — a shortlist of your best, most useful material presented in a form a model can parse easily. Understanding that distinction matters, because it tells you what the file is for: not to control access, but to curate attention toward the content you would most want an AI to rely on.

Is it actually respected yet?

The honest answer, and the one worth leading with, is that adoption by the major AI platforms is far from universal or guaranteed. Llms.txt is a proposed convention rather than a ratified standard that every engine honours, and the degree to which any given model actively reads and uses it varies and is not always transparent. This is the single most important thing to understand before investing: adding the file does not guarantee that any engine will use it, and claims to the contrary outrun the evidence. That does not make it worthless — conventions can gain traction — but it does mean you should treat it as a low-cost, speculative optimisation rather than a proven lever, and size your investment accordingly.

What a good llms.txt contains

If you do add one, the quality of what goes in it matters more than its mere existence. A useful llms.txt is curated, not dumped: it points to your genuinely important, high-value content — the definitive pages, the key documentation, the resources you would most want an AI to understand and cite — rather than listing everything. Brief, accurate context for each resource helps a model understand what it is looking at. The discipline is editorial: you are answering the question “if a model could only understand a handful of things about us, what should they be?” A thoughtful, well-maintained file that genuinely represents your best material is the only version worth making; a sprawling, neglected one offers nothing a sitemap does not.

The honest cost-benefit

The case for adding llms.txt rests on an asymmetry: it is cheap to create and maintain, and if the convention gains adoption you are already positioned to benefit. The case against is opportunity cost and false confidence — time spent perfecting a file that may not be read is time not spent on the content quality and authority that unquestionably drive AI visibility. The resolution is proportion. For most sites, a simple, accurate llms.txt is worth the modest effort as a hedge, provided it does not become a distraction from the fundamentals. What it should never be is a centrepiece of strategy or a substitute for the harder work. Treat it as a cheap bet with an uncertain payoff, not as a solution.

Who should prioritise it, and who shouldn't

The file makes most sense for sites with substantial, well-organised, genuinely valuable content — documentation-heavy products, deep resource libraries, reference material — where curating a model's attention toward the best resources has real potential upside. It makes least sense as an early priority for sites whose more pressing problem is that their content is thin, their authority unestablished, or their entity unclear to the engines in the first place. For those sites, pointing a model at a short list of mediocre pages changes nothing. The rule of thumb: if your content is strong and plentiful enough that curation would genuinely help, consider it; if your content is the bottleneck, fix that first and come back to llms.txt later.

What matters far more

It would be a mistake to let llms.txt crowd out the things that actually determine whether AI engines cite you. No file changes the fundamentals: genuinely valuable, distinctive content; clear structure that makes your answers extractable; established authority and a well-understood entity; accuracy and freshness. An engine that encounters your best content through normal crawling and retrieval will weigh it on those merits whether or not an llms.txt points at it, and no amount of curation compensates for content that is not worth citing. So the file belongs firmly in the “nice hedge” category, several tiers below the work that reliably moves AI visibility. Add it if you like, but never let it feel like progress on the things that actually matter.

How to add one without over-investing

If you have decided it is worth a modest bet, keep the implementation lightweight. Create a concise file at your root that points to your genuinely best and most useful resources, with a line of accurate context for each, and keep it current as your key content changes. Resist the urge to turn it into a sprawling project or to agonise over format details that no engine has standardised. The whole point is that it should be cheap — an afternoon, not a quarter — precisely because its payoff is uncertain. Set it up, keep it honest and maintained, and then return your attention to the content and authority work that does the real lifting. A small, accurate, maintained file is the entire ambition.

Where the standard might go

It is reasonable to add llms.txt partly as a bet on the future rather than the present. Conventions sometimes start as proposals that few honour and become expectations that most do, and if llms.txt follows that path, sites that adopted it early and maintained it well will be positioned to benefit the moment adoption broadens. But that is a possibility, not a promise, and the standard could equally stall or be superseded. The sensible posture is to hold it lightly: adopt it cheaply as an option on a future that may or may not arrive, keep an eye on whether the major engines begin to honour it meaningfully, and be ready to invest more only if and when the evidence of real adoption actually materialises.

Frequently asked questions

What is llms.txt?

A proposed plain-text (Markdown) file placed at your site root that lists and briefly describes your most important pages, intended to give AI models a clean, curated map of your site — conceptually a 'sitemap for LLMs.'

Do ChatGPT, Gemini or Perplexity actually read llms.txt?

As of now, no major AI engine has publicly confirmed it reads llms.txt as part of how it retrieves or cites sources. For robots.txt and sitemap.xml to matter, search engines had to agree to honour them; that explicit commitment doesn't yet exist for llms.txt.

Should I add an llms.txt file?

Only if it's genuinely a quick job, and treat it as a cheap bet on a possible future, not a priority. The downside is near zero, so if you can write an accurate one in half an hour, do it — but don't let it displace work that provably moves AI visibility.

What actually improves how AI engines see my site?

Clean, crawlable HTML a model can read; structured data engines demonstrably use (Organization, Article, FAQPage schema); pages that answer specific questions directly; and genuine authority from relevant links and consistent mentions. Each has a known mechanism and observable effect.

Is there any real benefit to writing one today?

Yes, indirectly. The exercise of curating your most important pages with crisp one-line descriptions forces clarity — often exposing thin pages to consolidate and an inconsistent entity description to fix. That clarity helps your AI visibility whether or not any model reads the file.