What is llms.txt?
llms.txt is a proposed text file, written in Markdown, that a website places at /llms.txt to give AI agents a short guide to the site: what it is, and which pages matter most. The idea is that a language model reading the file can find the right page quickly instead of crawling and parsing full HTML.
The proposal was published by Jeremy Howard of Answer.AI on September 3, 2024 [1], and the specification is maintained in an open GitHub repository [7]. llms.txt is a community proposal. It is not a web standard, and no search engine or AI company is obliged to read it.
The file was designed mainly for use at inference time, when an agent is answering a question, rather than for training. The proposal states: "Our expectation was that llms.txt would mainly be useful for inference rather than training, and that is how it has been used" [1].
What does an llms.txt file look like?
An llms.txt file is a Markdown document with one required element: an H1 heading naming the site or project [1]. Everything else is optional: a blockquote summary, short notes, and H2 sections containing lists of links, each with an optional description after a colon [1].
# Example Plumbing Co
> Licensed plumbers serving Denver since 2009. Emergency call-outs,
> water heaters and drain repair.
## Services
- [Emergency plumbing](https://example.com/emergency.md): 24/7 call-outs, response times, pricing
- [Water heater repair](https://example.com/water-heaters.md): tank and tankless, brands serviced
## Company
- [About us](https://example.com/about.md): license numbers, service areaThe proposal also asks sites to offer clean Markdown versions of key pages, either by appending .md to the URL (page.html.md) or, since v2, by swapping the extension (page.md) [1][2].
Does Google use llms.txt?
No. Google does not use llms.txt for Search, AI Overviews or AI Mode, and has said it has no plans to. At Google's Search Central Deep Dive event in July 2025, Gary Illyes said Google doesn't support llms.txt "and isn't planning to" [4].
Google's own documentation is explicit: "You don't need to create new machine readable files, AI text files, or markup to appear in these features," referring to AI Overviews and AI Mode [5]. A page needs to be indexed and eligible to show with a snippet. An llms.txt file changes neither.
John Mueller of Google compared llms.txt to the keywords meta tag in April 2025, noting that "none of the AI services have said they're using LLMs.TXT (and you can tell when you look at your server logs that they don't even check for it)" [3].
Do ChatGPT, Claude or Perplexity read llms.txt?
No major AI answer engine has announced that it reads llms.txt to decide which sites to cite. Mueller's April 2025 observation was that AI services had made no such statement and that server logs showed them not even requesting the file [3]. Search Insider found no announcement since then that changes this.
That is different from AI companies publishing llms.txt files. OpenAI, Anthropic and Google each publish one for their developer documentation, according to the llms.txt proposal site [1]. Publishing a file so that coding agents can read your API docs is not the same as using other sites' files to rank or cite them. Many guides blur the two and present the publishing as proof of adoption.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they are allowed to access, while llms.txt tells an agent what is worth reading. The proposal draws the line itself: robots.txt "lets automated tools know what access to a site is considered acceptable," while llms.txt information "is instead used on demand, when an agent needs information" [1].
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Purpose | Access rules for crawlers | Full list of indexable URLs | Curated summary for AI agents |
| Format | Plain-text directives | XML | Markdown |
| Supported by Google Search | Yes | Yes | No [4] |
| Can block an AI crawler | Yes, if the crawler honors it | No | No |
| Typical size | Short | Every URL | A handful of key pages |
An llms.txt file cannot block anything. To keep AI crawlers out, use robots.txt rules for the specific crawler, such as GPTBot or Google-Extended.
When was llms.txt proposed, and how has it changed?
llms.txt was proposed on September 3, 2024, and revised as v2 in August 2026 [1][2]. In between, Google said three times, in different ways, that it would not use the file.
September 2024: proposed by Jeremy Howard
The original proposal defined the file at /llms.txt, the H1 requirement, link lists, an "Optional" section for secondary links, and .md copies of pages [1].
April 2025: Google compares it to the keywords meta tag
Mueller said no AI service had announced using llms.txt and raised a further concern: a separate file for AI could show different content to machines than to people, which he likened to cloaking [3].
July 2025: Google says it won't support it
Gary Illyes said at Search Central Deep Dive in the Asia Pacific region that Google does not support llms.txt and is not planning to [4].
December 2025: Google's AI features guide
Google's documentation for site owners, in the version last updated December 10, 2025, says no AI text files are needed to appear in AI Overviews or AI Mode [5].
June 2026: "I have the best website ever"
On Google's Search Off the Record podcast, Mueller argued that a self-written file cannot help a model choose between sites, because "you're telling these systems, like, I have the best website ever." He allowed one use: "If someone is already on your website, maybe some kind of automated system is helpful" [6].
August 2026: v2
Version 2 added standard link relations so agents can find the files: rel="alternate" type="text/markdown" points to a page's Markdown copy and rel="describedby" points to the relevant llms.txt, either as HTML <link> elements or HTTP headers [2]. It also defined that a file in a subpath covers the pages under that path, with the most specific file applying, and removed the technical meaning of the "Optional" section, which is now only a convention [2]. Guides written before August 2026 still describe "Optional" as a section agents are told to skip.
Does llms.txt help SEO or AI citations?
No published evidence shows llms.txt improving rankings or AI citations. Google does not read it [4], and no AI answer engine has said it uses the file to choose sources [3]. Any claim that adding llms.txt raised a site's AI visibility is anecdotal unless it isolates the file from everything else that changed.
Mueller's June 2026 comment points to the one place the file plausibly helps: after an agent has already reached your site and needs to find the right page, such as a coding assistant reading API documentation [6]. That is a usability benefit for agents, not a discovery or ranking benefit.
How do you create an llms.txt file?
To create an llms.txt file, write a Markdown file with an H1 naming the site, an optional one-paragraph summary in a blockquote, and H2 sections listing your most useful pages as links with one-line descriptions [1]. Save it as llms.txt and upload it to the site root so it loads at https://yourdomain.com/llms.txt.
Keep the file honest and short. List pages that answer real questions (documentation, pricing, policies, product specs), not every blog post. The content must match what the pages say. A file that tells AI systems something different from what people see is the cloaking risk Mueller described [3].
Should you add an llms.txt file?
Add an llms.txt file if your site has documentation, an API or a product catalog that AI agents and coding assistants work with directly, because the file can help an agent already on your site find the right page [6]. It costs little and does no harm.
Do not add one expecting more traffic, rankings or AI Overview citations. Google has said it will not use the file [4] and that no AI text files are needed [5]. Time spent on the pages themselves, answering the questions people ask in clear, quotable sections, does more for AI visibility than any file pointing at them.