llms.txt
llms.txt is a proposed Markdown file at /llms.txt that gives language models a curated map of a site. Jeremy Howard proposed it in September 2024. As of September 2026, Google says Google Search does not use it, no other major answer engine has documented using it, and an Ahrefs study of 137,210 sites found 97% of published llms.txt files received no requests in May 2026. It is harmless to publish but not a ranking or citation lever.
What the proposal says
The llmstxt.org proposal suggests a Markdown file with an H1 name, a short blockquote summary and sections of links, plus optional Markdown copies of pages, to help LLMs use a site at inference time.[1]
Adoption status (September 2026)
| Engine | Documented position |
|---|---|
| Its AI guide lists llms.txt and special Markdown files as not used by Google Search: fine to keep for other purposes, but they neither help nor hurt.[2] “You don’t need to create new machine readable files, AI text files.”[3] Vendor-documented | |
| OpenAI, Anthropic, Perplexity, Microsoft | No crawler or publisher documentation mentions llms.txt.[4][5][6][7] |
Some developer tools and documentation platforms read or generate llms.txt files, which is its most practical use today. Practitioner practice
Do AI bots read llms.txt?
Rarely. Ahrefs checked all 137,210 domains in its Web Analytics that had traffic in May 2026: 28% published an llms.txt file (an upper bound, since the sample skews toward SEO-aware sites), and 97% of those files received zero requests. Of the requests that did arrive, 96% came from bots and only 1.1% from AI retrieval bots such as OAI-SearchBot and PerplexityBot. No AI bot requested llms.txt on sites that don’t have one.[8] Independent study
Should you publish one?
- Yes, if you run developer docs or want a cheap, curated index for AI tools that do read it.
- Don’t expect more AI citations or traffic; nothing documents that effect, and most files are never fetched.[8]
- Never put content in it that contradicts your pages.
Template
# Example Plumbing Co.
> Licensed plumbers serving Springfield since 2009.
## Services
- [Water heaters](https://example.com/water-heaters): repair and replacement
- [Drain cleaning](https://example.com/drains)
## About
- [Licenses and insurance](https://example.com/about)This site publishes an example at /llms.txt. It lists every page and notes the file’s status.
Frequently asked questions
Does Google use llms.txt?
No. Google's documentation says Google Search doesn't use llms.txt.
Is llms.txt like robots.txt?
No. robots.txt is a standard (RFC 9309) for crawl permissions. llms.txt is an unofficial proposal for a content map and controls nothing.
Can llms.txt hurt my site?
Google says it neither helps nor hurts in Google Search.
Do AI crawlers actually read llms.txt?
Rarely. Ahrefs found 97% of llms.txt files on 137,210 sites received zero requests in May 2026, and AI retrieval bots made 1.1% of the requests that did arrive.
See also
References
Pages accessed September 28–30, 2026 unless a date is given. See all sources and our editorial policy.
- ^ "The /llms.txt file (proposal by Jeremy Howard)". llmstxt.org. Proposed September 3, 2024.
- ^ "Optimizing your website for generative AI features on Google Search". Google Search Central. Added May 15, 2026; last updated July 10, 2026.
- ^ "AI features and your website". Google Search Central. Last updated December 10, 2025.
- ^ "Overview of OpenAI Crawlers". OpenAI Platform documentation.
- ^ "Does Anthropic crawl data from the web, and how can site owners block the crawler?". Anthropic (Claude Help Center).
- ^ "Perplexity Crawlers". Perplexity documentation.
- ^ "Bing Webmaster Guidelines". Microsoft Bing Webmaster Tools.
- ^ "We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read (Louise Linehan)". Ahrefs. Published June 15, 2026; May 2026 data. Vendor study.