Definition
llms.txtllms.txt is a proposed standard for a markdown file at a website's root (example.com/llms.txt) that summarizes the site and links to its most useful pages for language models. Jeremy Howard of Answer.AI proposed it in September 2024. Google says its search ignores the file, and no other major AI search engine has said it uses one.
How llms.txt works
A web page built for people carries a lot a language model doesn't need, like navigation menus and cookie banners. llms.txt is meant to cut through that. The site owner writes one markdown file that says what the site is and lists the pages worth reading, with a short note on each.
Jeremy Howard, co-founder of Answer.AI, published the proposal on September 3, 2024, at llmstxt.org. It's a community convention. No standards body such as the W3C or IETF has adopted it.
The idea is that an AI tool can fetch one small file and know where to look, instead of crawling and cleaning dozens of pages to piece the same picture together.
The llms.txt format
The format is deliberately simple. In order, the file contains:
- An H1 with the site or project name. This is the only required section.
- A blockquote with a short summary of what the site is.
- Any other markdown a model needs for context, such as paragraphs or lists.
- H2 sections of links, each a markdown list of
[name](url)entries with an optional note after a colon.
By convention, a section titled "Optional" holds secondary links that a tool can skip when it's short on space.
llms.txt example
Here's a short example for a fictional project management company:
# Acme
> Acme is project management software for creative agencies,
> with built-in time tracking and client approvals.
Plans start at $12 per user per month. All plans include unlimited projects.
## Product
- [Features](https://acme.com/features): Overview of task boards, time tracking, and approvals
- [Pricing](https://acme.com/pricing): Plans, limits, and billing FAQs
- [Integrations](https://acme.com/integrations): Slack, Google Drive, and QuickBooks
## Docs
- [Getting started](https://acme.com/docs/start): Set up a workspace in 10 minutes
- [API reference](https://acme.com/docs/api): REST endpoints and authentication
## Optional
- [Company blog](https://acme.com/blog): Agency operations articles
llms.txt vs llms-full.txt vs robots.txt
These files come up together, but they do different jobs.
| File | What it does | Controls access? |
|---|---|---|
| robots.txt | Tells crawlers which URLs they may fetch | Yes |
| llms.txt | Summarizes the site and links to key pages | No |
| llms-full.txt | Puts the full text of key pages in one file | No |
| XML sitemap | Lists every URL you want crawled | No |
You still block or allow AI crawlers in robots.txt. See AI crawlers for the user agents involved, including GPTBot.
Do AI engines use llms.txt?
The evidence you can check is thin.
- Google says it doesn't. Its guide to optimizing for generative AI features says you don't need AI text files to appear in Google Search, including AI Overviews and AI Mode, and that keeping one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."
- OpenAI, Anthropic, Perplexity, and Microsoft document their crawlers and robots.txt rules, but as of September 2026 we found no public statement from any of them that their AI search products read llms.txt when choosing sources.
- Developer tools are where the file gets real use. Documentation sites such as Anthropic's and Stripe's publish one, so coding assistants (and the developers using them) can load clean docs on request.
So llms.txt helps the tools that go looking for it, and nothing suggests AI search engines treat it as a ranking or citation signal. When an AI engine answers a question it uses retrieval-augmented generation: it searches the web and reads the pages it finds, which means those pages have to be crawlable and clear on their own.
Should you add an llms.txt file?
It's a small job and it does no harm. Documentation sites and developer products have the most to gain, because their readers are the ones most likely to point an AI tool at the file.
For AI search visibility, though, the effort pays off more on the pages themselves. Serve your content in the HTML so crawlers that don't run JavaScript can read it (see server-side rendering), and put a clear answer near the top of each page in a passage that makes sense on its own. Mentions on the third-party sites AI engines cite matter as well, especially when they recommend brands in your category.
Searchable publishes its own file at searchable.com/llms.txt. We also serve markdown versions of our pages directly to AI crawlers, so they get clean text wherever they land. Our dual-served content system explains how that works.
Frequently asked questions
At the root of your domain, so it loads at example.com/llms.txt, the same place robots.txt lives. The proposal also allows one in a subpath, such as example.com/docs/llms.txt, to cover that section of a site.
See how AI engines talk about your brand
Get a free report on where ChatGPT, Perplexity, and Google AI name and cite you.