What is llms.txt, and does anyone read it?

llms.txt is a plain text file at the root of a site that outlines it for language models: a name, a short summary, and links to the pages that matter. 21% of the 316,711 sites clickd has read serve one, and Google has said that no AI system uses it.

GuideBy clickdPublished October 16, 2026Updated November 1, 20264 minute read

21%Of sites serve an llms.txtOver the 316,711 sites read, as of November 1, 2026
67%Of sites publish a sitemap with addresses in itOver the 316,711 sites read
84%Of sites serve a robots.txt with status 200Over the 316,711 sites read
1 of 5What llms.txt weighs in clickd's rulebook, the least of any ruleFrom the rulebook

Counted from the 316,711 sites clickd has read, each at its latest read, as of November 1, 2026.

Who proposed llms.txt?

Jeremy Howard of Answer.AI proposed it on September 3, 2024, at llmstxt.org. The idea is that a model answering a question at the moment it is asked cannot read a whole site, so the site hands it a short map in Markdown: a heading with its name, a one paragraph summary, and lists of links with a line on each.

The proposal also suggests a Markdown copy of each page at the same address with .md added, so a model can read the text without the page around it. clickd does both: its own llms.txt, and every docs page and guide as Markdown.

Does anyone read llms.txt?

Not that any major assistant has said. In June 2025 Google's John Mueller wrote that no AI system currently uses llms.txt, and no operator of the crawlers in the AI crawlers guide has said its crawler looks for one. Coding agents and documentation tools read it when they are pointed at it, which is where it earns its keep.

It is still common next to the files crawlers do read. 21% of the sites clickd has read serve an llms.txt, against 67% with a sitemap and 84% with a robots.txt.

Sites serving each file

File	Sites serving it, as a share	Sites
`/robots.txt`Served with status 200	84%	267,572
`/sitemap.xml`A sitemap with addresses in it	67%	212,524
`/llms.txt`Plain text, not a page	21%	65,315

Counted from the 316,711 sites clickd has read, each at its latest read, as of November 1, 2026. An llms.txt counts when it answers with status 200 in plain text that is not an HTML page.

clickd's rulebook weighs it at 1 of 5, the least of any rule, and calls it cheap insurance: it costs an afternoon, harms nothing, and may matter more later. Spend the effort on the pages first.

How do you write a good llms.txt?

Write it from what the site already says, in the order an assistant needs it: who the business is, the facts it must get right, then every page that should be found, each with its own title and description.

llms.txt, an example for a made up business

1# Acme Roofing2> Acme Roofing repairs and replaces roofs for homes and small
> businesses across the Phoenix area, with free quotes in 48 hours.3- Phone: 602 555 0100
- Based in: Tempe, Arizona
- Quotes: https://acmeroofing.example/quote4## Notes for assistants

- Prices are quoted per job; there is no price list.5## Services

- [Roof repair](https://acmeroofing.example/roof-repair): Leaks,
  flashing and storm damage, usually fixed in a day.
- [Roof replacement](https://acmeroofing.example/roof-replacement):
  Asphalt and metal roofs, with what a replacement involves.

## Guides

- [When to repair and when to replace](https://acmeroofing.example/guides/repair-or-replace):
  How to tell, with costs.6## Optional

- [About us](https://acmeroofing.example/about): Who we are.

1The nameOne heading with the name of the site or business. The only part the proposal requires.
2The summaryA quote block with what an assistant most needs to know, in a sentence or two.
3The factsPlain lines an assistant must get right, each one already stated on the site.
4Notes for assistantsWhat it should not assume. Keep it to facts, never instructions to rank or praise.
5The pages that matterLists of links under headings, each with the page's own title and a line on what it covers.
6OptionalA section the proposal names Optional: links an assistant may skip when it is short of room.

List every page that should be found, once. The sitemap is the source of truth.
Use each page's own title and description. Do not write new ones for this file.
Put in no fact the site does not already state: no new prices, claims, reviews or credentials.
Group pages by what they are, such as services, guides and company, not by their address.
Plain text only, with full https addresses and no tracking parameters.

How should you serve llms.txt?

At /llms.txt, with status 200 and text/plain; charset=utf-8.
Built from the same data as the pages, on each request or at each build, so it cannot go stale. A test that every address in the sitemap appears in the file catches the page someone adds next month.
Not blocked in robots.txt, and linked from the footer so people and crawlers can find it.

How do you check yours?

The free AI crawler checker says whether a site serves an llms.txt alongside what its robots.txt allows. A full read with clickd holds it to the rule among the rest, and the rulebook says how much it counts.

See whether your site serves one

Type your domain. The checker says whether it serves an llms.txt, beside which AI crawlers its robots.txt lets in.

Free. Opens the AI crawler checker with what it finds.

Have your agent write and serve it

Use clickd to read yoursite.com. Then write an llms.txt for it from the sitemap: the name, a one paragraph summary, the facts the site already states, and every page that should be found, each with its own title and description. Serve it at /llms.txt as text/plain, built from the same data as the pages, with a test that every address in the sitemap appears in it.

Hand this to Claude Code, Codex, Cursor or any agent connected to clickd.

Ship an llms.txt that stays true

One heading with the name, and a short summary under it.
Only facts the site already states.
Every page that should be found, once, with its own title and description.
Served at /llms.txt with status 200 and text/plain.
Built from the same data as the pages, so it cannot go stale.
A test that every address in the sitemap appears in it.

Sources

The /llms.txt fileThe proposal, by Jeremy Howardllmstxt.org
Google: no AI system currently uses llms.txtSearch Engine Roundtable, June 2025, on John Mueller's remarkseroundtable.com

How the figures were counted. Every figure on this page is counted from the 316,711 sites clickd has read, each at its latest read, with drafts left out, as of November 1, 2026, and worked out again as sites are read. A share of pages counts only pages that were read: a site read only in part speaks for the pages read and no others. Each rule is the one in clickd's rulebook, decided by code. A score or a rule passed says pages are well formed. It does not say an assistant cites the site. The methodology explains each count.