Quick answer
llms.txt is a plain-text file at a site's root, like robots.txt, that tells AI assistants and agents what the site is and which pages matter, linking to markdown versions. It gives language models a clean map instead of the HTML.
llms.txt
DefinitionThe format was proposed by Jeremy Howard in 2024: a Markdown file at /llms.txt with the site's name, a short summary, and sections of links, each with a one-line description. Some sites add llms-full.txt with the full content of every page in one file. The companion practice is serving each page as markdown at the same path with a .md suffix, so an agent that follows a link gets text, not a web page.
What does an llms.txt file look like?
A Markdown file at the site root. An H1 with the site's name; a blockquote with a one-paragraph summary; optional prose with the facts a model should know; then H2 sections of links, each link followed by a colon and a one-line description. An optional section marked "Optional" lists pages a model can skip when context is short. Sites that want to go further publish llms-full.txt, the full text of every page in one file, and serve each page as Markdown at its own URL with a .md suffix.
Keep it to the pages that answer questions: what the product does, how buying works, pricing if public, docs, the main guides, and contact. A file that lists every URL is a sitemap with extra steps.
Does llms.txt actually improve how AI assistants cite you?
The evidence is mixed and the file is cheap, which is the whole case. Some AI crawlers fetch it; coding agents and assistants read it when a developer points them at a site; the large chat assistants mostly still read the HTML. Nobody has shown a ranking effect in answers from the file alone.
What does move citations is the content it points to: pages that answer a real question in the first paragraph, with specifics, comparisons and a visible date. Publish llms.txt as the map, and spend the effort on the pages. Then measure: assistant referrals in analytics and prompt tracking in a tool built for it.
Common mistakes
- Listing every URL instead of the pages that answer questions.
- Claims in llms.txt that the pages do not make.
- Publishing it once and never updating it.
- Treating it as a ranking lever and skipping the content work.
How developers experience it
- They point a coding agent at your docs and it reads llms.txt first.
- They ask an assistant what you do and get back whatever your pages say plainly.
- They notice when the Markdown version is cleaner than the web page.
How should a developer-marketing site use llms.txt?
- Describe the offer in plain facts: what you sell, to whom, and how it is bought.
- Link the pages buyers ask about: products, how pricing works, audience data, comparisons, customer stories.
- Serve Markdown twins of those pages so an agent that follows a link gets clean text.
- Keep it current: a stale llms.txt teaches agents facts your site no longer makes.
- Never put a claim in it that the pages do not make. It is documentation for machines, and it is quoted.
The file is also a useful audit: if you cannot write a one-line honest description of a page, the page probably needs rewriting for people too.
How do llms.txt, Markdown routes and robots.txt fit together?
robots.txt says which crawlers may fetch what, and increasingly carries content-use signals for AI training versus search. The sitemap lists every URL for indexing. llms.txt says what the site is and which pages matter, in prose. Markdown routes give each of those pages a clean text version. Together they let an agent discover, read and quote a site without parsing navigation, cookie banners and scripts.
On daily.dev
How it works on daily.dev
business.daily.dev publishes one at /llms.txt, with every product page, guide and audience page available in markdown. Developers evaluating tools increasingly ask an agent first; the file makes sure the agent reads the facts you wrote.
Explore my optionsllms.txt vs robots.txt vs sitemap.xml vs Markdown routes
| Capability | llms.txt | robots.txt | sitemap.xml | Markdown routes |
|---|---|---|---|---|
| Says | What the site is and which pages matter | What crawlers may fetch | Every URL that exists | The page itself, as clean text |
| Format | Markdown prose and links | Directives | XML | Markdown at page.md |
| Read by | Agents, coding tools, some AI crawlers | All crawlers | Search engines | Agents that follow a link |
| Effort | An hour, then upkeep | Minutes | Automatic | A build step per page |
| Effect on citations | Unproven on its own | None | Indirect | Cleaner quotes when agents read it |