---
title: "What is llms.txt? | daily.dev Ads glossary"
url: https://business.daily.dev/glossary/llms-txt/
description: "llms.txt is a plain-text file at a site's root, like robots.txt, that tells AI assistants and agents what the site is and which pages matter, linking to markdown versions. It gives language models a clean map instead of the HTML."
lastUpdated: "2026-10-20"
llmsTxt: https://business.daily.dev/llms.txt
---

## What is llms.txt?

llms.txt is a plain-text file at a site's root, like robots.txt, that tells AI assistants and agents what the site is and which pages matter, linking to markdown versions. It gives language models a clean map instead of the HTML.

The format was proposed by Jeremy Howard in 2024: a Markdown file at /llms.txt with the site's name, a short summary, and sections of links, each with a one-line description. Some sites add llms-full.txt with the full content of every page in one file. The companion practice is serving each page as markdown at the same path with a .md suffix, so an agent that follows a link gets text, not a web page.

Robots.txt says what crawlers may fetch; a sitemap lists every URL; llms.txt says what matters and why, in prose a model can use. It is read by some AI crawlers and by many agents and coding tools when a developer points them at a site, which is why documentation sites adopted it first.

Evidence that it changes citations is still mixed, and it costs little: keep it factual, keep it current, and never put claims in it that the pages do not make. Treat it as documentation for machines, with the same honesty as documentation for people.

## What does an llms.txt file look like?

A Markdown file at the site root. An H1 with the site's name; a blockquote with a one-paragraph summary; optional prose with the facts a model should know; then H2 sections of links, each link followed by a colon and a one-line description. An optional section marked "Optional" lists pages a model can skip when context is short. Sites that want to go further publish llms-full.txt, the full text of every page in one file, and serve each page as Markdown at its own URL with a .md suffix.

Keep it to the pages that answer questions: what the product does, how buying works, pricing if public, docs, the main guides, and contact. A file that lists every URL is a sitemap with extra steps.

## Does llms.txt actually improve how AI assistants cite you?

The evidence is mixed and the file is cheap, which is the whole case. Some AI crawlers fetch it; coding agents and assistants read it when a developer points them at a site; the large chat assistants mostly still read the HTML. Nobody has shown a ranking effect in answers from the file alone.

What does move citations is the content it points to: pages that answer a real question in the first paragraph, with specifics, comparisons and a visible date. Publish llms.txt as the map, and spend the effort on the pages. Then measure: assistant referrals in analytics and prompt tracking in a tool built for it.

## How should a developer-marketing site use llms.txt?

- **Describe the offer in plain facts:** what you sell, to whom, and how it is bought.

- **Link the pages buyers ask about:** products, how pricing works, audience data, comparisons, [customer stories](https://business.daily.dev/customers.md).

- **Serve Markdown twins** of those pages so an agent that follows a link gets clean text.

- **Keep it current:** a stale llms.txt teaches agents facts your site no longer makes.

- **Never put a claim in it that the pages do not make.** It is documentation for machines, and it is quoted.

The file is also a useful audit: if you cannot write a one-line honest description of a page, the page probably needs rewriting for people too.

## How do llms.txt, Markdown routes and robots.txt fit together?

robots.txt says which crawlers may fetch what, and increasingly carries content-use signals for AI training versus search. The sitemap lists every URL for indexing. llms.txt says what the site is and which pages matter, in prose. Markdown routes give each of those pages a clean text version. Together they let an agent discover, read and quote a site without parsing navigation, cookie banners and scripts.

**Common mistakes**

- Listing every URL instead of the pages that answer questions.
- Claims in llms.txt that the pages do not make.
- Publishing it once and never updating it.
- Treating it as a ranking lever and skipping the content work.

**How developers experience it**

- They point a coding agent at your docs and it reads llms.txt first.
- They ask an assistant what you do and get back whatever your pages say plainly.
- They notice when the Markdown version is cleaner than the web page.

## On daily.dev

business.daily.dev publishes one at /llms.txt, with every product page, guide and [audience page](https://business.daily.dev/audience.md) available in markdown. Developers evaluating tools increasingly ask an agent first; the file makes sure the agent reads the facts you wrote.

## llms.txt vs robots.txt vs sitemap.xml vs Markdown routes

| | llms.txt | robots.txt | sitemap.xml | Markdown routes |
| --- | --- | --- | --- | --- |
| **Says** | What the site is and which pages matter | What crawlers may fetch | Every URL that exists | The page itself, as clean text |
| **Format** | Markdown prose and links | Directives | XML | Markdown at page.md |
| **Read by** | Agents, coding tools, some AI crawlers | All crawlers | Search engines | Agents that follow a link |
| **Effort** | An hour, then upkeep | Minutes | Automatic | A build step per page |
| **Effect on citations** | Unproven on its own | None | Indirect | Cleaner quotes when agents read it |

## Go deeper

- [Agent-readable docs checklist, including llms.txt](https://business.daily.dev/resources/agent-readable-docs-checklist-llms-txt.md)
- [Measuring AI assistant referrals to developer docs](https://business.daily.dev/resources/measure-ai-assistant-referrals-developer-docs.md)
- [This site's llms.txt](https://business.daily.dev/llms.txt)

Related terms: [Answer engine optimization](https://business.daily.dev/glossary/answer-engine-optimization.md) | [Developer experience](https://business.daily.dev/glossary/developer-experience.md) | [Technical content marketing](https://business.daily.dev/glossary/technical-content-marketing.md)


### Frequently asked questions

**Do AI crawlers actually read llms.txt?**

Some do, and agents and coding assistants read it when a developer points them at your site. Treat it as a low-cost, honest map rather than a guaranteed ranking lever, and measure assistant referrals to see its effect.

**What is the difference between llms.txt, robots.txt and a sitemap?**

Robots.txt sets crawl permissions, a sitemap lists URLs for indexing, and llms.txt describes the site and its important pages in prose for language models, with links to clean text versions.

**What should go in an llms.txt file?**

A one-line description of the site, a short summary of what it offers, and grouped links to the pages that answer real questions: product pages, pricing or how-buying-works, docs, guides, and contact. Facts only, kept current.

**What is llms-full.txt?**

A companion file with the full text of every important page in one Markdown document, for agents that want to load the whole site at once. Useful for documentation sites; optional for marketing sites.

**Should every page have a Markdown version?**

Every page an agent might quote: products, pricing, docs, guides and comparisons. Serving them at the same path with a .md suffix is the convention this site follows.

[All terms](https://business.daily.dev/glossary.md) | [Explore your options](https://business.daily.dev/schedule/?ref=glossary-llms-txt) | [How pricing works](https://business.daily.dev/pricing.md)