# llms.txt (/ai/llmstxt)
Related: [Bring your own model](/ai/bring-your-own-model.md), [Contextual menu](/ai/contextual-menu.md), [Markdown export](/ai/markdown-export.md), [Admin Model Context Protocol (MCP) server](/ai/mintlify-mcp.md), [Search Model Context Protocol (MCP) server](/ai/model-context-protocol.md), [skill.md](/ai/skillmd.md)
The [llms.txt file](https://llmstxt.org) is an industry standard that helps large language models (LLMs) index content more efficiently, similar to how a sitemap helps search engines. AI tools can use this file to understand your documentation structure and find content relevant to user queries.
Mintlify automatically hosts an `llms.txt` file at the root of your project that lists all available pages in your documentation. This file is always up to date and requires zero maintenance. You can optionally add a custom `llms.txt` file to the root of your project.
Authentication affects `llms.txt` and `llms-full.txt` differently depending on your site configuration:
* **Fully authenticated sites**: Both files require authentication. AI tools that cannot authenticate cannot access them.
* **Partially authenticated sites**: Both files are publicly accessible, but only list public pages and exclude pages restricted to user groups.
For more information on how authentication affects AI features, see [Feature availability](/deploy/authentication-setup#feature-availability).
View your `llms.txt` by appending `/llms.txt` to your documentation site's URL. Mintlify also hosts the file at `/.well-known/llms.txt` for compatibility with tools that follow the `.well-known` convention.
Open the llms.txt for this site.
Mintlify adds HTTP headers to every response, including [Markdown responses](/ai/markdown-export) and 404 pages, so AI tools can discover your `llms.txt` files and other agent resources without prior knowledge of their location:
* `Link`: Follows the standard HTTP `Link` header format for resource discovery. Advertises `llms.txt`, `llms-full.txt`, your API catalog, [MCP server card](/ai/model-context-protocol#discovery-endpoint), [agent card](/ai/skillmd#agent-card), and [agent skills index](/ai/skillmd#skills-discovery-endpoints).
* `X-Llms-Txt`: A convenience header for tools that check for `llms.txt` support.
```http title="Response headers"
Link: ; rel="llms-txt", ; rel="llms-full-txt", ; rel="api-catalog", ; rel="mcp-server-card", ; rel="agent-card", ; rel="agent-skills"
X-Llms-Txt: /llms.txt
```
Sites that use authentication also advertise their OAuth protected resource metadata with `rel="oauth-protected-resource"`. If you serve your documentation under a base path, every advertised path includes that prefix.
## Structure of the llms.txt file [#structure-of-the-llmstxt-file]
An `llms.txt` file is a plain Markdown file that contains:
* **Site title** as an H1 heading.
* **Site description** as a blockquote summary below the title, sourced from the `description` field in your `docs.json` configuration.
* **Custom agent instructions** as an `Agent Instructions` block after the description, if you set [`markdown.instructions`](/ai/markdown-export#custom-agent-instructions) in your `docs.json`.
* **Structured content sections** with links and a description of each page in your documentation.
* **API specification links** to your OpenAPI and AsyncAPI specs, if your documentation includes them.
* **External links** in an `Optional` section, for any absolute URLs in your navigation.
The `llms.txt` file lists your pages in the order they appear in your `docs.json` navigation. Pages that aren't in your navigation but that `seo.indexing: "all"` includes appear last, in alphabetical order. Page links in the `llms.txt` file include a `.md` extension so AI tools can fetch the Markdown version of each page directly.
Each page's description comes from the `description` field in its frontmatter. Mintlify uses the first paragraph of the description and truncates it at 300 characters. Pages without a `description` field appear in the `llms.txt` file without a description.
Both `llms.txt` and `llms-full.txt` list pages from your default language and default version. They exclude [hidden pages](/organize/hidden-pages) and pages with `noindex: true` in their frontmatter unless you set `seo.indexing: "all"` in your `docs.json`.
This structured approach allows LLMs to efficiently process your documentation at a high level and locate relevant content for user queries, improving the accuracy and speed of AI-assisted documentation searches.
## Split indexes under `/_llms` [#split-indexes-under-_llms]
Automatically generated `llms.txt` index files cannot exceed 100,000 characters. When an index is larger than this limit, Mintlify keeps the main `llms.txt` file as a directory and moves groups of page links into generated Markdown files under `/_llms/`. Every page stays discoverable: splitting never omits pages from the index. This limit does not apply to `llms-full.txt`.
Links to generated indexes appear in an `Indexes` section. Each link includes the group's breadcrumb, its page count, and a description:
```mdx
> The links below point to documentation indexes. Follow each `/_llms/` index recursively until you reach documentation pages.
## Indexes
- [API reference (250 pages)](https://docs.example.com/_llms/api-reference.md): Endpoint documentation for the example API
```
A generated index can link to further indexes. For example, `/_llms/api-reference.md` can link to `/_llms/api-reference/admin.md` when a group is too large for a single file. Agents should follow these index links recursively until they reach documentation page links. Mintlify may shorten page descriptions in a split index to keep each file under the character limit.
The generated files are part of `llms.txt` and do not need to exist in your repository. They are separate from `llms-full.txt`.
The `/_llms/` route uses the same base path as your documentation:
* A root-hosted site serves an index at `https://docs.example.com/_llms/api-reference.md`.
* A site hosted at `/docs` serves it at `https://example.com/docs/_llms/api-reference.md`.
If you use a reverse proxy or path allowlist, forward the generated route in addition to `llms.txt`. A broad `/*` rule already includes `/_llms/*`. With granular rules, add `/_llms/*` explicitly. See [Reverse proxy](/deploy/reverse-proxy#routing-configuration) for routing guidance.
A directory such as `/docs` in your repository does not set your site's public base path. Configure the base path in your Mintlify dashboard and use the same prefix in your reverse proxy. Otherwise, `llms.txt` can be reachable at the public prefix while its generated `/_llms/` links point somewhere else.
```mdx title="Example llms.txt"
# Site title
> A brief description of the documentation site.
## Docs
- [API](https://example.com/docs/api.md): Endpoint list and usage
- [Install](https://example.com/docs/install.md): Setup steps
- [Getting started](https://example.com/docs/start.md): Intro guide
## OpenAPI Specs
- [openapi](https://example.com/docs/openapi.json)
## AsyncAPI Specs
- [asyncapi](https://example.com/docs/asyncapi.yaml)
## Optional
- [Community](https://example.com/community)
```
## llms-full.txt [#llms-fulltxt]
The `llms-full.txt` file combines your entire documentation site into a single file as context for AI tools and LLM indexing. Each page appears as its title, source URL, description, and full Markdown content.
Mintlify automatically hosts an `llms-full.txt` file at the root of your project. View your `llms-full.txt` by appending `/llms-full.txt` to your documentation site's URL. Mintlify also hosts the file at `/.well-known/llms-full.txt` for compatibility with tools that follow the `.well-known` convention.
Open the llms-full.txt for this site.
## Custom files [#custom-files]
To add a custom `llms.txt` or `llms-full.txt` file, create an `llms.txt` or `llms-full.txt` file at the root of your project. Adding a custom file overrides the automatically generated file of the same name. If you delete a custom file, Mintlify restores the automatically generated file.
Your custom `llms.txt` or `llms-full.txt` file must have a site title as an H1 heading. Other content is optional. See [Format](https://llmstxt.org/#format) in the `llms.txt` specification for more information on optional sections and best practices.
On sites that use [authentication](/deploy/authentication-setup), custom `llms.txt` and `llms-full.txt` respect authentication.
If a custom file links to any page that isn't public, only authenticated users can access the file. Anonymous visitors receive the automatically generated `llms.txt` or `llms-full.txt`, which only include publicly available links.