llms.txt
A plain-text file at your domain root that tells language models what your site is about and which pages matter most.
llms.txt is a proposed standard for a Markdown file hosted at the root of a website (e.g., example.com/llms.txt) that provides a concise, curated overview of the site's most important content specifically for large language models. Unlike sitemaps which list all pages, llms.txt offers a selective summary with brief descriptions and links to key resources—formatted to fit within an LLM's limited context window. The goal is to help AI assistants quickly understand what a site offers and access its most relevant documentation or content at inference time.
LLMs have limited context windows and cannot efficiently crawl entire websites during inference. llms.txt solves this by providing a pre-curated briefing document that gives AI assistants immediate access to essential information about your site. This is especially valuable when users ask chatbots questions about your product, library, or service—the llms.txt file helps ensure accurate, well-informed responses rather than hallucinations or outdated information.
- Developer DocumentationHelp AI coding assistants understand your library or API by linking to key docs and examples
- Product InformationGive AI chatbots accurate context about your product when users ask questions
- Curated Content AccessProvide a selective summary of your most important pages, not an exhaustive list
- Framework & Library SupportEnable AI tools to provide accurate code suggestions and answers about your technology
- Context Window OptimizationStructure content to fit within LLM token limits while covering essential information
Effective llms.txt files use clean Markdown formatting with a clear title (H1), an optional blockquote description, and organized sections with descriptive links. Each link should include a brief explanation of what the page contains. Use an 'Optional' section for secondary content that can be skipped if context is limited. Keep the file concise—prioritize quality over quantity to fit within typical LLM context windows.
Success of llms.txt implementation can be measured by monitoring AI assistant accuracy when answering questions about your site, tracking llms.txt file requests in server logs, and gathering user feedback on AI-generated responses related to your content.
LLMs interpret llms.txt as a curated briefing document that provides essential context about a website. It's designed for inference time—when a user asks an AI assistant about a topic, the assistant can fetch and include the relevant llms.txt content to provide accurate, up-to-date information within its context window.
Knowing the term is step one.
Geordy operationalizes every term in this glossary - generating the structured files AI engines actually read.