Plain definition
Definition of llms.txt
A proposed Markdown index file hosted at /llms.txt that gives large language models and AI coding tools a concise, curated map of a website's most useful pages.
# Site Name | > Short summary | ## Docs | - [Title](https://example.com/docs.md): One-line note | ## OptionalWorked example
Proposed in September 2024 by Jeremy Howard and documented at llmstxt.org, the /llms.txt convention solves a practical context-window problem: full HTML pages contain navigation bars, scripts, and layout markup that waste tokens when an AI assistant or agent tries to read documentation. The specification uses a strict, easy-to-parse Markdown layout starting with a single H1 project name (the only required element), followed by a blockquote summary, H2 sections containing bulleted links with short descriptions, and an optional "## Optional" section for lower-priority links.
Say a made-up webhook relay called HookBridge publishes /llms.txt listing its quickstart, authentication guide, API reference, and pricing page, alongside /llms-full.txt combining its full reference into one plain Markdown file. When a developer pastes HookBridge's docs URL into an AI coding assistant that supports llms.txt, the tool reads a clean summary of endpoints and payload rules instead of scraping cluttered marketing layouts.
Why it matters when you launch
If you build a developer tool, API, or SaaS with technical documentation, publishing a tidy /llms.txt file takes under an hour and makes it much easier for coding assistants and research agents to answer questions accurately about your product. You can generate it from the same data source that builds your sitemap so links never drift out of date.
Be realistic about its scope, though. Unlike RFC 9309 for robots.txt, llms.txt is an open community proposal rather than a formal web standard, and Google has stated that Google Search does not require or use llms.txt for search ranking. Treat it as clean, low-effort documentation infrastructure alongside solid HTML, structured data, and crawlable docs, never as a shortcut that replaces core SEO.
Common mistakes
- Treating llms.txt as a crawler blocker or security file like robots.txt, when llms.txt is strictly a curation guide and cannot restrict bots.
- Writing /llms.txt manually once at launch and letting URLs rot into 404 errors as you rename routes, change pricing, or ship new features.
- Dumping every thin marketing page and legal disclaimer into /llms.txt instead of curating the highest-signal pages and putting secondary links under the Optional section.
- Ignoring fundamentals like clean HTML headings, XML sitemaps, and clear product copy while expecting an llms.txt file alone to win AI search citations.
Tools and guides that use it
Free tool
llms.txt GeneratorCreate an llms.txt file that tells AI assistants what your product is and where the key pages are.Free tool
AI Search Readiness CheckerAudit your startup's visibility for Perplexity, ChatGPT Search, and Claude with an actionable 10-point GEO checklist.Founder guide
How to Get Your App Found in ChatGPT, Perplexity and Google AI SearchWhat AI search actually reads, the crawler settings that quietly block you, and the pages that make assistants mention your product, without tricks or guarantees.Founder guide
The Next.js Technical SEO Launch Checklist (2026)What to check before and after launch day: crawlability, metadata, canonical URLs, structured data, sitemaps, internal links, Core Web Vitals and AI crawlers.Related glossary terms
Official sources
Pritam Ghosh runs Hauntboard and maintains this glossary.
Published · Updated
Building something real? Launch it in public.
A person reviews every launch on Hauntboard, and every view and visit it sends your site is published on your launch page. Free to submit.