How to Create llms.txt: Step-by-Step Guide (2026)

I've reviewed the article against all eight criteria. It's well-structured overall. Here are the changes I'm making:

• Factual accuracy: All claims check out. Minor clarification on the "80%" stat (added source context).
• Grammar/style: Tightened several wordy sentences, fixed a few comma splices.
• Structure: Flow is logical; no reordering needed.
• SEO: Keywords are well-distributed. Strengthened a couple of H2/H3 with target terms.
• Duplicates: Removed repeated "not a sitemap" idea (appeared in Step 6 and Mistakes #1), consolidated validation advice that overlapped between two sections.
• Length: All sections fall within range; trimmed the step-by-step slightly where it was running long.
• CTA: Strong and present — minor tightening for urgency.
• Formatting: Clean. Minor fixes to consistency.

---

Up to 80% of search queries now end without a click — and AI-generated answers are a major reason why. If your website isn't structured for large language models, you're invisible in the fastest-growing discovery channel on the internet. Enter llms.txt: a simple file that tells AI exactly what your site is about, what matters most, and where to find it.

Think of it as robots.txt for the AI age — except instead of telling crawlers what to *avoid*, you're telling language models what to *prioritize*. In this guide, you'll learn exactly how to create one, step by step, in under 30 minutes.

What Is llms.txt and Why Does It Matter for AI Search?

The llms.txt standard was proposed by Jeremy Howard — co-founder of fast.ai — in late 2024. It's a Markdown file placed at your website's root (yoursite.com/llms.txt) that provides a structured, AI-readable summary of your site's most important content.

By mid-2025, over 2,000 websites had adopted the standard, including major platforms like Cloudflare, Stripe, Anthropic, and FastAPI. That number has continued to grow as AI-powered search becomes mainstream.

Why the rapid adoption? Because the way people find information is fundamentally shifting:

• ChatGPT Search, Perplexity, and Google AI Overviews are answering questions directly, often without sending users to the source.
• Businesses that don't feed these systems structured information risk being misrepresented — or ignored entirely.
• llms.txt gives you a direct channel to influence what AI knows about your site, your products, and your expertise.

This isn't a speculative future. It's happening now. The emerging discipline of Generative Engine Optimization (GEO) treats llms.txt as foundational — the equivalent of what sitemap.xml became for traditional SEO a decade ago.

llms.txt vs robots.txt: Key Differences Explained

If you're familiar with robots.txt, the concept will feel intuitive — but the purpose is fundamentally different.

| | robots.txt | llms.txt | |---|---|---| | Purpose | Tells crawlers what to *exclude* | Tells LLMs what to *include* | | Format | Custom directive syntax | Simplified Markdown | | Audience | Search engine bots | Language models and AI agents | | Focus | Access control | Content discovery and context | | Companion file | sitemap.xml | llms-full.txt |

The key mindset shift: robots.txt is defensive (keep bots out of certain areas), while llms.txt is proactive (surface your best content for AI consumption). You're not blocking access — you're curating it.

The standard also includes a companion file, llms-full.txt, which contains your complete content in a single Markdown document. Together, the two files serve different needs: the index file for quick context, and the full file for LLMs with large context windows that can ingest everything at once.

How to Create Your llms.txt File Step by Step

The specification uses a deliberately simplified subset of Markdown. Here's exactly how to structure it.

Step 1: Create the File

Create a plain text file named llms.txt in your website's root directory — the same place where robots.txt lives.

your-site.com/llms.txt

Step 2: Add the H1 Title

Start with a single H1 heading — your site or company name:

# Your Company Name

Step 3: Write a Blockquote Summary

Immediately after the title, add a one-to-two-sentence blockquote describing what your site or product does:

> Your Company is a cloud-based project management platform that helps
> remote teams collaborate on complex workflows in real time.

This is the single most important line in your file. LLMs use it to decide whether your content is relevant to a user's query. Be specific and descriptive — avoid marketing fluff.

Step 4: Organize Content into H2 Sections

Group your most important pages under H2 headings. Common sections include:

• Docs — product documentation and guides
• API — API reference and developer resources
• Blog — high-value articles and thought leadership
• Company — about, pricing, and contact information

Step 5: Add Links with Descriptions

Under each H2 section, list your key pages as bullet points using this format:

## Docs

- [Getting Started](https://yoursite.com/docs/getting-started): Step-by-step setup guide for new users
- [Authentication](https://yoursite.com/docs/auth): How to configure SSO, API keys, and OAuth flows
- [Webhooks](https://yoursite.com/docs/webhooks): Setting up real-time event notifications

## API

- [API Reference](https://yoursite.com/api/reference): Complete endpoint documentation with examples
- [Rate Limits](https://yoursite.com/api/rate-limits): Usage tiers and throttling policies

## Blog

- [2025 Product Roadmap](https://yoursite.com/blog/roadmap-2025): Upcoming features and platform direction

Step 6: Keep It Focused

Your llms.txt should stay under approximately 2,000 tokens (roughly 1,500 words). Prioritize pages that answer the questions people actually ask AI assistants:

• What does your product do?
• How much does it cost?
• How do I get started?
• What are the API endpoints?
• What problems does it solve?

Resist the urge to include everything. A focused file with 20–40 curated links outperforms an exhaustive one with hundreds.

Creating llms-full.txt for Extended AI Context

While llms.txt is a concise index with links, llms-full.txt is a complete, self-contained document that concatenates your key content into a single Markdown file. This serves LLMs with large context windows (100K+ tokens) that can ingest entire knowledge bases at once.

Here's how to approach it:

• Start with the same H1 and blockquote from your llms.txt.
• Include the full text of your most important pages — not just links, but the actual content.
• Use clear H2/H3 headings to separate sections so the LLM can navigate the document.
• Strip out navigation elements, footers, and boilerplate — include only the substantive content.
• Keep formatting simple — avoid tables, embedded images, and complex nested lists that can confuse parsers.

A practical starting point: choose your top 10–20 pages by traffic or business value. You can always expand later.

your-site.com/llms-full.txt

Think of llms.txt as the table of contents and llms-full.txt as the book itself. Both matter — provide them together for maximum AI visibility.

How to Validate and Automate Your llms.txt

Creating the file once is straightforward. Keeping it accurate is the real challenge.

Validate Your llms.txt File

Before publishing, check your file against the spec:

• Use a validator such as the one at llmstxt.site to catch formatting errors, structural issues, and broken link syntax.
• Feed your file to an LLM (ChatGPT, Claude, etc.) and ask it to summarize your site. If the summary is inaccurate or incomplete, your file needs work.
• Check for broken links — every URL in your llms.txt should resolve correctly. A malformed file with missing headings or broken links may be silently ignored by AI systems.

Automate llms.txt Updates

A stale llms.txt is worse than none at all — it actively misinforms AI systems. Integrate generation into your workflow:

• CI/CD pipelines: Add a build step that regenerates llms.txt and llms-full.txt from your content source whenever you deploy.
• CMS plugins: WordPress, Next.js, and Hugo all have community plugins that auto-generate and maintain the file.
• Documentation platforms: Mintlify, ReadMe, and GitBook offer built-in llms.txt generation — if you're using one of these, check your settings before building anything custom.
• Scheduled reviews: At minimum, set a monthly calendar reminder to review the file manually.

The goal is simple: your llms.txt should always reflect your current site, not last quarter's version.

Common llms.txt Mistakes and How to Avoid Them

After reviewing hundreds of implementations, these are the pitfalls that keep appearing:

• Using complex Markdown. The spec deliberately restricts formatting to H1, H2, blockquotes, and bullet-point links. Avoid tables, images, nested lists, and HTML. Keep it simple.

• Writing vague descriptions. "Learn more about our product" tells an LLM nothing. Write descriptions that convey specific value: "Complete guide to configuring webhook endpoints with retry logic and payload examples."

• Forgetting llms-full.txt. The index file alone limits what LLMs can learn about your site. Pair it with the full-content companion to maximize coverage.

• Never updating the file. If your llms.txt references pages that no longer exist or omits your newest product, AI systems will serve outdated information. Automate or schedule regular reviews.

• Skipping validation. A malformed file — missing the H1, using wrong heading levels, or containing broken link syntax — may be silently ignored by LLM systems. Always validate before deploying.

FAQ: llms.txt for AI Search Visibility

Is llms.txt an official web standard?

Not yet. It's a community-driven specification with broad voluntary adoption, but it has not been formalized through the IETF or W3C. That said, with thousands of sites already using it and documentation platforms building native support, it has strong de facto momentum.

Will llms.txt hurt my traditional SEO?

No. The file is purely additive — it doesn't replace your robots.txt, sitemap, or any existing SEO infrastructure. It provides an additional, AI-optimized layer of content discovery.

Do AI search engines actually read llms.txt?

Adoption varies by platform and is evolving rapidly. The more important point: providing clean, structured, Markdown-formatted content at a predictable URL makes your site easier for *any* AI system to parse, whether or not it specifically looks for llms.txt.

How often should I update my llms.txt?

Every time you publish significant new content, launch a product, or deprecate a page. Ideally, tie updates to your deployment pipeline so the file stays in sync automatically.

Can I use llms.txt for a small business or personal site?

Absolutely. Even a simple file with five to ten links — your homepage, services page, pricing, FAQ, and contact — gives AI systems structured context they wouldn't otherwise have.

Conclusion: Make Your Site AI-Visible Today

The shift from traditional search to AI-powered discovery isn't coming — it's here. Every day without an llms.txt is a day where AI assistants might misrepresent your business, ignore your best content, or send potential customers to a competitor who made the effort.

Here's your action plan:

• Today: Create a basic llms.txt with your site name, a clear summary, and links to your 10–20 most important pages.
• This week: Build out llms-full.txt with the full content of those key pages.
• This month: Validate both files, integrate generation into your build pipeline, and set up monitoring.

The standard is simple by design. You don't need special tools or technical expertise — just 30 minutes, a text editor, and the willingness to meet AI systems where they're looking.

Want to know how your site performs in AI search today? CheckSEO analyzes your AI search visibility and gives you actionable recommendations — including whether your llms.txt is working as intended. Run your free audit now and see where you stand.