Publish an llms.txt file

llms.txt is a proposed convention: a Markdown file at the site root that maps the most useful pages for AI systems. It is not a Google standard and adoption is unproven, so publish it as a low-cost experiment, keep it current, and measure referrals instead of assuming impact.

lowIn codeimpact effort

Why this matters

A curation experiment, not a ranking tactic.

Get the full breakdown

The extended brief explains when this works, when it does not, and the trade-offs. Join the waitlist and it unlocks for every tip when sign-in opens; MCP access is the paid part.

Join the waitlist

Before you start

  • A list of canonical, indexable key pages

How to apply it

  1. Select the pages worth mapping General best practice

    Choose canonical documentation, product and reference pages that answer common questions. Exclude gated, duplicate and low-value URLs.

    Check: The page list contains only URLs that return 200, are indexable and are canonical.

  2. Write the file General best practice

    Create /llms.txt in Markdown with an H1 title, a summary paragraph and sections of links with one-line descriptions. Keep it plain text.

    Check: curl your production domain/llms.txt returns 200 with text/plain or text/markdown and contains the title and at least one link.

  3. Keep it current and measure General best practice

    Regenerate it when key pages change, and track AI referrers separately to judge whether the file has any effect.

    Check: The file lists no URL that returns 4xx or 3xx, and AI referral traffic is tracked in a dedicated channel group.

Get the prompt for your agent

A ready-to-paste prompt for your agent that implements this tip and verifies the result in your repo.

Join the waitlist

Source: seo-mcp playbook · curated entry · report an error

Framework notes

astro

Add a file at `public/llms.txt`, or generate it from content collections in an endpoint so the list stays current with the site.

nextjs-app

Serve it from `public/llms.txt` for a static file, or add a route handler returning `text/plain` if the list is generated at request time.

Official references

Let your agent apply this + 161 more tips.

Pre-order — $19