Free tool

llms.txt generator

A markdown index for AI crawlers: one heading, one summary, and grouped links to the pages that matter. Built only from URLs we actually saw on your site or in your sitemap.

Reads the homepage and up to six other public pages. First runs may take a few minutes. Full-URL results are cached for up to 24 hours; requests are rate limited.

What llms.txt is

A convention, similar to robots.txt, for a markdown file at the root of your site that gives language models a curated map: what the site is, a one-line summary, and links grouped under headings like Docs, Product and Company.

Why it helps

Crawlers with a limited budget read the map first and spend the rest on the pages you listed. It does not guarantee citations, but it removes the guesswork about which of your hundred pages carry the substance.

Where to put it

Serve it as plain text at https://yourdomain/llms.txt. Most hosts let you drop it in the public folder. Keep it under sixty lines and update it when major sections change.

FAQ

Common questions

Will every link be real?

Yes. The generator is only allowed to use URLs it found on your pages or in sitemap.xml. It cannot invent paths.

What if my site has no sitemap?

It falls back to the links discovered on the homepage and the pages it read. The file will be shorter; add links by hand.

Should I also make llms-full.txt?

Only if you have documentation worth including in full. For a marketing site, llms.txt alone is enough.

Do all AI crawlers read it?

Adoption is uneven and changes month to month. It is cheap to add and there is no downside, which is why we recommend it.

Is it cached?

Per domain, yes. If you change your site and want a fresh file, wait for the cache to age out or edit the file directly.

Other free tools

COMPANY.md generator DESIGN.md generator Invisible character cleaner

Want the full loop: research, writing in your voice, publishing and tracking? Start a free trial.