Blog
Does your business website need an llms.txt file?
By Bhavesh Barot, Updated

Your website doesn't need an llms.txt (a plain text file that tells AI tools what your site is about) to appear in Google's AI answers; Google says no new "AI text files" are needed. We checked all 220 websites in Google's ordinary top 10 for 37 local searches on 15 November 2026, and 140 of the 179 that answered had none.
The other 41 blocked our request, timed out or returned a server error. The 140 show that a site can rank in Google's regular list of links without the file. On WordPress, the Yoast SEO and AIOSEO plugins (add-ons for WordPress) can generate one for you. We did not measure whether any AI tool reads these files, or whether one changes what AI answers say about a business, and none of the sources we read reports such a test.
How many websites in local Google results have one?
We counted. On 15 November 2026 we ran 37 local searches through DataForSEO, a service that supplies Google search data, and took every website in Google's top 10 ordinary results for each one:
- 12 "best ..." searches set in Bengaluru
- 8 "near me" searches set in Bengaluru
- 10 "best lawyer" searches in ten US cities
- 7 "best CA firm" searches in seven Indian cities
That gave 220 different websites, a mix of firms, directories, listing sites and social sites.
At 02:57 UTC we requested /llms.txt from each site with a plain request and a normal browser identity, so that sites saw an ordinary visitor. A site counted as having an llms.txt only if it answered normally (HTTP 200, the code a server sends when a request works) and sent something other than a web page. Its text also had to begin with a # sign, the proposal's required title line, after any byte-order mark (an invisible marker some editors add at the start of a file) or blank space. Sites that blocked the request, timed out or returned a server error count as unreadable, and no blocked request was retried.
| What we found | Websites |
|---|---|
| Checked | 220 |
| Blocked, timed out or server error | 41 |
| Answered our request | 179 |
| Had an llms.txt that passed our test | 39 |
| No file that passed our test | 140 |
So 39 of the 179 that answered had one. That is 22%, a little more than one in five.
Some files say which tool made them. Of the 39:
| Tool named in the file | Files |
|---|---|
| Yoast SEO, a WordPress plugin | 3 |
| AIOSEO (All in One SEO), a WordPress plugin | 1 |
| Another tool | 5 |
| No tool named | 30 |
Thirty of the 39 named no tool, so we can't tell whether they were written by hand or made by software that doesn't sign its work. Our own site has an llms.txt, built by our own code, and it names no tool either.
The count covers Google's ordinary results only. Whether any AI tool read the 39 files, or changed an answer because of one, is outside what we measured. We also asked only at the root, though the proposal allows files in subfolders, and the 41 sites that did not answer are left out of the 22%. This is one day's look at 37 searches.
What is an llms.txt file?
It is a proposal by Jeremy Howard, first published on 3 September 2024 and revised on 10 August 2026. The name comes from LLMs (large language models), the systems behind AI chat tools. He suggests a short file at your web address followed by /llms.txt, written in Markdown (plain text with simple marks for headings and links), so an AI tool can find a summary of your site and links to its key pages in one place.
The proposal requires one line only, the name of your site or business after a # sign. A short summary after a > sign and any notes can follow, then lists of links under headings. A list headed "Optional" holds pages an AI tool can skip when it needs a shorter version.
Its author says the files are used most heavily for software documentation, where AI coding tools follow them to reference pages and tutorials, and names a business outlining its structure and policies as another fit. Chrome's Lighthouse testing tool now has an experimental check that looks for the file at the root (the top folder) of a site, the same place our count looked.
Is llms.txt the same as robots.txt?
No. A robots.txt file is far older, and it tells crawlers (programs that fetch web pages for search engines) which pages they may visit. Martijn Koster first defined it in 1994, and in September 2022 the Internet Engineering Task Force (IETF) set it out as a proposed internet standard, RFC 9309. The RFC describes rules that crawlers are requested to honour and says they are "not a form of access authorization". In Google's plainer words, a robots.txt file cannot enforce what a crawler does; obeying it is up to the crawler.
| robots.txt | llms.txt | |
|---|---|---|
| Job | Tells crawlers which pages they may visit | Gives AI tools a summary of your site and links to key pages |
| Where it goes | The root of your site, one file per site | The root of your site, or a subfolder for the pages under it |
| Status | A proposed internet standard, RFC 9309 (September 2022) | A proposal, first published in September 2024 |
| What's inside | Groups of User-agent, Allow and Disallow lines | A # name line, a > summary, then lists of links under headings |
The proposal says llms.txt can add context for content that robots.txt allows. Google's guide to its AI features lists making sure crawling is allowed in robots.txt, and by your hosting, as the first of its basics.
To see yours, open your web address followed by /robots.txt. A Disallow: / line under User-agent: * asks crawlers (Google's AdsBot crawlers excepted) to stay out of your whole site. If you find one and did not mean it, ask whoever runs your website.
Will an llms.txt help you show up in ChatGPT or Google's AI answers?
Google's guide to its AI features doesn't mention llms.txt either way. It says no extra requirements or special optimisations are needed to appear in them, with no new machine-readable files, AI text files or markup, and no special schema.org structured data (code that labels facts on a page for search engines). To be shown as a supporting link, a page must be indexed (in Google's list of pages it can show) and eligible to appear in Google Search with a snippet, the short text under a result.
Its other basics include links between your own pages, important content available as text, structured data that matches the visible text on the page (our post on FAQ schema covers one kind) and an up-to-date Business Profile, the listing Google shows for your business.
For ChatGPT, the direct way to know where you stand is to look at what it says about you. Two of our other posts cover those answers: how to rank in ChatGPT and Google AI Overviews.
Seen in Search writes buyer questions from what your homepage says you sell, then scores the AI answers to them in code, with the weights on our methodology page. The free report a visitor gets today checks ChatGPT only. The full paid report also covers Google AI Overviews and Perplexity. What your AI visibility score means explains the number, and the report is private when it finishes.
One free report per business. Use your business email. No card.
What should a service business put in its llms.txt?
Keep to the proposal's format, and keep the file short. Its author's tips: clear, concise language; a brief note on what each link covers; no unexplained jargon; and a test, giving an AI assistant only your llms.txt and asking it about your business.
For a service firm, that means your name and a line on what you do and where, followed by links to your service and contact pages. Here is an illustrative example for a made-up firm, one line of the file per row, in order. Nothing in it describes a real business, and the links use example.com, an address set aside for examples.
| Line in the file | What it does |
|---|---|
# Acme Chartered Accountants | Your firm's name after a single #, the only required line |
> Chartered accountants in Bengaluru for startups: company tax and GST returns. | A short summary of what you do and where |
We do not file personal income tax returns. | A plain note, here on work the firm does not take |
## Services | A heading that starts a list of links |
- [Company tax returns](https://example.com/company-tax): what we file and what we need from you | One page per line: a link, then a short note on what it covers |
## Contact | A second heading |
- [Contact us](https://example.com/contact): office address and phone number | How to reach the firm |
Keep it in step with your pages. A file that speaks for your business goes out of date when your services or prices change.
Ours did. On 13 November 2026, our own llms.txt still listed old plan prices because the code that builds it read an out-of-date price list, and we corrected it on 14 November 2026. If a plugin refreshes your file every week, as Yoast says its feature does, read the result after you change a service.
How do you add an llms.txt to your website?
Of the 39 files in our count, 3 named Yoast SEO and 1 named AIOSEO as the tool that made them. Both are WordPress plugins:
- Yoast SEO. The feature is off until you switch it on. Then, Yoast says, the plugin generates the file automatically and refreshes it every week, drawing on your most recently updated content, your sitemap (the list of your pages for search engines) and the descriptions you've added about your site. Yoast announced it on 10 June 2025.
- AIOSEO. Its page says users of the free Lite version can generate llms.txt files automatically. Extras, such as a fuller file called llms-full.txt, need a Pro account.
On any other site, write the file in a plain text editor, save it as llms.txt and upload it to the top folder of your site, where a robots.txt file also goes. If your site runs on a website builder, look in its help pages for an llms.txt setting first.
Then test it the way we did. Open your web address followed by /llms.txt in a browser. You should see plain text that starts with # and your business name, the same test the 39 files in our count passed. If you see a web page or an error instead, the file is not in place yet.
Should your business add one?
For a service business, we'd call the file optional. In our count, 140 of the 179 sites that answered ranked in Google's ordinary top 10 without one, and Google's guide asks for no such file for its AI answers. Check what Google does list first: crawling allowed in robots.txt and by your hosting, and your services described in text on your pages. Then, if your WordPress plugin can generate an llms.txt, switch it on and read what it writes. Read it again whenever your services change.
Questions owners ask about llms.txt
Is llms.txt worth it?
Is llms.txt still relevant?
Where to upload llms.txt file?
How to create an llm.txt file?
How can I create an LLMs.txt file online?
How to write a good LLMs txt?
Can you provide an example of an LLM txt file?
What is an AI txt file?
What is the format of a robots.txt file?
Is robots.txt legal?
One free report per business, private unless you publish it. Use your business email. No card. Questions about the report? Contact us.
Sources
- Jeremy Howard, The /llms.txt file, first published 3 September 2024, revised 10 August 2026, read 15 November 2026.
- Google Search Central, AI features and your website, last updated 10 December 2025, read 15 November 2026.
- Google Search Central, Robots.txt introduction and guide, last updated 10 December 2025, read 15 November 2026.
- Google Crawling Infrastructure, Create and submit a robots.txt file, last updated 21 November 2025, read 15 November 2026.
- M. Koster, G. Illyes, H. Zeller and L. Sassman, RFC 9309: Robots Exclusion Protocol, Internet Engineering Task Force, September 2022, read 15 November 2026.
- Chrome for Developers, Lighthouse agentic browsing scoring, last updated 5 May 2026, read 15 November 2026.
- Yoast (Beth Parker), New: Future proof your website for tomorrow's visitors with Yoast SEO llms.txt, 10 June 2025, read 15 November 2026.
- All in One SEO (AIOSEO), Control AI Indexing With the WordPress LLMs.txt Generator, read 15 November 2026.
- Spawning (Cullen Miller), ai.txt: A new way for websites to set permissions for AI, 30 May 2023, read 15 November 2026.
- Our own check: llms.txt on the 220 websites in Google's top 10 for 37 local searches (found through DataForSEO's Google search data), checked 15 November 2026, 02:57 UTC.
- Our own llms.txt: old plan prices when read on 13 November 2026, corrected on 14 November 2026, read again on 15 November 2026 (no tool named in it).