Only 10% Of The World’s Top Sites Are Using LLMs.txt Files

With nearly all clients recently asking me about AI/GEO, a common question is ‘do we need an LLMs.txt file’?
My answer is ‘no’ but I was curious as to whether it’s commonly adopted yet.

Before I get into the data, it’s worth me calling out I’m not the first to do this:

• SERanking analysed 300,000 sites and found similar (10%ish using LLMs.txt)
• There is also a directory of sites using LLMs.txt at LLMs-text.com – although obviously this tells us that some sites are using them, rather than percentages

How I Got The Data

• I took the top 1000 domains (top being as per DataForSEO.com ‘Top 1000 Websites By Ranking Keywords’)
• Then, knowing the file location should always be domain.com/llms.txt – did a status code lookup in Google Sheets
• This wasn’t always accurate so I also manually spot checked (hey, I’m a dad of three – I need distractions)
• I then categorized the domains – I manually did some then got Gemini to do the rest

Data Caveats

As you can see, my mini-study of 1000 domains pales to the 300,000 SERanking did. I’m not claiming my data is a comprehensive picture of the entire web. So, I would caveat my findings on LLMs.txt adoption by telling you:

• It tells us 10% of the top ranking sites use LLMs.txt – this doesn’t mean the bottom ranking sites are or aren’t
• As this is the top ranking sites worldwide (and with countries like the US or India having larger populations) this doesn’t allow us to say which countries are more likely to use LLMs.txt
• My categorizations are high level and subjective
• Many of the sites in the list will be owned by one company – so I can imagine that if one SEO team is responsible for several sites, their approach will be the same (either to use or not use LLMs.txt files)
• A big chunk (254) of the sites are publishers – so the data is skewed towards that category
• Some categories only had one or a few domains in them – so saying X% of sites in that category do/don’t use LLMs.txt files isn’t robust

OK with all that said, here’s what I found..

Headline Data

• Only 10% of the 1000 domains I looked at had an LLMs.txt file (broadly what I’d expected as I was familiar with the SERanking study)
• 95% of publishers didn’t have LLMs.txt files
• 85% of ecommerce sites didn’t have LLMs.txt files
• 89% of learning sites didn’t have LLMs.txt files
• 16% of travel sites did have LLMs.txt files

Bonus – AI Tracking Platforms Using LLMs.txt

Just for fun, I took a bunch of AI/GEO tracking platforms to see if they’re using LLMs.txt:

10/17 were.

I’d expected this to be 100%, so was surprised, especially to see Ahrefs and Profound aren’t using LLMs.txts, especially as a page on Profound states

“Act now: Start with a minimal llms.txt file, link your most critical documentation, and iterate based on LLM performance. The era of AI-native content delivery is here.“

Should You Bother Using LLMs.txt?

In my opinion – no. Here’s my rationale:

1) It’s a proposed standard

2) A well structured, accessible site shouldn’t need to ‘dumb down’ what it’s about for an LLM

3) Studies show most LLMs.txts don’t even get crawled

4) For LLMs.txt’s that do get crawled, I’ve yet to see any substantial evidence to suggest the content is needed by or influences AI search platforms

5) Ask yourself this – if the majority of the most popular sites in the world aren’t using LLMS.txts (presumably because they feel it’s not needed) – why would your site?

I also don’t buy into the rationale of “well, it doesn’t hurt having it”. Well, sure, that’s correct, but then why not add tons of txt files or other experiments?
An exception being cats.txt which is bloody amazing.

And as aside, I would say most sites I work with (even big ones) have serious ‘normal’ SEO deficits and are just chasing shiny looking distractions like this.

Dataset

• Google Sheet with LLMs.txt usage