llms.txt has become one of the most talked-about “must-do” tactics in AI SEO circles this year, but recent data suggests most llms.txt files are receiving little or no automated traffic. Here’s what’s actually happening, what Google says about the format, and what site owners should focus on instead.
What Is an llms.txt File?
An llms.txt file is a simple text file placed in your website’s root directory, similar to a robots.txt file, designed to give AI systems and language models a clean, structured summary of what your site contains. The idea is that instead of an AI crawler trying to parse your full HTML, navigation, ads, and design elements, it can read a short, plain-text overview and understand your site’s key pages and purpose much faster.
It sounds like a smart idea on paper. Structured, distraction-free text should, in theory, be easier for any automated system to work with than a fully rendered webpage.
Does Google Use llms.txt?
If your main concern is Google Search, you don’t need an llms.txt file. Google Search Central’s current guidance says that llms.txt files aren’t needed for Google Search and don’t have a positive or negative effect on a site’s visibility or rankings. Google also notes that site owners can maintain an llms.txt file if they want to support other services or systems that may use the format.
Why It Became So Popular So Fast
As AI-powered browsing and research tools grew rapidly over the past year, site owners understandably wanted a way to make sure their content was being read and represented accurately by these systems. llms.txt was proposed as a lightweight, low-effort solution — add one file, and theoretically improve your chances of being understood correctly by AI tools.
Agentic AI activity — bots and automated agents that browse and act on behalf of users — has grown enormously. At the same time, AI agents are becoming a more visible part of web traffic. These systems can browse websites, retrieve information, and sometimes take actions on a user’s behalf. That growth has naturally increased interest in ways to make websites easier for AI systems to understand.
What the Data Actually Shows
Here’s where things get interesting. A large-scale Ahrefs study of 137,000 domains found that 28% of the sites in its dataset published an llms.txt file. More importantly, 97% of those files received zero requests in May 2026. Of the small percentage that did receive requests, some traffic came from named AI tools, while other requests came from bots, research tools, and llms.txt-related services.
For most sites in the study, there was little evidence that automated systems actively requested their llms.txt files during the measurement period. The file was not requested during the measurement period, indicating no evidence of active use by automated systems.
This doesn’t necessarily mean the concept is worthless forever. It’s common for standards like this to need time before they’re widely adopted by the companies building AI crawlers. But right now, treating llms.txt as a priority task is likely misplaced effort for most site owners — especially smaller blogs and businesses with limited time to spend on technical SEO.
What the Data Doesn’t Prove
It’s important not to overinterpret the numbers. The fact that most llms.txt files in the Ahrefs study received no requests doesn’t prove that the format can never be useful. It shows that, during the measurement period, the files were not widely being requested across the study’s dataset. Adoption and usage can change as AI platforms evolve, so the best conclusion for now is that llms.txt remains an optional experiment rather than an essential SEO requirement.
So Should You Bother Creating One?
Given the low adoption by actual AI crawlers right now, here’s a practical way to think about it:
Skip it if: You have limited time and need to prioritize higher-impact tasks like page speed, content quality, or fixing broken links and crawl issues.
Go ahead and add one if: You’ve already got your core technical SEO in order and want to cover your bases for the future, since creating a basic llms.txt file takes very little time and carries no real downside.
Don’t rely on it as your main AI-visibility strategy either way. Whether or not you add the file, it shouldn’t replace the fundamentals that we know actually matter to how AI systems find and cite content.
What to Prioritize Instead
If your real goal is showing up well in AI-generated answers and staying easy for AI crawlers to understand, these fundamentals matter far more than llms.txt right now:
- Clean, crawlable site architecture. Make sure your important pages are easy to reach through normal internal links, not buried behind complex navigation or JavaScript-heavy menus.
- Clear, direct writing. Answer questions plainly and organize information with descriptive headings. This makes content easier for people and automated systems to understand, although no particular writing format guarantees inclusion in an AI-generated answer.
- Solid technical health. The crawl budget fundamentals — fast load times, no broken links, clean sitemaps — affect whether your content gets discovered and re-crawled in the first place, which matters more than an unread text file.
- Structured data and schema markup. Structured data is a well-established way of helping search engines understand information about a page and its entities. It can support eligible search features, although it doesn’t guarantee enhanced rankings or AI citations.
- Genuine expertise and originality. Content that demonstrates first-hand knowledge, useful detail, and clear expertise can provide stronger value to both search users and AI systems, although no single content characteristic guarantees AI citations.
Notice that all of these overlap heavily with standard SEO best practices. That’s not a coincidence — the things that make your site easy for search engines to understand tend to make it easy for AI systems too.
| SEO Task | Priority for Small Sites | Why |
|---|---|---|
| Helpful, original content | 🔴 High | Supports search visibility and user value |
| Internal linking | 🔴 High | Helps discovery and site structure |
| Technical SEO | 🔴 High | Helps crawling and accessibility |
| XML sitemap | 🟠 Medium | Helps search engines discover URLs |
| Structured data | 🟠 Medium | Helps search engines understand content |
| llms.txt | 🟢 Optional | Limited evidence of current usage |
Why This Matters for Small Sites Specifically
When you’re managing a small site, time is tight, and it’s easy to get pulled into chasing whatever “AI SEO” trend is buzzing at the moment. What happened with llms.txt shows why it pays to verify a tactic actually works before committing time to it.
Right now, the honest answer is that llms.txt adoption by AI crawlers is still very low. That could change as the technology matures, but for today, your time is almost certainly better spent on the fundamentals that are proven to matter.
Frequently Asked Questions (FAQs)
Does adding an llms.txt file hurt my SEO?
No, it won’t hurt anything. It simply isn’t likely to help much right now, based on current adoption data.
Will AI crawlers eventually read llms.txt files more often?
Standards like this may gain wider adoption over time, but there’s no guarantee, and no clear timeline for when or if that will happen.
Is llms.txt the same as robots.txt?
No. Robots.txt tells crawlers which parts of your site they’re allowed to access. llms.txt is meant to summarize your content for AI systems — they serve different purposes, and robots.txt is far more established and consistently respected.
What should I focus on instead if I want better AI search visibility?
Clear, direct writing, clean site structure, fast page speed, and genuine expertise in your content matter far more right now than llms.txt adoption.
Is agentic SEO worth paying attention to at all?
The broader trend of AI agents browsing and acting on the web is real and growing, but the specific tactics for optimizing for it are still developing. It’s worth staying aware of, without over-investing in unproven tools just yet.
Does llms.txt improve Google rankings?
No. Google Search Central says llms.txt isn’t needed for Google Search and doesn’t positively or negatively affect Google Search visibility or rankings.
Should I add llms.txt to my small blog?
You don’t need to add it for Google Search. If your core SEO is already in good shape, creating one is a reasonable optional experiment, but you shouldn’t expect it to improve rankings or AI visibility automatically.
Final Thoughts
llms.txt is a good example of a well-intentioned idea whose adoption and practical use are still developing. Current evidence suggests that most files receive little or no automated attention, while Google’s guidance makes clear that the file isn’t needed for Google Search. Rather than treating it as an urgent task, it’s fine to add one quickly if you have a spare few minutes. Still, your actual AI-visibility efforts are better spent on the fundamentals: clean technical health, direct writing, and genuinely useful content.
The sites most likely to remain competitive in AI search are those that make their content easy to discover, understand, and trust. That means strong technical SEO, clear organization, useful information, and genuine expertise — regardless of whether an llms.txt file is present. For now, llms.txt is optional. Google says it isn’t needed for Google Search, while current data shows that most files receive little or no automated traffic. Adoption may evolve, but small websites should prioritize proven SEO fundamentals first.
Want help figuring out where to actually focus your SEO time for the best results? Marketing Glob Hub offers technical SEO reviews starting at $50/month — get in touch to get started.
