AI search · guide 2 of 6
AI search myths: llms.txt, “GEO” and other things you may not need
Popular AI-search claims checked against what Google and AI companies have actually published: llms.txt, GEO, schema, AI content and “SEO is dead”, with the uncertain parts labelled honestly.
25 min readUpdated 9 Nov 2026
- 12 questions to test yourself
- 1 calculator
- 7-item checklist
On this page9 sections
A new acronym does not mean a new rulebook
This month's email offers “GEO” for €800 a month. Last month's offered “AEO”. Next month it will be something else. Each says your SEO stopped working and that the fix is a new monthly fee.
Some of these offers describe real, useful work. Others rename things you already have, or sell a file that no major search engine has said it reads. From the outside they look the same, which is exactly why they work.
By the end of this guide you will be able to check the claims sellers make most often against what Google, OpenAI and others have actually published, work out what an offer really costs per hour of work, and leave with a one-week list to tick off on your own site. Where nobody knows the answer yet, we say so.
Two things to watch for on the way. There is one small setting, which many owners have never looked at, that can hide a website from Google and AI assistants at the same time. And there is one sentence in AI-SEO sales emails that should end the conversation on its own. Both turn up below.
Start with one guess. Google publishes a page for site owners called “AI features and your website”, covering AI Overviews and AI Mode. How many extra requirements do you think it lists?
Make a guess
Beyond what normal Google Search needs, how many additional technical requirements does Google list for appearing in AI Overviews or AI Mode?
Why AI search is such fertile ground for myths
Three things make AI search a myth factory. First, it is new: Google launched AI Overviews in the US in May 2024 and rolled AI Mode out in the US in May 2025, and ChatGPT added its own search feature in late 2024. New things come with few published facts.
Second, the answers are hard to measure. The same question can give a different AI answer each time, and most tools cannot tell you exactly how often you were cited. When results are fuzzy, almost any tactic can be made to look as if it worked.
Third, there is money in confusion. A business owner who believes the rules changed overnight is easier to sell a new package to. That does not make every seller dishonest. It does mean the burden of proof is on the claim, not on you.
So you need one test you can apply in ten seconds. Try it on four sources before reading on.
Quick quiz
You want to know what Google's AI Overviews need from your website. Which source can you actually hold to its word?
Pick one. Your first answer is the one that counts.
llms.txt: what it is, and who actually reads it
llms.txt is a proposal published in September 2024 by Jeremy Howard of Answer.AI. The idea: put a plain Markdown file at yoursite.com/llms.txt with a short summary of your site and links to its most useful pages, so a language model can find the good stuff without wading through menus and cookie banners.
It is a sensible idea for some sites. Software companies with large documentation sites publish llms.txt files so that coding assistants can read their docs, and some AI developer tools use them that way.
What llms.txt is not is a standard that search engines have adopted. It has not been agreed by any standards body, Google representatives have said publicly that Google Search does not use it, and the major AI companies' crawler documentation talks about robots.txt, not llms.txt, when it explains how to control access. Google's own page on AI features lists no such file.
So: adding one does no harm and takes half an hour. Paying a monthly fee for it, or treating it as the reason you will or will not appear in AI answers, is not supported by anything the search companies have said.
Now compare it with the two files that do matter. One of them holds the setting promised at the start. Guess which before you look.
Quick quiz
One line in one of these files can hide your whole site from Google and AI assistants at once. Which file?
Pick one. Your first answer is the one that counts.
Side by side
| Question | robots.txt | sitemap.xml | llms.txt |
|---|---|---|---|
| What it does | Tells crawlers which parts of the site they may fetch | Lists the pages you want crawled | Summarises the site for language models |
| Status | An internet standard (RFC 9309, 2022) | A long-established protocol Google documents | A proposal from 2024 |
| Does Google Search use it? | Yes | Yes | Google has said it does not |
| Matters for AI crawlers? | Yes: OpenAI, Anthropic, Perplexity, Google and others document their crawler names for it | Helps them find pages, like any crawler | Not documented as a factor by the major assistants |
| Worth your time? | Check it today | Have one; most platforms make it for you | Optional, harmless, low priority |
“GEO” and “AEO”: new work or a new label?
“GEO”, generative engine optimisation, comes from a 2023 research paper by researchers at Princeton University and other institutions. They tested ways of rewriting content in an experimental setup and found that changes such as citing sources, adding quotations and including statistics made content more visible in AI-generated answers.
Read that list again: cite your sources, quote people who know, use real numbers. That is also what good writing for humans looks like. The research is real; the leap from it to “your SEO is obsolete, buy GEO” is not in the paper.
“AEO”, answer engine optimisation, is older. It was used in the era of featured snippets and voice assistants for writing that answers a question directly. Again, a reasonable habit with a new name.
There is some genuinely new work in AI search: checking that AI crawlers are not blocked, testing what assistants say about you, and making sure your business facts match everywhere they are listed. Most honest providers will tell you that this sits on top of ordinary SEO, not instead of it. The guide on getting mentioned by ChatGPT and other AI assistants walks through each of those checks step by step. First, try translating the jargon yourself.
Test yourself
Think of your answer, then flip the card.0 of 5 flipped
Quick quiz
Run the pitch through your hype meter. Which line is the only one describing work clearly worth paying for?
Pick one. Your first answer is the one that counts.
Does schema markup get you into AI answers?
Schema markup, or structured data, is code that labels facts on your page for machines: this is the business name, these are the opening hours, this is the price. It has real uses. Google uses it for features such as review stars, product prices and event listings in normal results.
For AI features, Google's guidance is plain: there are no additional requirements, so no special structured data is needed. Where you do use structured data, it should match what is visibly on the page, the same rule that has always applied. If you want to know which types are worth adding, the guide on schema markup covers the handful a small business actually needs.
The FAQ story is a useful warning. Years ago, FAQ markup could make your result take up more space in Google. In August 2023 Google limited FAQ rich results to well-known, authoritative government and health websites. Plenty of businesses had pasted FAQ markup onto every page for that effect, and it simply stopped showing for them. Chasing a format because it works today is fragile; describing your business accurately is not.
What would you do?
Your developer's suggestion
This is an illustrative example. You run a small accounting firm with 30 pages. Your developer offers to add FAQ markup to every page, with five invented questions each, “so AI tools pick us as the answer”. It would cost two days of work.
What do you do?
Is AI-written content penalised?
Many owners have been told that Google can spot AI writing and punishes it, so they either avoid AI tools entirely or hide that they use them. Before the answer, test your instinct on four pages.
Quick quiz
Which of these pages is the kind Google's spam policies target?
Pick one. Your first answer is the one that counts.
So the answer is: not for being AI-written. In February 2023 Google published its guidance on AI-generated content: it rewards high-quality content however it is produced, and what breaks its spam policies is using automation to generate content mainly to manipulate rankings.
In March 2024 Google sharpened this with the scaled content abuse policy you just met. The question Google asks is not “was this made by AI?” but “why does this page exist, and does it help anyone?”
In practice, the risk is not the tool. It is the shortcut the tool makes easy: fifty near-identical city pages, a blog post a day that says what every other post says, a product description that invents features. The guide on using AI to write content safely goes deeper; here is the difference at a glance.
Before and after
Two pages written with AI help (illustrative)
Shortcut
“Boiler Repair in Varna: Your Trusted Experts. Looking for boiler repair in Varna? Our expert team provides top-quality boiler repair services in Varna at competitive prices. Contact us today for all your boiler repair needs in Varna.”
The same page exists for 40 other towns, with only the town name changed.
Assisted
“We repair Vaillant, Bosch and Ariston boilers in Varna and the villages up to Aksakovo. A call-out is €40, most repairs take one visit, and we carry common parts in the van. The most frequent fault we see in winter is low pressure: here is how to check yours before you call.”
AI helped draft and tidy it; the owner added the brands, prices, area and the winter tip from real jobs.
- Both used AI. Only the second contains anything a reader could not get from any other page.
- Specific facts, real prices and first-hand tips are what make a page worth citing, by Google or by an AI assistant.
- The first page is the pattern Google's scaled content abuse policy describes, whoever or whatever wrote it.
Is SEO dead?
This one has been announced many times since search engines began. The current version says that because AI answers questions directly, nobody will click on websites any more.
Some of the worry is real. AI answers do settle some questions on the results page, and for simple facts that can mean fewer clicks. Forecasts exist, such as Gartner's prediction in early 2024 that traditional search engine volume would fall 25% by 2026, but a forecast is not a measurement, and nobody outside the search companies has complete figures. If your clicks have fallen while impressions rose, the guide on zero-click searches, next in this series, shows how to find the calls and visits that never show up as clicks.
What is not in doubt is how the AI systems find their sources. Google's AI features draw on pages in Google's index. ChatGPT search, Perplexity and Claude find pages through web search and fetch them with their own crawlers. Each of them needs pages that can be crawled, understood and trusted, which is what SEO has always been about. The work is shifting in emphasis, not disappearing. Test yourself on the claims you have probably been sent.
Myth or fact?
Six claims you have probably been sent
1.Blocking Google-Extended in robots.txt will remove you from Google Search.
2.To be shown as a link in AI Overviews, a page has to be indexed and eligible for a snippet in normal Search.
3.If you block OpenAI's search crawler, ChatGPT search will generally not show your pages in its answers.
4.AI tools only cite the biggest brands, so small businesses should not bother.
5.Google has confirmed that pages written for AI get a ranking boost in AI Mode.
6.Nobody outside the AI companies can see exactly how their systems choose sources.
Judge any AI-SEO offer in two minutes
Strip away the acronyms and the list of things that genuinely help is short and unglamorous. Let crawlers in. Get your pages indexed. Say clearly who you are, what you do, where and for how much. Publish things only you can say. Keep your business facts identical on your site, Google Business Profile and the main directories. Earn mentions from other real sites.
None of that is new, and none of it can guarantee you a place in an AI answer, because nobody controls those answers except the companies that run them. What it does is remove the reasons an AI system, or Google, would have to skip you.
So the question for any offer is: how much of that list does it actually do, and at what price? Put the numbers from the last proposal you received into the calculator, with the hourly rate you would pay a skilled freelancer as your yardstick, then score it. It works on any proposal, including ours.
Try the numbers
What does the offer cost per hour of real work?
Price per hour of described work
€200 per hour
Paid above that rate each month
€640
What you commit to before you can stop
€9,600
Your own numbers, not a judgement of any provider. A big monthly gap is not proof of a bad deal, but it is the question to ask: what else does the fee cover, and can you check it? Count only tasks they name and you could verify; if they cannot name the hours at all, that answers the first question in the scorecard below.
Score yourself
Is this AI-SEO offer worth it?
They describe, in plain words, the actual work they will do each month
The work would still help your Google results if AI search disappeared tomorrow
They do not guarantee placement in ChatGPT, AI Overviews or any ranking
They start by checking that Google and AI crawlers can reach your site
No deliverable is only a file (such as llms.txt) or a renamed report
They explain how they will measure results and admit what they cannot measure
The price matches the hours of real work they describe
You can stop without a long lock-in contract
0 of 8 answered. Answer them all to see your result.
What to do this week instead of buying an acronym
Here is the same list turned into tasks, ordered so the first ones protect everything after them. If the robots.txt check is the only thing you do today, you have already done the most important AI-search work there is. Most of it takes an afternoon, and you can tick it off as you go.
The free SEO check linked at the end of this guide does the first and third checks for you on any page: it reads your robots.txt and tells you if it blocks Googlebot, and it flags a noindex tag. It does not test the AI crawlers by name, so for those, open the file and look for their names; it is rarely more than a dozen lines. If a page turns out not to be indexed, the guide on why pages are missing from Google lists the usual causes and fixes.
Checklist
Your one-week list, in this order
Saved on this device only
Questions people ask
Should I add an llms.txt file anyway?
If it costs you nothing, it does no harm, and it may help AI tools that do read it. Just do not expect it to change whether Google or the main assistants cite you, and do not pay for it as a service.
Should I block AI crawlers to protect my content?
It is your choice, and the controls are separate. You can block training crawlers such as GPTBot and ClaudeBot, or opt out of Gemini training with Google's Google-Extended token, and still allow search crawlers such as OAI-SearchBot, Claude-SearchBot and Googlebot. Blocking the search crawlers means those assistants cannot read your current pages when answering.
How do I know if AI assistants mention my business?
Ask them the questions your customers would ask, with search turned on, and note who is named and which sources are linked. Repeat monthly with the same questions. Answers vary between runs, so look for patterns over several months, not single results.
You now have the yardstick: zero extra requirements, one file that can hide you, and one sentence that ends a sales call. With the myths cleared away, the useful question is what does make a difference.
Two guides pick up from here. Zero-click searches, next in this series, shows how to count the value of searches that end on Google without a visit. And the guide on getting mentioned by ChatGPT and other AI assistants turns the honest list above into four levers you can pull, starting with the crawler names you just checked.
You reached the end
Your summary
This guide has 12 quick questions
Try them to see how much you already know. Your first answer is the one that counts.
AI search: 0 of 6 read
- 1. AI Overviews: what they change for your business and what to do: not read yet
- 2. AI search myths: llms.txt, “GEO” and other things you may not need: this guide, not read yet
- 3. Zero-click searches: getting value when people don't click: not read yet
- 4. Using AI to write content safely: what Google actually cares about: not read yet
- 5. Becoming a known brand to Google and AI: get your facts right everywhere: not read yet
- 6. How to get your business mentioned by ChatGPT and other AI assistants: not read yet
0 of 57 guides read · Saved on this device only
New guides by email when they are published.
Unsubscribe in one click.
What to do next
- Open yoursite.com/robots.txt today and confirm it does not block Googlebot or the AI search crawlers you want to allow.
- Ask three AI assistants about your business, list every wrong fact, and correct it on your site and Google Business Profile.
- Put your latest AI-SEO offer through the hourly-cost calculator and the eight-question scorecard before you reply to it.
- Run the free SEO check on your main page: it flags a robots.txt rule that blocks Googlebot and a noindex tag, the two blocks that keep a page out of Google's AI features.
Check your own site
Run our free site check: most of the technical points above, in under a minute.
Run the free checkNext in AI search
AI search
Zero-click searches: getting value when people don't click
Your impressions went up and your clicks went down. Before you panic, find out who called you without ever visiting your site.
24 min readStart reading
See it applied in an illustrative case study
Invented businesses and modelled figures, not client results: how the work in this guide fits into a whole engagement.
- Content Engine · 18 monthsIllustrative+261% trial sign-ups from organic searchFernlog
How a 60-person log analytics SaaS became the source developers' AI answers cite
Log analytics SaaS, 60 staff · Tallinn · Europe
- Content Engine · 13 monthsIllustrative+182% organic sessionsFynbos Actuarial
How a Cape Town actuarial consultancy turned its research into the go-to reference for fund trustees
Actuarial & retirement-fund consultancy · Cape Town · Middle East & Africa