Skip to content

AI search · guide 2 of 6

AI search myths: llms.txt, “GEO” and other things you may not need

Popular AI-search claims checked against what Google and AI companies have actually published: llms.txt, GEO, schema, AI content and “SEO is dead”, with the uncertain parts labelled honestly.

25 min readUpdated 9 Nov 2026

  • 12 questions to test yourself
  • 1 calculator
  • 7-item checklist
On this page9 sections

A new acronym does not mean a new rulebook

This month's email offers “GEO” for €800 a month. Last month's offered “AEO”. Next month it will be something else. Each says your SEO stopped working and that the fix is a new monthly fee.

Some of these offers describe real, useful work. Others rename things you already have, or sell a file that no major search engine has said it reads. From the outside they look the same, which is exactly why they work.

By the end of this guide you will be able to check the claims sellers make most often against what Google, OpenAI and others have actually published, work out what an offer really costs per hour of work, and leave with a one-week list to tick off on your own site. Where nobody knows the answer yet, we say so.

Two things to watch for on the way. There is one small setting, which many owners have never looked at, that can hide a website from Google and AI assistants at the same time. And there is one sentence in AI-SEO sales emails that should end the conversation on its own. Both turn up below.

Start with one guess. Google publishes a page for site owners called “AI features and your website”, covering AI Overviews and AI Mode. How many extra requirements do you think it lists?

Make a guess

Beyond what normal Google Search needs, how many additional technical requirements does Google list for appearing in AI Overviews or AI Mode?

4 extra requirements
0 extra requirements10 extra requirements

Why AI search is such fertile ground for myths

Three things make AI search a myth factory. First, it is new: Google launched AI Overviews in the US in May 2024 and rolled AI Mode out in the US in May 2025, and ChatGPT added its own search feature in late 2024. New things come with few published facts.

Second, the answers are hard to measure. The same question can give a different AI answer each time, and most tools cannot tell you exactly how often you were cited. When results are fuzzy, almost any tactic can be made to look as if it worked.

Third, there is money in confusion. A business owner who believes the rules changed overnight is easier to sell a new package to. That does not make every seller dishonest. It does mean the burden of proof is on the claim, not on you.

So you need one test you can apply in ten seconds. Try it on four sources before reading on.

Quick quiz

You want to know what Google's AI Overviews need from your website. Which source can you actually hold to its word?

Pick one. Your first answer is the one that counts.

llms.txt: what it is, and who actually reads it

llms.txt is a proposal published in September 2024 by Jeremy Howard of Answer.AI. The idea: put a plain Markdown file at yoursite.com/llms.txt with a short summary of your site and links to its most useful pages, so a language model can find the good stuff without wading through menus and cookie banners.

It is a sensible idea for some sites. Software companies with large documentation sites publish llms.txt files so that coding assistants can read their docs, and some AI developer tools use them that way.

What llms.txt is not is a standard that search engines have adopted. It has not been agreed by any standards body, Google representatives have said publicly that Google Search does not use it, and the major AI companies' crawler documentation talks about robots.txt, not llms.txt, when it explains how to control access. Google's own page on AI features lists no such file.

So: adding one does no harm and takes half an hour. Paying a monthly fee for it, or treating it as the reason you will or will not appear in AI answers, is not supported by anything the search companies have said.

Now compare it with the two files that do matter. One of them holds the setting promised at the start. Guess which before you look.

Quick quiz

One line in one of these files can hide your whole site from Google and AI assistants at once. Which file?

Pick one. Your first answer is the one that counts.

Side by side

Questionrobots.txtsitemap.xmlllms.txt
What it doesTells crawlers which parts of the site they may fetchLists the pages you want crawledSummarises the site for language models
StatusAn internet standard (RFC 9309, 2022)A long-established protocol Google documentsA proposal from 2024
Does Google Search use it?YesYesGoogle has said it does not
Matters for AI crawlers?Yes: OpenAI, Anthropic, Perplexity, Google and others document their crawler names for itHelps them find pages, like any crawlerNot documented as a factor by the major assistants
Worth your time?Check it todayHave one; most platforms make it for youOptional, harmless, low priority
A leftover “Disallow: /” is common after a redesign, when the site was hidden on purpose while it was being built and nobody removed the line at launch.

“GEO” and “AEO”: new work or a new label?

“GEO”, generative engine optimisation, comes from a 2023 research paper by researchers at Princeton University and other institutions. They tested ways of rewriting content in an experimental setup and found that changes such as citing sources, adding quotations and including statistics made content more visible in AI-generated answers.

Read that list again: cite your sources, quote people who know, use real numbers. That is also what good writing for humans looks like. The research is real; the leap from it to “your SEO is obsolete, buy GEO” is not in the paper.

“AEO”, answer engine optimisation, is older. It was used in the era of featured snippets and voice assistants for writing that answers a question directly. Again, a reasonable habit with a new name.

There is some genuinely new work in AI search: checking that AI crawlers are not blocked, testing what assistants say about you, and making sure your business facts match everywhere they are listed. Most honest providers will tell you that this sits on top of ordinary SEO, not instead of it. The guide on getting mentioned by ChatGPT and other AI assistants walks through each of those checks step by step. First, try translating the jargon yourself.

Test yourself

Think of your answer, then flip the card.0 of 5 flipped

Quick quiz

Run the pitch through your hype meter. Which line is the only one describing work clearly worth paying for?

Pick one. Your first answer is the one that counts.

Does schema markup get you into AI answers?

Schema markup, or structured data, is code that labels facts on your page for machines: this is the business name, these are the opening hours, this is the price. It has real uses. Google uses it for features such as review stars, product prices and event listings in normal results.

For AI features, Google's guidance is plain: there are no additional requirements, so no special structured data is needed. Where you do use structured data, it should match what is visibly on the page, the same rule that has always applied. If you want to know which types are worth adding, the guide on schema markup covers the handful a small business actually needs.

The FAQ story is a useful warning. Years ago, FAQ markup could make your result take up more space in Google. In August 2023 Google limited FAQ rich results to well-known, authoritative government and health websites. Plenty of businesses had pasted FAQ markup onto every page for that effect, and it simply stopped showing for them. Chasing a format because it works today is fragile; describing your business accurately is not.

What would you do?

Your developer's suggestion

This is an illustrative example. You run a small accounting firm with 30 pages. Your developer offers to add FAQ markup to every page, with five invented questions each, “so AI tools pick us as the answer”. It would cost two days of work.

What do you do?

Is AI-written content penalised?

Many owners have been told that Google can spot AI writing and punishes it, so they either avoid AI tools entirely or hide that they use them. Before the answer, test your instinct on four pages.

Quick quiz

Which of these pages is the kind Google's spam policies target?

Pick one. Your first answer is the one that counts.

So the answer is: not for being AI-written. In February 2023 Google published its guidance on AI-generated content: it rewards high-quality content however it is produced, and what breaks its spam policies is using automation to generate content mainly to manipulate rankings.

In March 2024 Google sharpened this with the scaled content abuse policy you just met. The question Google asks is not “was this made by AI?” but “why does this page exist, and does it help anyone?”

In practice, the risk is not the tool. It is the shortcut the tool makes easy: fifty near-identical city pages, a blog post a day that says what every other post says, a product description that invents features. The guide on using AI to write content safely goes deeper; here is the difference at a glance.

Before and after

Two pages written with AI help (illustrative)

Shortcut

“Boiler Repair in Varna: Your Trusted Experts. Looking for boiler repair in Varna? Our expert team provides top-quality boiler repair services in Varna at competitive prices. Contact us today for all your boiler repair needs in Varna.”

The same page exists for 40 other towns, with only the town name changed.

Assisted

“We repair Vaillant, Bosch and Ariston boilers in Varna and the villages up to Aksakovo. A call-out is €40, most repairs take one visit, and we carry common parts in the van. The most frequent fault we see in winter is low pressure: here is how to check yours before you call.”

AI helped draft and tidy it; the owner added the brands, prices, area and the winter tip from real jobs.

  • Both used AI. Only the second contains anything a reader could not get from any other page.
  • Specific facts, real prices and first-hand tips are what make a page worth citing, by Google or by an AI assistant.
  • The first page is the pattern Google's scaled content abuse policy describes, whoever or whatever wrote it.

Is SEO dead?

This one has been announced many times since search engines began. The current version says that because AI answers questions directly, nobody will click on websites any more.

Some of the worry is real. AI answers do settle some questions on the results page, and for simple facts that can mean fewer clicks. Forecasts exist, such as Gartner's prediction in early 2024 that traditional search engine volume would fall 25% by 2026, but a forecast is not a measurement, and nobody outside the search companies has complete figures. If your clicks have fallen while impressions rose, the guide on zero-click searches, next in this series, shows how to find the calls and visits that never show up as clicks.

What is not in doubt is how the AI systems find their sources. Google's AI features draw on pages in Google's index. ChatGPT search, Perplexity and Claude find pages through web search and fetch them with their own crawlers. Each of them needs pages that can be crawled, understood and trusted, which is what SEO has always been about. The work is shifting in emphasis, not disappearing. Test yourself on the claims you have probably been sent.

Myth or fact?

Six claims you have probably been sent

For each one: myth or fact?0 of 6 answered
  1. 1.Blocking Google-Extended in robots.txt will remove you from Google Search.

  2. 2.To be shown as a link in AI Overviews, a page has to be indexed and eligible for a snippet in normal Search.

  3. 3.If you block OpenAI's search crawler, ChatGPT search will generally not show your pages in its answers.

  4. 4.AI tools only cite the biggest brands, so small businesses should not bother.

  5. 5.Google has confirmed that pages written for AI get a ranking boost in AI Mode.

  6. 6.Nobody outside the AI companies can see exactly how their systems choose sources.

Judge any AI-SEO offer in two minutes

Strip away the acronyms and the list of things that genuinely help is short and unglamorous. Let crawlers in. Get your pages indexed. Say clearly who you are, what you do, where and for how much. Publish things only you can say. Keep your business facts identical on your site, Google Business Profile and the main directories. Earn mentions from other real sites.

None of that is new, and none of it can guarantee you a place in an AI answer, because nobody controls those answers except the companies that run them. What it does is remove the reasons an AI system, or Google, would have to skip you.

So the question for any offer is: how much of that list does it actually do, and at what price? Put the numbers from the last proposal you received into the calculator, with the hourly rate you would pay a skilled freelancer as your yardstick, then score it. It works on any proposal, including ours.

Try the numbers

What does the offer cost per hour of real work?

€800 a month
4 hours
€40 an hour
12 months

Price per hour of described work

€200 per hour

Paid above that rate each month

€640

What you commit to before you can stop

€9,600

Your own numbers, not a judgement of any provider. A big monthly gap is not proof of a bad deal, but it is the question to ask: what else does the fee cover, and can you check it? Count only tasks they name and you could verify; if they cannot name the hours at all, that answers the first question in the scorecard below.

Score yourself

Is this AI-SEO offer worth it?

  1. They describe, in plain words, the actual work they will do each month

  2. The work would still help your Google results if AI search disappeared tomorrow

  3. They do not guarantee placement in ChatGPT, AI Overviews or any ranking

  4. They start by checking that Google and AI crawlers can reach your site

  5. No deliverable is only a file (such as llms.txt) or a renamed report

  6. They explain how they will measure results and admit what they cannot measure

  7. The price matches the hours of real work they describe

  8. You can stop without a long lock-in contract

0 / 16

0 of 8 answered. Answer them all to see your result.

What to do this week instead of buying an acronym

Here is the same list turned into tasks, ordered so the first ones protect everything after them. If the robots.txt check is the only thing you do today, you have already done the most important AI-search work there is. Most of it takes an afternoon, and you can tick it off as you go.

The free SEO check linked at the end of this guide does the first and third checks for you on any page: it reads your robots.txt and tells you if it blocks Googlebot, and it flags a noindex tag. It does not test the AI crawlers by name, so for those, open the file and look for their names; it is rarely more than a dozen lines. If a page turns out not to be indexed, the guide on why pages are missing from Google lists the usual causes and fixes.

Checklist

Your one-week list, in this order

0 of 7 done

Saved on this device only

Questions people ask

Should I add an llms.txt file anyway?

If it costs you nothing, it does no harm, and it may help AI tools that do read it. Just do not expect it to change whether Google or the main assistants cite you, and do not pay for it as a service.

Should I block AI crawlers to protect my content?

It is your choice, and the controls are separate. You can block training crawlers such as GPTBot and ClaudeBot, or opt out of Gemini training with Google's Google-Extended token, and still allow search crawlers such as OAI-SearchBot, Claude-SearchBot and Googlebot. Blocking the search crawlers means those assistants cannot read your current pages when answering.

How do I know if AI assistants mention my business?

Ask them the questions your customers would ask, with search turned on, and note who is named and which sources are linked. Repeat monthly with the same questions. Answers vary between runs, so look for patterns over several months, not single results.

You now have the yardstick: zero extra requirements, one file that can hide you, and one sentence that ends a sales call. With the myths cleared away, the useful question is what does make a difference.

Two guides pick up from here. Zero-click searches, next in this series, shows how to count the value of searches that end on Google without a visit. And the guide on getting mentioned by ChatGPT and other AI assistants turns the honest list above into four levers you can pull, starting with the crawler names you just checked.

You reached the end

Your summary

This guide has 12 quick questions

Try them to see how much you already know. Your first answer is the one that counts.

Start with the first question

New guides by email when they are published.

Unsubscribe in one click.

What to do next

  1. Open yoursite.com/robots.txt today and confirm it does not block Googlebot or the AI search crawlers you want to allow.
  2. Ask three AI assistants about your business, list every wrong fact, and correct it on your site and Google Business Profile.
  3. Put your latest AI-SEO offer through the hourly-cost calculator and the eight-question scorecard before you reply to it.
  4. Run the free SEO check on your main page: it flags a robots.txt rule that blocks Googlebot and a noindex tag, the two blocks that keep a page out of Google's AI features.

Check your own site

Run our free site check: most of the technical points above, in under a minute.

Run the free check

AI search

Zero-click searches: getting value when people don't click

Your impressions went up and your clicks went down. Before you panic, find out who called you without ever visiting your site.

24 min readStart reading

See it applied in an illustrative case study

Invented businesses and modelled figures, not client results: how the work in this guide fits into a whole engagement.

See all 89 studies