Skip to content

How search really works · guide 1 of 5

How Google decides who ranks first, in plain English

Crawling, indexing and ranking explained as three gates every page must pass, so you can tell which one your site is stuck at before you spend a cent.

27 min readUpdated 9 Nov 2026

  • 9 questions to test yourself
  • 1 calculator
On this page10 sections

Your page can be perfect and still never be shown

Many sites that “don't rank” are not losing to competitors at all. They were never let into the contest. Before Google puts any page in order, it has to pass two earlier gates, and plenty of pages get stuck at the door.

By the end of this guide you will be able to look at your own site and say which gate it is waiting at: found, kept or ranked. That one answer tells you whether you need a quick technical fix, better pages, or patience, before you spend a cent on anyone.

Along the way you will meet a single tick box that can hide an entire new site from Google, and find out why searching for your business on your own phone gives you the wrong answer. Start with a quick puzzle.

Everything here follows Google's own How Search Works pages and Search Central documentation. Example businesses are made up and labelled.

Quick quiz

A bakery publishes a new “Wedding cakes” page. Three weeks later it does not show up on Google for anything, not even its exact title. What is the most likely reason?

Pick one. Your first answer is the one that counts.

Three gates: discovered, indexed, ranked

Google describes Search as three stages. First it crawls: an automated program called Googlebot finds pages and downloads them. Then it indexes: it reads each page, works out what it is about and decides whether to store it in the Google index, a huge database of pages. Finally it serves results: when someone searches, it picks the pages from the index that best answer that search and puts them in order.

The important part is in Google's own wording: not every page makes it through each stage. Google's documentation also says plainly that it does not guarantee it will crawl, index or show any page, even one that follows all its rules. So the useful question is not “why am I not first?” but “which gate is my page waiting at?”

Put it in order

Put the journey of a new page in order, from the moment it is published to the moment a customer sees it

Use the arrows to move each item, then check.

  1. Google learns the page's address, usually from a link or a sitemap
  2. The searcher sees your result and decides whether to click
  3. Googlebot visits and downloads the page
  4. Google reads it and decides whether to keep it in the index
  5. Someone searches, and Google picks and orders indexed pages

Gate 1: how Google finds your page

Google has no master list of every website. It finds new pages mostly by following links from pages it already knows, the way you might find a shop because a friend mentioned it. Google calls this URL discovery.

This is why the most common gate-one problem is an orphan page: a page that exists but that nothing links to. It is not in the menu, no other page mentions it, and it is not in the sitemap. To Google, it barely exists.

There are four roads that lead Google to a page, and you control most of them.

  • Links from your own pages: the menu, the footer and links inside your text. These are free and you control them completely.
  • Links from other websites: a supplier, a local directory, a newspaper mention. Each one is another road to your page.
  • A sitemap: a file listing the pages you want found, which you can submit in Google Search Console. It helps discovery, but Google says it does not guarantee every listed page is crawled or indexed.
  • The URL Inspection tool in Search Console, where you can ask Google to crawl one specific page. Useful for a key new page, but there is a daily limit and a request does not guarantee the page will be indexed.

Once a page is discovered, Googlebot has to be able to fetch it. If the server is down or very slow when Googlebot calls, or a file called robots.txt (a short list of rules at the root of your site telling crawlers where not to go) blocks the folder, the visit fails or never happens. Google also renders the page much like a recent version of Chrome would, so content that only appears after scripts run usually gets seen, but not always as quickly.

Make a guess

You may hear that Google only crawls a limited “crawl budget” of pages per site. From roughly how many pages does Google say a site whose content changes about weekly should start worrying about it?

0 pages
0 pages2,000,000 pages

Anything within ±250,000 pages counts as right.

Gate 2: why Google may choose not to keep a page

Being crawled is not being indexed. After downloading a page, Google analyses its text, its title, its images and their alt text (a short written description of an image), and then decides whether the page is worth storing. Google's documentation is clear that indexing is not guaranteed, and that not every page it processes is kept.

Google names the usual reasons: the content is low quality, a robots meta rule (a line in the page's code) tells Google not to index it, or the way the site is built makes indexing difficult. In practice, for small businesses, three causes come up again and again.

First, and this is the tick box we promised: a leftover noindex instruction. Web designers often switch it on while a site is being built, so the half-finished version stays out of Google, and then forget to switch it off at launch. Google crawls every page, obeys the instruction and keeps none of them. One setting can hide an entire site, and the fix takes a minute once you know to look. It is not only new sites: in one of the case studies on this site, a chimney sweep with eight crews found that a plugin had set 24 of its 27 service pages to noindex.

Second, near-duplicates. When several pages are very similar, Google groups them and picks one as the canonical, the main version it shows, and treats the others as copies. Ten town pages that differ only by the town's name often end up with one indexed page and nine copies. The Duplicate content and canonicals guide shows how to tell Google which version you prefer.

Third, thin pages. A page with two sentences and a stock photo gives Google little reason to keep it when millions of pages already answer the same question better.

Myth or fact?

Four beliefs about getting into Google's index

For each one: myth or fact?0 of 4 answered
  1. 1.Submitting a sitemap in Search Console makes sure every page in it gets indexed.

  2. 2.Blocking a page in robots.txt is a reliable way to keep it out of Google's results.

  3. 3.If several of your pages are almost identical, Google may show only one of them.

  4. 4.A page Google crawled but did not index may still be indexed later without you doing anything.

Gate 3: how Google decides the order

Only now, with your page safely in the index, does the contest begin. When someone searches, Google looks through its index for the pages that best answer that particular search. Its How Search Works pages describe the main ingredients in five groups.

  • The meaning of the search. Google works out what the person actually wants: “cake shop open now” wants a place nearby, “how to store a wedding cake” wants instructions. A page aimed at the wrong intent rarely wins.
  • Relevance. Does the page actually contain information about this? The simplest signal is that it uses the words people search for, in headings and in the text, but Google also looks beyond exact words to what the page is about.
  • Quality. Google tries to favour content that shows expertise, experience, authoritativeness and trustworthiness. One signal it names is whether other prominent websites link to or refer to the content.
  • Usability. All else being roughly equal, Google prefers pages that load well, work on phones and are safe to use. It also says relevance comes first: a slower page with the best answer can still beat a fast page with a weak one. The Core Web Vitals guide explains what “loads well” means in Google’s own measurements.
  • Context and settings. Who is searching, where, in which language and on which device. The same words can deserve different answers.

Before and after

The bakery from the opening puzzle, two ways (a made-up example)

Indexed, but answers nothing

Passion for baking since 1998. Every creation is made with love and the finest ingredients. Taste the difference.

Indexed and relevant

Wedding cakes in Plovdiv, made to order

Tiered wedding cakes for 40 to 200 guests, from €180. Free tasting on Saturdays at our Kapana shop. Order at least six weeks ahead; we deliver and set up at venues across Plovdiv.

  • The meaning of the search: someone typing “wedding cakes Plovdiv” wants to order, so the page says what, for how many, how much and how to order.
  • Relevance: the words a customer actually types now appear in the heading and the text. The first version could rank for almost nothing, because it contains almost nothing a customer searches for.
  • Quality and trust: concrete prices, a real address and a clear process are things a reader can check. Vague praise is not.

Quick quiz

A question many owners quietly wonder about: does paying for Google Ads improve where your site appears in the regular, unpaid results?

Pick one. Your first answer is the one that counts.

Why the results you see are not the results your customer sees

Now the second thing we promised: why your own phone gives you the wrong answer. Many owners search for their business every morning and worry when it moves. Google's documentation explains that results depend on the searcher's location, language and device, among other things. Its example is someone searching for bicycle repair shops seeing different results in Paris than in Hong Kong.

For a local business this effect is large. Two customers typing exactly the same words, one on a phone two streets from you and one on a laptop across town, can see a different map, different businesses and a different order. Switch between the two below.

This means checking your position from your own phone tells you very little. Search Console shows your average position across all the people who actually saw you, which is a far more honest number. And the map pack, the directories and the ads around your listing all change what a customer notices first; the Anatomy of a Google results page guide labels every part.

What the searcher sees

Same words, two searchers

emergency plumber
  1. Lozenets Pipe Repair

    4.8(112)Plumber

    Rated 4.8 out of 5 from 112 reviews

    0.6 km · Open 24 hours

  2. Rapid Flow Plumbing

    4.6(58)Plumber

    Rated 4.6 out of 5 from 58 reviews

    1.4 km · Open 24 hours

  3. Vitosha Heating and Water

    4.9(31)Plumber

    Rated 4.9 out of 5 from 31 reviews

    2.1 km · Closes 20:00

  1. lozenetspipes.example › emergency

    Emergency plumber in Lozenets, Sofia | Lozenets Pipe Repair

    Burst pipe or no hot water? We cover Lozenets and nearby areas day and night. Call and get an arrival time on the phone.

On a phone nearby, the map pack comes first and the closest open businesses lead. All businesses here are made up.

Illustration: a simplified mock-up of a results page, not a real search.

Which gate does each problem block? Guess, then flip

Here are eight common problems on small-business sites. For each one, decide which gate it blocks before you flip the card: discovery and crawling, indexing, or ranking. Keep a tally of how many you call right. This is the whole skill of diagnosing a site, and it is what separates a useful audit from a list of scary warnings.

Test yourself

Think of your answer, then flip the card.0 of 8 flipped

Notice the pattern: most of these eight are fixed with a link, a setting or a rewrite of one page, not with months of work. That is the payoff of thinking in gates. A site stuck at gate one or two can sometimes move quickly once the blocker is gone; a site at gate three is in the slower, ongoing work of being the best answer.

What you control, what you influence and what you do not

Once you see the gates, you can sort SEO work by how much power you actually have over it. This is also the best defence against anyone who promises results they cannot deliver.

Side by side

GateYou controlYou influenceYou do not control
1. Discovery and crawlingYour menu and internal links, your sitemap, robots.txt, keeping the site onlineLinks from other websites to yoursHow often Google chooses to come back
2. IndexingNoindex settings, one clear page per topic, enough real content on each pageWhich duplicate Google picks as the main versionWhether Google keeps a given page
3. RankingHow well each page answers the search, your titles, phone usabilityReviews, mentions and links that build trustCompetitors, the searcher's location and device, Google's updates
Notice where a guaranteed position would have to come from: the right-hand column. That is why nobody honest can promise one.

Find out which gate your site is at today

You do not need to pay anyone to answer this. You need Google Search Console, Google's free tool for site owners, and about 20 minutes. If you have never set it up, adding and verifying your site is the first step, and the Search Console guide walks you through it.

No Search Console yet, or want a quick first answer? Paste your main page into our free SEO check. It reads the page the way a crawler does and flags a noindex tag, a robots.txt block, a missing sitemap or title: most of the gate-one and gate-two blockers in this guide.

Step by step

The 20-minute gate check

Step 1 of 5

Search for your business name

Type your exact business name into Google. If your site does not appear at all, even below directories and social profiles, suspect gate one or two for the whole site rather than a ranking problem.

Try the numbers

How many of your pages has Google kept?

Count the pages you want customers to find (home, each service or product, contact, about), then copy the indexed number from the Page indexing report.

15 pages
9 pages

Pages to look up

6 pages

Share kept

60%

Your own count, not a target: Google does not promise to index any page. If pages are missing, open the Not indexed list in the same report: each page there has a reason, and the reason tells you its gate. If the indexed number is much higher than your count, Google may be keeping pages you never meant to publish, such as tag pages, test pages or old addresses, which is worth a look too.

Score yourself

Where is your site right now?

Answer for the one page that matters most to your business, usually your main service or product page. The questions run in gate order, so your first No is the gate to work on.

  1. The page can be reached from your menu or from links on other pages of your site

  2. Your site is verified in Search Console and has a sitemap that reads as successful

  3. URL Inspection says “URL is on Google” for this page

  4. The page has its own substantial text, not shared with other pages on your site

  5. In the Performance report, this page shows impressions for searches a customer would really type

0 / 5

0 of 5 answered. Answer them all to see your result.

Your first fix: what would you do?

Diagnosing the gate is half the job. The other half is not wasting effort once you know. Try this situation first; the five traps after it are worth keeping for the next time you are tempted.

What would you do?

A dog groomer's main service page (a made-up example)

Mila runs a dog grooming salon. Her “Grooming prices” page has 60 words and a photo. URL Inspection in Search Console says “Crawled – currently not indexed”. Her other pages are indexed, and the price page is linked from the menu.

Which gate is she stuck at, and what is the best first move?

Five traps that waste owners' time and money:

  • Rewriting a page for keywords when it is not even indexed. Check the gate first; effort at gate three is wasted on a page stuck at gate two.
  • Pressing “Request indexing” over and over. Once is enough. Repeating it does not move you up any queue.
  • Judging success by searching from your own phone. Use Search Console's numbers instead.
  • Deleting robots.txt rules or noindex settings in a panic without knowing why they were added. Some pages, such as a checkout or a thank-you page, are meant to stay out.
  • Paying for “guaranteed first page” packages. Google does not sell organic positions, so nobody can resell one.

You now know which gate your page is waiting at, and that is the question most owners skip. The next one is the one everybody asks: once the blocker is fixed, how long until it shows? Nothing changes in Google until Googlebot visits the page again, and gate-three work is measured in months, not days.

That is exactly where the next guide in this series picks up. How long does SEO take? starts by comparing a made-up locksmith, hidden from Google by the same leftover setting you met here, with a tax adviser facing twenty established rivals, then walks through what usually moves first, month by month, and how to tell real progress from waiting.

You reached the end

Your summary

This guide has 9 quick questions

Try them to see how much you already know. Your first answer is the one that counts.

Start with the first question

New guides by email when they are published.

Unsubscribe in one click.

What to do next

  1. Answer the five “Where is your site right now?” questions for your main service page, and note your first No: that is the gate to work on.
  2. Open Google Search Console and run URL Inspection on that page; if it says “Excluded by noindex tag”, ask whoever built the site to switch the setting off, and check robots.txt while they are there.
  3. Run our free SEO check on your homepage to catch noindex tags, missing titles and other gate-one and gate-two blockers in one go.

Check your own site

Run our free site check: most of the technical points above, in under a minute.

Run the free check

How search really works

How long does SEO take? An honest timeline

The phone rings last. Here is what moves first, month by month, and the one mistake that can reset the clock.

25 min readStart reading

See it applied in an illustrative case study

Invented businesses and modelled figures, not client results: how the work in this guide fits into a whole engagement.

See all 83 studies