Skip to content
Get Ranked On TopIllustrative case studySignal plan

A fictional business. The figures are a modelled scenario, not a client result.

Granitefold Insurance · 35-agent home & auto insurance brokerage · Burlington

How a Vermont insurance brokerage stopped its quote tool swallowing Google’s crawl

New pages indexed in 4 days, not 74, and +62% organic quote requests

11 months · North America · Signal plan · Crawl budget · URL parameter handling · Log-file analysis · XML sitemaps

Month 1
12%Stormy
Month 11
90%Fair

of Googlebot’s requests reach real pages

Crawl barometer · Burlington

01The challenge

A quote tool that made its own weather.

Its quote tool spun out endless parameter URLs that swallowed the crawl budget, so new coverage pages took months to reach Google’s index.

Granitefold’s quote tool was the best thing on the site for customers and the worst thing for Googlebot. Every ZIP code, vehicle, coverage level and deductible was a crawlable link, and every visit added a session ID. Googlebot kept following the combinations and rarely reached the pages that mattered: new guides on ice-dam claims, teen drivers and lake cottages sat undiscovered for weeks, then waited weeks more to be indexed.

One quote form, multiplied

  1. 1,180ZIP codes
  2. 30model years
  3. 46vehicle makes
  4. 5coverage levels
  5. 4deductibles

= 32,568,000

possible quote URLs

Then a fresh session ID on every visit, and the count has no ceiling.

What Googlebot could find · log scale

02Diagnosis

Reading the pressure before touching the glass.

Log files

81%

of Googlebot requests hit parameter URLs

Twelve weeks of server logs, 538,000 Googlebot hits. Session IDs alone took almost a quarter of the crawl; the 148 pages Granitefold actually wanted ranked shared 12%.

Finding 1 of 4

Discovery

412,000

parameter URLs known to Google

Search Console’s crawl stats and page indexing report listed hundreds of thousands of quote and calendar URLs as crawled or discovered, against 148 real pages.

Finding 2 of 4

Indexing

74

median days to index a new page

Only 63 of the 148 coverage and local pages were indexed. The ice-dam claims guide, published in December, was first crawled in late February.

Finding 3 of 4

Internal links

2,340

crawlable links into the quote tool

A “Quote in your ZIP” footer widget, vehicle shortcuts and a booking calendar with an endless “next month” link put GET links on every template.

Finding 4 of 4

Where Googlebot’s requests went

Month 1
  • Real pages 12%
  • Parameter URLs 81%
  • Redirects & errors 7%
Month 11
  • Real pages 90%
  • Parameter URLs 6%
  • Redirects & errors 4%

03The strategy

Seal the leaks, then let the pressure rise.

Built on our Signal plan: the technical engagement for sites where crawl, rendering and architecture decide the outcome.

Search Console’s URL Parameters tool was retired in 2022, so every control had to live on the site itself: robots.txt, forms, links, canonicals and sitemaps.

  1. I

    Crawl budget

    Robots.txt rules for the quote tool and the calendar, written only after the logs showed which families wasted the crawl. A disallow stops the fetch; noindex would not have, because Googlebot has to crawl a page to read it.

    User-agent: *
    Disallow: /quote?
    Disallow: /*?*date=
  2. II

    URL parameter handling

    The quote form now submits by POST, ZIP and vehicle choices live inside the form, sort links became buttons and internal tracking tags were stripped. Session IDs moved to a cookie, and old session URLs 301 to their clean twins.

    <form method="post" action="/quote">
    <link rel="canonical" href="/coverage/home/">
  3. III

    Log-file analysis

    Daily log ingestion for the whole engagement. Every sealed trap was checked within a week: requests to that family falling, requests to coverage pages rising. That reading drives the needle in the barometer below.

    66.249.66.1 "GET /quote?zip=05401&sessionid=8f3a2c91"
      200 "Googlebot/2.1"
  4. IV

    XML sitemaps

    One sitemap per section (coverage, local offices, guides) holding only canonical 200 URLs with honest lastmod dates, plus links from a coverage hub, so a new page was found within a day of going live.

    <loc>https://granitefold.example/coverage/auto/teen-drivers/</loc>
    <lastmod>2026-04-14</lastmod>

04The barometer

Close the traps one by one and watch the needle swing from Stormy to Fair.

Each switch seals one family of parameter URLs, in the order the work was done. The blued needle reads the share of Googlebot’s requests that reach real pages; the brass hand stays where it stood in month 1.

Reading

Stormy

Brass hand · month 1 · 12%

Crawl on real pages
12%
← 12%
Crawl on parameter URLs
81%
← 81%
Requests a day to real pages
770
← 768
Days to index a new page
74
← 74
Crawl mix
  • Real pages
  • Parameter URLs
  • Redirects & errors

Stormy: 12% of the crawl reaches real pages, and a new page is indexed in about 74 days.

  • Session IDs in links24% of crawl

    /quote?zip=05401&sessionid=8f3a2c91

    FixSessions moved to a cookie; every old sessionid URL now 301s to its clean twin, so Googlebot learns to drop them.

  • ZIP-prefilled quote links19% of crawl

    /quote?zip=05452

    FixThe “Quote in your ZIP” widget became a form that posts, and /quote? is disallowed in robots.txt.

  • Vehicle combinations14% of crawl

    /quote/auto?year=2019&make=subaru&model=outback

    FixYear, make and model are picked inside the form, never offered as crawlable links.

  • An endless booking calendar9% of crawl

    /agents/book?date=2031-02-01

    FixThe calendar stops at 60 days, “next month” became a button and date= URLs are disallowed.

  • Sorted and paged reviews8% of crawl

    /reviews?sort=oldest&page=212

    FixSort options became buttons; review pages keep plain ?page= URLs with self-referencing canonicals.

  • Tracking tags on internal links7% of crawl

    /coverage/home/?utm_source=newsletter&ref=footer

    FixUTM and ref tags stripped from internal links; canonicals point at the clean URL.

05Month by month

11 months from storm to fair.

Barometer reading by month
StormyRainChangeFairM1M2M3M4M5M6M7M8M9M10M1112%90%
Median days to index pages published that month
74M171M258M339M424M515M610M77M85M94M104M11
  1. Months 1–2

    Read

    Twelve weeks of logs, Search Console crawl stats, a full site crawl and a parameter inventory: 41 parameter names in six families.

  2. Months 3–4

    Seal

    Quote and vehicle links turned into forms, session IDs into cookies, and robots.txt rules for the quote tool and the calendar.

  3. Months 5–6

    Clean

    Tracking tags stripped from internal links, sort links turned into buttons, canonicals checked on every template and 1,960 redirect chains collapsed to one hop.

  4. Months 7–8

    Signpost

    Sectioned XML sitemaps with honest lastmod dates, a coverage hub linking every guide, and new coverage guides published into a calm crawl.

  5. Months 9–11

    Hold

    Weekly log checks so no new parameter could open a fresh leak; the needle held at Fair through the winter quote season.

06Results

Fair weather, and the quote requests followed.

Organic quote requests

+62%

410 → 664 organic quote requests a month

Crawl spent on parameter URLs

81% → 6%

Share of Googlebot requests, month 1 → month 11

Median days to index a new page

74 → 4

Median for pages published that month

Coverage pages indexed (of 148)

63 → 146

2 still waiting to be indexed

Modelled scenario for a fictional business — not a client result. Rankings and traffic are never guaranteed; see our Terms.

Organic quote requests per month
Traps sealed400500600700M1M2M3M4M5M6M7M8M9M10M11664410
148 pages, one square each
  • Indexed at the start 63
  • Indexed during the work 83
  • Not indexed yet 2

Get a forecast like this

Is a filter, a calendar or a quote tool eating your crawl? Start with the Signal plan, or tell us what your logs show.

← All case studies