Keyword Cannibalization: How to Find and Fix It with SERP Data
Michael Carter · · 17 min read

Key takeaways
- Keyword cannibalization means two of your pages compete for the same query and the same intent; two of your URLs in one top 10 that serve different needs are not a problem in themselves.
- In Search Console, click a query in the Queries tab and open the Pages tab to see which URLs Google showed; the chart counts two results from one site as one impression, the Pages tab counts each URL.
- Results pages change within minutes: on November 18, 2026, 5 of 10 US pages shared no site between two checks less than two minutes apart, so count a URL only when it ranks in two checks.
- Google's own documentation held two or more URLs in one top 10 for 2 of 10 SEO keywords (core web vitals, robots.txt) in repeated checks, each URL serving a different need, so nothing to fix.
- Fix by intent: merge and 301 when one page can do the job, rel="canonical" when both URLs must stay, re-target when intents differ; Google does not recommend noindex for choosing a canonical.
Keyword cannibalization is when two or more pages on your site compete for the same query and the same intent, so Google alternates between them or ranks neither as well as one strong page would. Find it in Search Console's Pages tab for one query and in your ranking history, then merge, redirect, canonicalize or re-target.
This guide is for SEO leads, content teams and developers of SEO tools: when two ranking pages are fine, the Search Console steps, a tested Python script for repeated SERP checks, results from 35 searches on November 18, 2026, and a decision tree for the fix.
What keyword cannibalization is (and when it is fine)
Keyword cannibalization is a page-selection problem: more than one of your pages could answer a query, and nothing on your site says which one should. Typical signs: the ranking URL flips from week to week, impressions for one query are split over several URLs, or an old blog post holds the query your product page was built for.
Two of your pages on one results page are not a problem by themselves. In its June 2023 SEO office hours, Google called two results from one domain, the second indented, "host groups" and said "it's a tell that you have more than 1 page that can rank well for a certain query. You might want to consolidate those two pages if possible?" That is a suggestion, not a warning. John Mueller, a Google Search Advocate, wrote on Bluesky in September 2025 (his profile says the posts are his own) that "pages aren't duplicates just because they happen to appear in the same search results page."
Even real duplication is no penalty: "Some duplicate content on a site is normal and it's not a violation of Google's spam policies," says Google's canonicalization documentation.
| What you see | Verdict |
|---|---|
| Two URLs in one top 10, each with its own job | Fine |
| Two URLs with the same intent and similar content | Worth fixing |
| The ranking URL changes week to week | Worth fixing |
| An older or weaker page ranks instead of the one you built | Worth fixing |
Search intent decides the row, so read the results page first, as in our SERP analysis framework.
How to find keyword cannibalization in Search Console
The Performance report lists the pages Google actually showed for a query; the steps are from Google's help page on common Performance report tasks (checked November 18, 2026):
- Open the Performance report for Search results, search type Web, at least the last 28 days.
- In the Queries tab, click the query to filter "all report data to the selected query". For an exact match, add a Query filter with Custom (regex) and
^your keyword$. - Open the Pages tab "to see which URLs were shown", with clicks, impressions and average position for each.
- For flipping, add a Page filter for each URL in turn, with the chart set to Daily: one rising as the other falls means Google alternates.
Mind the counting. The chart is aggregated by property: "if two results from the same site appear for one query, they count as a single impression in the chart total" (Performance report help). The Pages tab counts each URL separately, so its impressions can add up to more than the query's total; our explainer on Search Console's property and page counting walks through Google's example. The report also mixes all countries and devices unless you filter, and data usually arrives two to three days late. To check every query at once, the Search Console API returns rows grouped by query and page (see our overview of Google's official search APIs).
How to find keyword cannibalization in your ranking history
Rank tracking data shows the same overlap from the outside: which of your URLs ranks for each keyword, check after check, in one country. The script below flags two patterns in the organic results:
- Two URLs in the same top 10, in at least two checks.
- A URL swap: your top URL changes between two checks whose pages are otherwise similar, and both URLs rank in at least two checks.
Two checks, because results pages are not stable: in our SERP competitor analysis, 5 of 10 US pages fetched again minutes later had replaced at least five sites, and our test of rankings in four countries found the same. A one-off fetch can show a URL that never ranks again. So the script counts a URL only once it appears in two checks, and compares top URLs only between checks that share at least half their sites (a Jaccard index of 0.5 or more: shared sites divided by all sites in either check).
Step 1: Fetch fresh results with final URLs
You need Python 3 with requests and a BestSERP key in the BESTSERP_API_KEY environment variable. BestSERP, the Google SERP API we run, gives 3,000 free searches, then charges $0.50 per 1,000; the quickstart covers a first request. We tested the script on November 18, 2026 with Python 3.12.10 and requests 2.34.2. Paste the three parts into one file, cannibalization.py.
"""Find keyword cannibalization in SERP data: which of your URLs rank for each keyword, check after check.
export BESTSERP_API_KEY=... # never hard-code the key
python cannibalization.py check keywords.txt --save checks/ # one fresh check per keyword (1 credit each)
python cannibalization.py report keywords.txt example.com --save checks/
Run `check` on a schedule (daily is typical); `report` reads the saved checks only and costs nothing.
Tested 2026-11-18 with Python 3.12.10 and requests 2.34.2.
"""
import argparse
import json
import os
import re
import time
from collections import Counter
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlsplit
import requests
API = "https://api.bestserp.com"
HEADERS = {"X-API-KEY": os.environ.get("BESTSERP_API_KEY", "")}
def search(q, gl="us", hl="en"):
"""A fresh top 10 with final URLs: polls 202 and linksPending (free), retries 429/502/503."""
body = {"q": q, "gl": gl, "hl": hl, "num": 10, "noCache": True}
for attempt in range(5):
r = requests.post(f"{API}/search", headers=HEADERS, json=body, timeout=130)
rid = r.json().get("requestId") if r.status_code in (200, 202) else None
deadline = time.time() + 180
# 202 = still searching; linksPending = some links are still Google redirects. Poll until both are done.
while rid and (r.status_code == 202 or r.json().get("linksPending")) and time.time() < deadline:
time.sleep(5)
r = requests.get(f"{API}/requests/{rid}", headers=HEADERS, timeout=130)
if r.status_code == 200:
return r.json()
if r.status_code in (429, 502, 503): # rate limit, failed or busy: failed searches are refunded
time.sleep(int(r.headers.get("Retry-After", 10 * (attempt + 1))))
continue
r.raise_for_status() # 400, 401, 402: fix the request, do not retry
raise RuntimeError(f"{q!r}: still failing after 5 attempts (HTTP {r.status_code})")
def slug(q):
return re.sub(r"[^a-z0-9]+", "-", q.lower()).strip("-")
def check(keywords, gl, hl, save):
"""One check = one timestamped folder with a JSON file per keyword."""
folder = Path(save, datetime.now(timezone.utc).strftime("%Y-%m-%dT%H%MZ"))
folder.mkdir(parents=True, exist_ok=True)
def one(q):
serp = {**search(q, gl, hl), "fetchedAt": datetime.now(timezone.utc).isoformat(timespec="seconds")}
Path(folder, slug(q) + ".json").write_text(json.dumps(serp, indent=1), encoding="utf-8")
return q, serp
with ThreadPoolExecutor(max_workers=4) as pool: # 4 at a time stays well under 60 searches a minute
for q, serp in pool.map(one, keywords):
print(f"{q:<30} from {serp.get('ipCountry', '?')} unresolved links: {serp.get('unresolvedLinks', 0)}")
noCache: true asks for a fresh page instead of the one-hour cache. The first answer usually says linksPending: true: some result links are still Google redirects, and the final URLs arrive by polling /requests/{requestId}, usually within 30 seconds (link resolution). The script polls until linksPending is false and matches on the final page URL; all 35 of our responses ended with no unresolved link. A 429 waits for Retry-After, and a 502 or 503 is retried; failed searches are refunded.
Step 2: Match your URLs
def page(item):
"""(host, page URL) from the final link, without scheme, www or #fragment. URL None if still a redirect."""
u = urlsplit(item["link"])
host = (u.hostname or "").removeprefix("www.")
if host == "google.com" and u.path.startswith("/goto"): # not resolved yet: fall back to `domain`
return (item.get("domain") or "?").removeprefix("www."), None
return host, host + u.path.rstrip("/") + (f"?{u.query}" if u.query else "")
def ours(organic, domain):
"""{page URL: position} for results on the domain or its subdomains."""
found = {}
for item in organic:
host, url = page(item)
if host == domain or host.endswith("." + domain):
found.setdefault(url or f"{host} (redirect)", item["position"])
return found
def jaccard(a, b):
return len(a & b) / len(a | b) if a | b else 1.0
def load(keywords, save, gl):
"""{keyword: [check, ...]} in time order. Checks that ran from another country are left out."""
out = {q: [] for q in keywords}
for folder in sorted(p for p in Path(save).iterdir() if p.is_dir()):
for q in keywords:
f = Path(folder, slug(q) + ".json")
if f.exists():
serp = json.loads(f.read_text(encoding="utf-8"))
if serp.get("ipCountry", gl) != gl:
print(f"skipped {folder.name} {q!r}: ran from {serp['ipCountry']}, not {gl}")
continue
organic = serp.get("organic", [])
out[q].append({"time": folder.name, "organic": organic, "sites": {page(i)[0] for i in organic}})
return out
page() turns each final link into host, path and query string, and ours() keeps results on your domain or any subdomain, like a Search Console Domain property. The final URL matters: for "robots.txt", Google's robots.txt file ranked fifth with no domain field, and only its final URL put it on google.com. load() skips any check whose ipCountry differs from gl, because it ran from another country.
Step 3: Decide and report
def verdict(checks, domain, min_seen=2):
"""A URL counts once it ranks in min_seen checks; a swap counts only where the page as a whole held."""
found = [ours(c["organic"], domain) for c in checks]
seen = Counter(url for f in found for url in f)
steady = {url for url, n in seen.items() if n >= min_seen}
together = sum(len(steady & f.keys()) >= 2 for f in found)
held = [jaccard(a["sites"], b["sites"]) for a, b in zip(checks, checks[1:])]
tops = [min(f, key=f.get) if f else None for f in found]
swaps = sum(1 for i, h in enumerate(held)
if h >= 0.5 and tops[i] and tops[i + 1] and tops[i] != tops[i + 1] and {tops[i], tops[i + 1]} <= steady)
if together >= min_seen:
label = "2+ URLs in the same top 10"
elif swaps:
label = "URL swap"
elif steady:
label = "one URL"
else:
label = "unconfirmed" if seen else "not in the top 10"
return label, found, seen, held
def report(keywords, domain, save, gl, min_seen=2):
tally = Counter()
for q, checks in load(keywords, save, gl).items():
label, found, seen, held = verdict(checks, domain, min_seen)
tally[label] += 1
overlap = ", ".join(f"{h:.2f}" for h in held) or "-"
print(f"\n{q}: {label} ({len(checks)} checks; site overlap between checks: {overlap})")
for url in sorted(seen, key=lambda u: min(f.get(u, 99) for f in found)):
cells = "".join(f"{f.get(url, '-'):>4}" for f in found)
flag = "" if seen[url] >= min_seen else f" <- in {seen[url]} of {len(checks)} checks"
print(f" {cells} {url}{flag}")
print("\n" + ", ".join(f"{n} {label}" for label, n in tally.most_common()))
if __name__ == "__main__":
ap = argparse.ArgumentParser()
ap.add_argument("command", choices=["check", "report"])
ap.add_argument("keywords", help="text file, one keyword per line")
ap.add_argument("domain", nargs="?", help="your domain, for report (subdomains included)")
ap.add_argument("--save", default="checks", help="folder that holds one subfolder per check")
ap.add_argument("--gl", default="us")
ap.add_argument("--hl", default="en")
ap.add_argument("--min-seen", type=int, default=2, help="checks a URL must rank in before it counts")
a = ap.parse_args()
kws = [k.strip() for k in Path(a.keywords).read_text(encoding="utf-8").splitlines() if k.strip()]
if a.command == "check":
check(kws, a.gl, a.hl, a.save)
else:
report(kws, a.domain.lower().removeprefix("www."), a.save, a.gl, a.min_seen)
Run check on a schedule (daily for a real tracker) and report whenever you like; the report reads the saved files and costs nothing:
export BESTSERP_API_KEY="YOUR_API_KEY"
python cannibalization.py check keywords.txt --save checks/ # 1 credit per keyword
python cannibalization.py check keywords.txt --save checks/ # later: a second check
python cannibalization.py report keywords.txt example.com --save checks/
Our tutorial on building a rank tracker in Python shows how to schedule it with cron or Windows Task Scheduler.
Example: Google's own SEO docs, checked three times
For a large site with several pages per topic, we used Google's own documentation: Search Central (developers.google.com), Search Console Help (support.google.com) and every other google.com host. We checked 10 SEO keywords twice, less than two minutes apart, from US IPs, and 3 of them a third time. Real output, trimmed:
canonical tag: one URL (3 checks; site overlap between checks: 0.00, 0.45)
- 1 1 developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls
- 6 - support.google.com/webmasters/answer/10347851?hl=en <- in 1 of 3 checks
robots.txt: 2+ URLs in the same top 10 (3 checks; site overlap between checks: 0.00, 0.75)
- 1 1 developers.google.com/search/docs/crawling-indexing/robots/intro
- 5 8 google.com/robots.txt
- 8 6 developers.google.com/crawling/docs/robots-txt/create-robots-txt
hreflang: unconfirmed (3 checks; site overlap between checks: 0.00, 0.00)
4 - - developers.google.com/search/docs/specialty/international/localized-versions <- in 1 of 3 checks
core web vitals: 2+ URLs in the same top 10 (2 checks; site overlap between checks: 1.00)
2 2 developers.google.com/search/docs/appearance/core-web-vitals
3 3 support.google.com/webmasters/answer/9205520?hl=en
3 not in the top 10, 3 unconfirmed, 2 one URL, 2 2+ URLs in the same top 10
| Keyword | Google URLs: position in each check | Verdict |
|---|---|---|
| core web vitals | Search Central 2, 2; Search Console Help 3, 3 | 2+ URLs |
| robots.txt | Intro -, 1, 1; google.com/robots.txt -, 5, 8; Create guide -, 8, 6 | 2+ URLs |
| canonical tag | Search Central -, 1, 1; Search Console Help -, 6, - | One URL |
| xml sitemap | Search Central 4, 3 | One URL |
| 301 redirect, structured data | One page, second check only | Unconfirmed |
| hreflang | Search Central 4, -, - | Unconfirmed |
| noindex, duplicate content, search console performance report | None | Not in the top 10 |

What the data says:
- Two keywords had two or more Google URLs in one top 10, and neither needs a fix. On "core web vitals", Search Central's explainer (2) and the Search Console report's help page (3) held in both checks; on "robots.txt", an introduction, a how-to and Google's robots.txt file held three places. Each serves a different need. web.dev, a separate Google developer site, also held 1 and 5 there: a learning-path hub and its main article.
- No URL swap held. No keyword's top Google URL changed between two checks of a similar page.
- Half the pages were replaced. 5 of 10 keywords shared no site between checks 1 and 2, all run from US IPs. Four of those first checks had no Google page at all.
- The third check changed one verdict. With a single check (
--min-seen 1), "canonical tag" is flagged too: Search Central at 1, a Search Console Help page at 6. The third check kept only the first.

Two URLs from one site on page one was the exception: in a pilot the same morning, HubSpot ranked in 8 of 12 US top 10s for marketing keywords such as "free crm", each time with one URL.
How to fix keyword cannibalization: a decision tree
Pick the fix with two questions: do the pages serve the same intent, and must both URLs stay? Google's documentation on consolidating duplicate URLs and on redirects describes the tools.

| Situation | Fix |
|---|---|
| Same intent, one page can do the job | Merge into the stronger page, 301 the other |
| Same content, both URLs must stay (variants, print or parameter versions) | rel="canonical" on the duplicate |
| Different intents, but the wrong page ranks | Re-target titles, H1s and internal links |
| A page that should not be in Search at all | noindex |
| Different intents, both pages rank well | Nothing |
Google calls a redirect "a strong signal that the target of the redirect should become canonical", for use "only when deprecating a duplicate page"; rel="canonical" is "a strong signal" too, but any canonical preference "is a hint, not a rule." Keep noindex out of it: Google does not recommend it "to prevent selection of a canonical page within a single site, because it will completely block the page from Search." Save noindex for pages that should never be found through Search, such as internal search results.
A merge takes five steps:
- Choose the page to keep: more clicks for the query, more links from other sites, the cleaner URL.
- Move what is unique from the other page into it.
- Redirect the old URL with a server-side 301 (or 308); Google recommends "a permanent server-side redirect whenever possible" when a URL in search results changes.
- Update internal links and the sitemap: "link to the canonical URL rather than a duplicate URL."
- Keep checking the query for a week or two: the old URL should drop out and the kept one hold.
Preventing keyword cannibalization
Most cases start with a new page for a query an old page already covers. Four habits prevent it:
- Keep a keyword map: one primary query per URL, checked before you brief a new page.
- Update instead of republishing: refresh the existing URL for yearly topics.
- Store the ranking URL, not just the position: it catches swaps early; see our keyword rank tracking guide and rank tracking with BestSERP.
- Check before you publish: a
site:search in a browser shows which of your pages Google ties to a topic; our tested list of Google search operators explains which operators still work.
Common problems and fixes
| Problem | Cause | Fix |
|---|---|---|
| A URL is flagged once, then gone | That check got a replaced results page | Keep --min-seen 2 |
| A page you know ranks is missing | Its link was still a redirect, or it is on another domain | Poll until linksPending is false; run report once per domain |
| "skipped ... ran from" lines | No IP was free in that country | Run check again later |
| The Pages tab shows more impressions than the query | Each URL is counted separately | Compare URLs with each other, not with the query total |
Method
- Keywords: canonical tag, noindex, robots.txt, 301 redirect, hreflang, xml sitemap, structured data, core web vitals, duplicate content and search console performance report.
- Searches: BestSERP
/searchwithgl=us,hl=en,num=10andnoCache: true; links polled untillinksPendingwas false. All 35 responses:ipCountryus, no unresolved links, not cached. - Timing (November 18, 2026, UTC): check 1 at 04:34-04:35, check 2 at 04:36-04:37, check 3 (canonical tag, robots.txt, hreflang) at 04:38; HubSpot pilot (12 keywords, one check) at 04:31-04:32.
- Counting: google.com with all subdomains, web.dev separately; organic results only. Site overlap is the Jaccard index of the hosts in two consecutive checks.
- Cost: 35 searches, 35 credits.
Next steps
Fix keyword cannibalization only when two pages chase the same intent and more than one check shows it.
- Before you decide, read the results page with our guide to SERP analysis.
- To watch your URLs across a keyword set: SERP data for SEO tools.
- To see where your pages rank today in any country: the free SERP Checker.