Google Alerts Alternative: Brand Monitoring with Google News Results
Michael Carter · · 24 min read

Key takeaways
- Google's help page for Google Alerts, checked November 18, 2026, documents email delivery and six settings (frequency, site types, language, region, number of results, account) and mentions no API, webhook or export.
- A brand monitor built on Google results needs 2 searches per brand per run, one News search and one web search; at $0.50 per 1,000 searches, 20 brands checked daily cost about $0.60 a month.
- Keep -site: out of Google News queries: in our November 2026 runs, 28 News results from queries with -site: had no snippet, while 37 from queries without it all had one.
- Key mentions on the final URL and on the headline: across four runs, 17 results were stories already stored at another URL, including Google News RSS links that later came back with their final URLs.
- Brand names that are also places need context words: our first Patagonia search returned 3 results about the clothing company and 12 about the region in Chile and Argentina or a town in Arizona.
A practical Google Alerts alternative for brand monitoring is a small program that searches Google News and web results for your brand on a schedule, remembers every article it has already seen, and sends you only the new ones. This tutorial builds one in Python with BestSERP, a Google SERP API, plus SQLite and a Slack webhook.
It is for developers and marketing or PR teams who want mentions in their own database, chat channel or dashboard instead of an inbox. You get a tested program of about 250 lines, real output from four runs on November 18, 2026 for four brands, and the cost per brand. It belongs to our guide to Shopping, local and news data.
Why look for a Google Alerts alternative
Look for a Google Alerts alternative when email is the wrong place for mentions, or when you need control over the queries, the de-duplication and the data. If a few names in your inbox are enough, keep Google Alerts.
Google's help page for the product, Create an alert (checked 2026-11-18), says you "can get emails when new results for a topic show up in Google Search", for example news, products or mentions of your name. The options are how often you get notifications, the types of sites, the language, "the part of the world you want info from", how many results, and which account gets the alert. Troubleshooting covers the signed-in account, alerts that "have been disabled", a full inbox and the spam folder.
That is a lot for a web form with nothing to host. The page mentions no API, webhook, export or price, and does not say how an alert decides what is new. Those are the parts you build here.
| Google Alerts (per Google's help page) | Your own monitor (this tutorial) | |
|---|---|---|
| Delivery | Printed digest, a Slack webhook, or anything your code can call | |
| Query control | A topic, plus six settings | Any Google query: quotes, OR, -site:, a recency window, country (gl) and language (hl) |
| What "new" means | Not documented | Not yet in your SQLite table, by final URL and headline |
| Data you keep | The emails | Every response and every mention, as rows you can query |
| Setup | A web form | Python, an API key and a scheduler (cron, Task Scheduler) |
| Cost | No price on the help page | 2 searches per brand per run; $0.50 per 1,000 searches |
What you will build and what you need
The monitor is one Python file you schedule once a day or once an hour. Each run searches every brand (twice by default), waits for the final article links, drops what it has seen before and sends a digest of the rest:

- Python 3.11 or newer (we used 3.12.10) and
requests(2.34.2). SQLite ships with Python. - A BestSERP API key in the
BESTSERP_API_KEYenvironment variable. The quickstart shows the first request; a new account has 3,000 free searches. - Optional: a Slack incoming webhook URL in
SLACK_WEBHOOK_URL, for the digest.
What to monitor: brand queries that work
Search for each brand's exact name in quotes, keep your own site out of web searches, and add context words when the name is also an ordinary word or a place. Every brand is one entry in brands.json:
[
{"brand": "Rivian", "query": "\"Rivian\"",
"terms": ["rivian"], "own_domains": ["rivian.com"], "exclude": ["stock price"]},
{"brand": "Patagonia", "query": "\"Patagonia\" (clothing OR jacket OR apparel OR company OR Chouinard)",
"terms": ["patagonia"], "own_domains": ["patagonia.com", "patagonia.co.nz"], "exclude": [],
"endpoints": ["news"]},
{"brand": "Framework Laptop", "query": "\"Framework Laptop\"",
"terms": ["framework"], "own_domains": ["frame.work"], "exclude": ["for sale"]}
]
querygoes to Google as written: quotes keep the exact name,ORadds context.termsare the words that count as naming the brand in a headline or snippet (Step 3 scores on them).own_domainsbecome-site:operators on web searches, and the code also drops them from every result.excludeholds headline words that mark pages you never want: stock quote pages, job ads, listings.endpoints(optional) limits a brand to["news"]when its web results are mostly shops.
Keep -site: out of News searches. In a test on November 18, "Duolingo" -site:duolingo.com returned 7 News results without a single snippet, while "Duolingo" returned 10, all with snippets. Across our runs, 28 News tab items from queries with -site: had no snippet, and 37 from queries without it all had one. In our last run, without -site:, no News result came from the brands' own domains, and the code filters them anyway:
def query_for(brand, endpoint):
"""-site: for your own domains on web searches only: on /news it cost us the snippets and some results."""
if endpoint == "news":
return brand["query"]
return brand["query"] + "".join(f" -site:{d}" for d in brand["own_domains"])
We tested four brands that fail in different ways: Rivian (many stories a day), Duolingo (a busy week with a game partnership), Patagonia (a brand that is also a region) and Framework Laptop (a niche product whose short name is an ordinary word).
Step 1: Search Google News results from the last 24 hours
One /news request returns the Google News tab for a query as JSON, and tbs: "qdr:d" limits it to the past 24 hours. BestSERP answers an identical news request within 5 minutes from its cache at the normal price, so noCache: true costs nothing extra. The function polls a 202, waits on 429 and retries 502 and 503, which are refunded:
def search(endpoint, query, gl="us"):
"""One /news or /search request for the past 24 hours; polls 202, retries 429, 502 and 503."""
headers = {"X-API-KEY": os.environ["BESTSERP_API_KEY"]}
body = {"q": query, "gl": gl, "hl": "en", "tbs": RECENCY, "noCache": True}
for attempt in range(4):
resp = requests.post(f"{API}/{endpoint}", headers=headers, json=body, timeout=130)
if resp.status_code == 202: # still running: poll the request ID
request_id = resp.json()["requestId"]
while resp.status_code == 202:
time.sleep(3)
resp = requests.get(f"{API}/requests/{request_id}", headers=headers, timeout=130)
if resp.status_code == 200:
return resp.json()
if resp.status_code in (429, 502, 503): # 429 is never charged; 502 and 503 are refunded
time.sleep(int(resp.headers.get("Retry-After", 10)))
continue
resp.raise_for_status() # 400, 401, 402: fix the request instead of retrying
raise RuntimeError(f"/{endpoint} {query!r}: still failing after {attempt + 1} attempts")
Each item has a headline, the link, usually a snippet, the displayed date, publishedAt in ISO 8601 UTC, the publisher (source), its domain and an origin. One item from a Rivian search, trimmed:
{
"position": 1,
"title": "Lucid Group vs. Rivian Automotive: Here's What Their Revenue Trends Tell Investors",
"link": "https://www.fool.com/coverage/charts/2026/11/17/lucid-group-vs-rivian-automotive-here-s-what-their-revenue-trends-tell-investors/",
"date": "7 hours ago",
"publishedAt": "2026-11-17T21:25:00+00:00",
"source": "The Motley Fool",
"domain": "fool.com",
"origin": "google_search"
}
origin is google_search for the News tab and google_news_rss when Google blocked the News tab and the API used the Google News RSS feed, whose ranking differs (/news reference). That happened for 7 of our 16 /news requests: each took about 47 seconds instead of 3 to 20, with news.google.com/rss/articles/... links, no snippets, www. in domain and no ipCountry. Every other response ran from a US IP. Every item had a domain in these runs (a November 17 test had found News items without one); the code falls back to the link's host anyway.
Step 2: Add web results and wait for final links
News sites are only part of the conversation. A /search request with the same recency adds organic web results (forum threads, social posts, videos, blogs) and, when Google shows it, the Top stories block in news[], for one more credit per brand.
Google serves many result links as redirects. BestSERP replaces them with final URLs; when Google spaces the redirects out, the first answer says linksPending: true and the final links arrive by polling the request (how link resolution works). The monitor sends every search first, then waits, so later searches resolve while it polls the first:
def final_links(data):
"""Poll the request until Google's redirect links are replaced by final URLs (linksPending false)."""
headers = {"X-API-KEY": os.environ["BESTSERP_API_KEY"]}
deadline = time.time() + LINK_WAIT_S
while data.get("linksPending") and time.time() < deadline:
time.sleep(5)
resp = requests.get(f"{API}/requests/{data['requestId']}", headers=headers, timeout=130)
if resp.status_code == 200:
data = resp.json()
return data
In a test before the runs, final links arrived 28 seconds after a /news request and 40 seconds after a /search request; after the monitor's wait, no link in any run was still a redirect. Polling costs nothing. A link still pending after 150 seconds stays a redirect, so the monitor keys that item on its headline.
Step 3: De-duplicate and score mentions
De-duplicate on the final URL, then on the headline, and rank what is left by where the brand appears. Across four runs, 17 results were a story already stored at another URL:
- The same story on two sites: "Rivian Stock Falls Off A Cliff" ran on Yahoo Finance and on 247wallst.com.
- The same story through two routes: Notebookcheck's Framework keyboard article came first as a Google News RSS link, later as a web result with its final URL. Only the headline connects the two.
- The same page under a new URL: the r/Rivian subreddit page came back as a "top" listing instead of "rising", with a new
after=cursor. The same headline on the same site counts as seen.
The same article in /news and again in /search needs no extra rule: the URL key catches it. url_key strips tracking parameters, the fragment, www. and a trailing slash; title_key reduces a headline to lowercase words without the "..." Google adds to cut titles:
TRACKING = re.compile(r"^(utm_|fbclid$|gclid$|mc_|ocid$|cmpid$|smid$|guccounter$)")
def url_key(link):
"""The final URL without tracking parameters, fragment, "www." or trailing slash: one key per article."""
parts = urlsplit(link)
host = parts.netloc.lower().removeprefix("www.")
query = urlencode([(k, v) for k, v in parse_qsl(parts.query) if not TRACKING.match(k.lower())])
return urlunsplit(("https", host, parts.path.rstrip("/"), query, ""))
def title_key(title):
"""Lowercase ASCII words of a headline, without the "..." Google adds to cut titles."""
text = unicodedata.normalize("NFKD", title).encode("ascii", "ignore").decode().lower()
text = re.sub(r"\s*\.\.\.\s*$", "", text)
return re.sub(r"[^a-z0-9]+", " ", text).strip()
def score(item, brand):
"""2: the headline names the brand; 1: only the snippet does; 0: Google matched text we cannot see."""
def names(text):
return any(re.search(rf"\b{re.escape(t)}\b", (text or "").lower()) for t in brand["terms"])
return 2 if names(item.get("title")) else 1 if names(item.get("snippet")) else 0
Every result goes into SQLite, new or not, with the response that carried it, so you can test new rules on stored data without a new search. The mentions table is keyed by brand and URL:
CREATE TABLE IF NOT EXISTS mentions (
brand TEXT NOT NULL,
url_key TEXT NOT NULL, -- final URL without tracking parameters: one row per article
title TEXT,
title_key TEXT, -- lowercase headline words, to spot the same story at another site
link TEXT, source TEXT, domain TEXT,
published TEXT, -- publishedAt, or the displayed "N hours ago" turned into a UTC time
endpoint TEXT,
origin TEXT, -- google_search, or google_news_rss when the News tab was blocked
score INTEGER, -- 2: brand in the headline, 1: only in the snippet, 0: neither
twin_of TEXT, -- url_key of the same headline seen at another site
check_id INTEGER REFERENCES checks(id),
first_seen TEXT NOT NULL,
PRIMARY KEY (brand, url_key)
);
Before the database check, record drops your own domains, anything published more than 26 hours ago and headlines with an exclude word. The age filter exists because of the RSS fallback: with qdr:d, it returned an eBay listing dated August 15, 2024.
Step 4: Send a daily digest to Slack
The digest groups new mentions by brand, headline mentions first and newest first, one line each with the publisher and the published time. With SLACK_WEBHOOK_URL set, notify posts the same text:
def notify(text, has_new):
print(text)
webhook = os.environ.get("SLACK_WEBHOOK_URL") # a Slack incoming webhook; the URL is a secret, keep it out of git
if webhook and has_new:
requests.post(webhook, json={"text": text}, timeout=30).raise_for_status()
Slack's guide to incoming webhooks (checked 2026-11-18) describes this payload: an HTTP POST with a JSON body whose minimum is text, to a URL tied to one channel. It warns: "Your webhook URL contains a secret. Don't share it online, including via public version control repositories."
We had no Slack workspace, so nothing went to Slack: SLACK_WEBHOOK_URL pointed at a local server that saved each request (webhook_capture.py in the post's code). Each run sent one JSON POST with the digest in text, from 2,834 characters (9 mentions) to 14,337 (59). Another chat tool needs only a different payload in notify.
What four runs found
We ran the monitor four times on November 18, 2026, between 04:32 and 06:30 UTC. The first run reports everything from the past 24 hours; later runs report only what the database has not seen:
| Run (UTC) | Searches | Results | New | Seen before | Same headline | Dropped |
|---|---|---|---|---|---|---|
| 04:32 (first version) | 8 | 70 | 59 | 4 | 7 | 0 |
| 05:15 | 8 | 68 | 21 | 40 | 1 | 6 |
| 06:05 | 7 | 55 | 9 | 36 | 5 | 5 |
| 06:30 (final version) | 7 | 64 | 21 | 34 | 4 | 5 |
"Dropped" counts your own site, items over 26 hours old and excluded headlines. A run took 1 minute 47 seconds when every News tab answered and 5 minutes 24 seconds when three requests fell back to RSS. The last digest, as cards:

Run 1: four fixes. The first digest was useful for three brands and mostly noise for the fourth:
- Patagonia is also a place. Of 19 results for
"Patagonia", 3 were about the clothing company and 12 about the region in Chile and Argentina or the town in Arizona (a weather forecast, the local time in Chile, horse-riding posts). We added context words withOR. - The RSS fallback needs guards. Two of four
/newsrequests used it, one returning the 2024 eBay listing. We added the age filter and removewww.from every domain. - Some web results are not mentions. MarketWatch's DUOL stock quote page and an Indeed job search came back as results from the past day. We added
excludewords, matched on the headline only. - Syndication is common. Seven results repeated a headline already stored from another site.
Run 2, 43 minutes later: only what is new. For the three unchanged brands, five mentions were new, one published 20 minutes before the run. Three of the five had been published before the first run but were missing from its results: Google's lists change between requests. A sixth, The Verge's "Best of 2026", slipped through as a repeat because the first version stored its domain with www.. Patagonia's context words removed the places but filled the web results with products: of 9, 5 were stores, listings or job ads and 1 was Patagonia's New Zealand site. We limited Patagonia to news and added patagonia.co.nz to its domains.
Run 3: a route switch. Rivian's News tab was blocked, so its news came from RSS. Three of those 10 items were stories already stored with their final URLs, matched only by headline. The run reported 9 new mentions.
Run 4: no -site: on News searches. The final version keeps -site: for web searches only. Rivian's News tab returned 9 results instead of 3 or 4, Duolingo's 10 instead of 7, all with snippets, and the digest grew to 21; 15 of them had been published before the first run, but the earlier queries had not returned them. Patagonia is still not clean: one story was about data centers in the region, and 5 of its 7 new mentions named the brand in neither headline nor snippet.
Method. Four runs on November 18, 2026 (04:32, 05:15, 06:05 and 06:30 UTC); gl: "us", hl: "en", tbs: "qdr:d", noCache: true; 30 searches plus 5 tests (2 before the first run, 3 on snippets). Run 1 used a first version (no age filter, exclude on snippets too, www. kept, Patagonia without context words), and runs 1 to 3 put -site: in every query; the first version's stored domains were normalized before run 3. Raw responses are in the post's data folder. Two hours of runs show how the monitor behaves, not how much coverage a brand gets over weeks.
How much a Google Alerts alternative costs to run
Each brand costs 2 searches per run, one /news and one /search, and BestSERP charges $0.50 per 1,000 searches after 3,000 free ones valid for 30 days (pricing, checked 2026-11-18). For a 30-day month:
| Brands | Once a day | Every 6 hours | Every hour |
|---|---|---|---|
| 5 | 300 searches, $0.15 | 1,200 searches, $0.60 | 7,200 searches, $3.60 |
| 20 | 1,200 searches, $0.60 | 4,800 searches, $2.40 | 28,800 searches, $14.40 |
| 100 | 6,000 searches, $3 | 24,000 searches, $12 | 144,000 searches, $72 |
- News only: the cost halves, at the price of forum threads, social posts and videos.
- More countries: each
glis a separate search, so 20 brands in 3 countries cost as much as 60 brands in one. - Failures are free:
502and503answers are refunded, and polling for final links costs nothing. - Speed: the program sends one search at a time, 3 to 47 seconds each in our runs. For 100 brands, send several at once, up to the default limit of 60 searches per minute; running many searches reliably shows how.
For hourly runs, keep qdr:d rather than qdr:h. In run 2, 3 of 5 genuinely new mentions had been published before the previous run, so an hour-wide window would have missed them; the database already removes the repeats of a wider window.
Common problems and fixes
| Problem | Cause | Fix |
|---|---|---|
| Places, weather or travel posts for your brand | The name is also an ordinary word or a place | Add context words with OR, and exclude words for what remains |
| News results without snippets, and fewer of them | -site: in the /news query |
Put -site: on web searches only, as query_for does |
| Web results full of shops and job ads | The brand name sells products | exclude words, or "endpoints": ["news"] |
| The same story twice in a digest | Two sites ran it under different headlines | Accept it, or compare snippets too |
Links start with news.google.com/rss/articles/ |
The News tab was blocked; the RSS fallback answered | They open the article in a browser; a later run often brings the final URL |
ipCountry missing or different from gl |
The RSS fallback answered, or no IP in that country was free | The monitor prints a note; check it for local brands |
Full code
The complete Google Alerts alternative, as run on November 18, 2026 with Python 3.12.10, requests 2.34.2 and SQLite 3.49.1. Save it next to brands.json, set BESTSERP_API_KEY and schedule python brand_monitor.py with cron or Windows Task Scheduler.
"""Brand monitoring with Google News and web results, a Google Alerts alternative (BestSERP /news and /search).
python brand_monitor.py # search every brand in brands.json, store the results, send only new mentions
Needs BESTSERP_API_KEY in the environment. Optional: SLACK_WEBHOOK_URL (the digest is also posted there),
MENTIONS_DB (path of the SQLite file, default mentions.db).
Tested 2026-11-18 with Python 3.12.10, requests 2.34.2 and SQLite 3.49.1.
"""
import json
import os
import re
import sqlite3
import sys
import time
import unicodedata
from datetime import datetime, timedelta, timezone
from pathlib import Path
from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
import requests
API = "https://api.bestserp.com"
HERE = Path(__file__).resolve().parent
RECENCY = "qdr:d" # the past 24 hours; qdr:h is the past hour, qdr:w the past week
MAX_AGE = timedelta(hours=26) # drop older items (the RSS fallback can return them); 2 h of slack for "N hours ago"
LINK_WAIT_S = 150 # how long to wait for final links (usually under a minute)
TRACKING = re.compile(r"^(utm_|fbclid$|gclid$|mc_|ocid$|cmpid$|smid$|guccounter$)")
SCHEMA = """
CREATE TABLE IF NOT EXISTS checks (
id INTEGER PRIMARY KEY,
brand TEXT NOT NULL,
endpoint TEXT NOT NULL, -- news or search
checked_at TEXT NOT NULL, -- UTC, ISO 8601
ip_country TEXT, -- ipCountry: where the search ran from (missing when unknown)
request_id TEXT,
results INTEGER,
unresolved INTEGER, -- links still Google redirects after waiting
raw_json TEXT -- the whole response, to re-check rules later without new searches
);
CREATE TABLE IF NOT EXISTS mentions (
brand TEXT NOT NULL,
url_key TEXT NOT NULL, -- final URL without tracking parameters: one row per article
title TEXT,
title_key TEXT, -- lowercase headline words, to spot the same story at another site
link TEXT,
source TEXT,
domain TEXT,
published TEXT, -- publishedAt, or the displayed "N hours ago" turned into a UTC time
endpoint TEXT,
origin TEXT, -- google_search, or google_news_rss when the News tab was blocked
score INTEGER, -- 2: brand in the headline, 1: only in the snippet, 0: neither
twin_of TEXT, -- url_key of the same headline seen at another site
check_id INTEGER REFERENCES checks(id),
first_seen TEXT NOT NULL,
PRIMARY KEY (brand, url_key)
);
CREATE INDEX IF NOT EXISTS mentions_by_title ON mentions(brand, title_key);
"""
def query_for(brand, endpoint):
"""-site: for your own domains on web searches only: on /news it cost us the snippets and some results."""
if endpoint == "news":
return brand["query"]
return brand["query"] + "".join(f" -site:{d}" for d in brand["own_domains"])
def search(endpoint, query, gl="us"):
"""One /news or /search request for the past 24 hours; polls 202, retries 429, 502 and 503."""
headers = {"X-API-KEY": os.environ["BESTSERP_API_KEY"]}
body = {"q": query, "gl": gl, "hl": "en", "tbs": RECENCY, "noCache": True}
for attempt in range(4):
resp = requests.post(f"{API}/{endpoint}", headers=headers, json=body, timeout=130)
if resp.status_code == 202: # still running: poll the request ID
request_id = resp.json()["requestId"]
while resp.status_code == 202:
time.sleep(3)
resp = requests.get(f"{API}/requests/{request_id}", headers=headers, timeout=130)
if resp.status_code == 200:
return resp.json()
if resp.status_code in (429, 502, 503): # 429 is never charged; 502 and 503 are refunded
time.sleep(int(resp.headers.get("Retry-After", 10)))
continue
resp.raise_for_status() # 400, 401, 402: fix the request instead of retrying
raise RuntimeError(f"/{endpoint} {query!r}: still failing after {attempt + 1} attempts")
def final_links(data):
"""Poll the request until Google's redirect links are replaced by final URLs (linksPending false)."""
headers = {"X-API-KEY": os.environ["BESTSERP_API_KEY"]}
deadline = time.time() + LINK_WAIT_S
while data.get("linksPending") and time.time() < deadline:
time.sleep(5)
resp = requests.get(f"{API}/requests/{data['requestId']}", headers=headers, timeout=130)
if resp.status_code == 200:
data = resp.json()
return data
def is_redirect(link):
parts = urlsplit(link)
return parts.netloc.lower().endswith("google.com") and parts.path in ("/url", "/goto")
def url_key(link):
"""The final URL without tracking parameters, fragment, "www." or trailing slash: one key per article."""
parts = urlsplit(link)
host = parts.netloc.lower().removeprefix("www.")
query = urlencode([(k, v) for k, v in parse_qsl(parts.query) if not TRACKING.match(k.lower())])
return urlunsplit(("https", host, parts.path.rstrip("/"), query, ""))
def title_key(title):
"""Lowercase ASCII words of a headline, without the "..." Google adds to cut titles."""
text = unicodedata.normalize("NFKD", title).encode("ascii", "ignore").decode().lower()
text = re.sub(r"\s*\.\.\.\s*$", "", text)
return re.sub(r"[^a-z0-9]+", " ", text).strip()
def published(item, now):
"""publishedAt when the API has it, else the "N hours ago" shown in the date or at the start of the snippet."""
if item.get("publishedAt"):
return item["publishedAt"]
m = re.match(r"(\d+) (minute|hour|day)s? ago", item.get("date") or item.get("snippet") or "")
if m:
return (now - timedelta(**{m[2] + "s": int(m[1])})).isoformat(timespec="minutes")
return None
def score(item, brand):
"""2: the headline names the brand; 1: only the snippet does; 0: Google matched text we cannot see."""
def names(text):
return any(re.search(rf"\b{re.escape(t)}\b", (text or "").lower()) for t in brand["terms"])
return 2 if names(item.get("title")) else 1 if names(item.get("snippet")) else 0
def store_check(db, brand, endpoint, data, now):
for key in ("news", "organic", "videos"): # publisher icons are not needed
for item in data.get(key, []):
item.pop("favicon", None)
items = data.get("news", []) + data.get("organic", [])
cur = db.execute(
"INSERT INTO checks (brand, endpoint, checked_at, ip_country, request_id, results, unresolved, raw_json)"
" VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
(brand["brand"], endpoint, now.isoformat(timespec="seconds"), data.get("ipCountry"), data.get("requestId"),
len(items), sum(is_redirect(i["link"]) for i in items), json.dumps(data)))
return cur.lastrowid
def record(db, brand, endpoint, data, check_id, now, new):
"""Store every result of one response and append the mentions never seen before to `new`."""
counts = dict.fromkeys(("results", "new", "seen", "same headline", "own site", "excluded", "too old"), 0)
# /news returns the News tab; /search returns the organic results and the Top stories block.
for item in data.get("news", []) + data.get("organic", []):
counts["results"] += 1
link = item["link"]
domain = (item.get("domain") or urlsplit(link).netloc).lower().removeprefix("www.") # RSS items keep www.
if any(domain == d or domain.endswith("." + d) for d in brand["own_domains"]):
counts["own site"] += 1 # your own pages are not mentions (and -site: may not catch them all)
continue
when_published = published(item, now)
if when_published and datetime.fromisoformat(when_published) < now - MAX_AGE:
counts["too old"] += 1
continue
if any(word.lower() in item["title"].lower() for word in brand["exclude"]):
counts["excluded"] += 1 # quote pages, job ads, listings: words that mark them in the headline
continue
tkey = title_key(item["title"])
ukey = f"unresolved:{tkey}" if is_redirect(link) else url_key(link)
if db.execute("SELECT 1 FROM mentions WHERE brand = ? AND url_key = ?", (brand["brand"], ukey)).fetchone():
counts["seen"] += 1
continue
# The same story at another URL: a syndicated copy, a cut title ("...") from the web results,
# or the same page with new URL parameters (same headline, same site).
twin = db.execute(
"SELECT url_key FROM mentions WHERE brand = ? AND ((title_key = ? AND (domain = ? OR length(?) >= 25))"
" OR (length(title_key) >= 25 AND length(?) >= 25 AND (title_key LIKE ? || '%' OR ? LIKE title_key || '%')))",
(brand["brand"], tkey, domain, tkey, tkey, tkey, tkey)).fetchone()
mention = {"brand": brand["brand"], "title": item["title"], "link": link, "domain": domain,
"source": item.get("source") or domain, "published": when_published,
"endpoint": endpoint, "origin": item.get("origin"), "score": score(item, brand)}
db.execute("INSERT INTO mentions VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(brand["brand"], ukey, item["title"], tkey, link, mention["source"], domain,
mention["published"], endpoint, mention["origin"], mention["score"],
twin[0] if twin else None, check_id, now.isoformat(timespec="seconds")))
if twin:
counts["same headline"] += 1
original = next((m for m in new if m["brand"] == brand["brand"] and m["key"] == twin[0]), None)
if original: # the first copy is in this digest: list this site next to it
original["also"].append(domain)
continue
counts["new"] += 1
new.append({**mention, "key": ukey, "also": []})
return counts
def when(iso):
return datetime.fromisoformat(iso).astimezone(timezone.utc).strftime("%Y-%m-%d %H:%M UTC") if iso else "time unknown"
def digest(brands, new, now):
"""Plain text: new mentions per brand, headline mentions first, newest first."""
lines = [f"Brand monitor, {now:%Y-%m-%d %H:%M} UTC: {len(new)} new mentions in the past 24 hours"]
for brand in brands:
mine = [m for m in new if m["brand"] == brand["brand"]]
mine.sort(key=lambda m: m["published"] or "", reverse=True)
mine.sort(key=lambda m: m["score"], reverse=True)
lines.append(f"\n{brand['brand']}: {len(mine) or 'no'} new")
for m in mine:
where = {2: "headline", 1: "snippet", 0: "page text"}[m["score"]]
also = f" (also at {', '.join(m['also'])})" if m["also"] else ""
lines.append(f" [{where}] {m['title']} | {m['source']} | {when(m['published'])}{also}")
lines.append(f" {m['link']}")
return "\n".join(lines)
def notify(text, has_new):
print(text)
webhook = os.environ.get("SLACK_WEBHOOK_URL") # a Slack incoming webhook; the URL is a secret, keep it out of git
if webhook and has_new:
requests.post(webhook, json={"text": text}, timeout=30).raise_for_status()
def main():
sys.stdout.reconfigure(encoding="utf-8", errors="replace") # headlines are not always ASCII
brands = json.loads((HERE / "brands.json").read_text(encoding="utf-8"))
db = sqlite3.connect(os.environ.get("MENTIONS_DB", "mentions.db"))
db.executescript(SCHEMA)
now = datetime.now(timezone.utc)
responses = []
for brand in brands: # send every search first: their final links resolve while the others run
for endpoint in brand.get("endpoints", ["news", "search"]): # news first: a story in both counts as news
try:
responses.append((brand, endpoint, search(endpoint, query_for(brand, endpoint), brand.get("gl", "us"))))
except (requests.RequestException, RuntimeError) as e: # failed searches are refunded
print(f"{brand['brand']} /{endpoint}: skipped, {e}")
new = []
for brand, endpoint, data in responses:
data = final_links(data)
check_id = store_check(db, brand, endpoint, data, now)
counts = record(db, brand, endpoint, data, check_id, now, new)
ip = data.get("ipCountry")
note = f" | ran from a {ip} IP, not {brand.get('gl', 'us')}" if ip and ip != brand.get("gl", "us") else ""
if any(i.get("origin") == "google_news_rss" for i in data.get("news", [])):
note += " | News tab blocked: Google News RSS used (other ranking, news.google.com links)"
print(f"{brand['brand']:<17} /{endpoint:<6} " + ", ".join(f"{v} {k}" for k, v in counts.items()) + note)
db.commit()
print()
notify(digest(brands, new, now), bool(new))
if __name__ == "__main__":
main()
Next steps
This Google Alerts alternative keeps its own memory, reports only new mentions and costs a tenth of a cent per brand per daily run. Next:
- The Google News API page and the /news reference for every parameter.
- The same alert pattern for prices: competitor price monitoring with Google Shopping results; for rankings: detecting ranking drops automatically.
- Fresh news for language models: web search for AI agents, and the Google News RSS feed and its limits.