Google search for AI agents and RAG

Language models do not know what happened after their training data ends. A search tool lets an agent ask Google, read the titles, links and snippets of the top results, and answer with sources. With BestSERP each search costs one credit, so 10,000 agent searches a month cost $5.

Last updated:

Key takeaways

  • 1,000 questions, 1 search each: 1,000 credits, about $0.50 a month.
  • 10,000 questions, 1 search each: 10,000 credits, about $5.00 a month.
  • Built on the Google Search API and the Google News API.
  • A complete Python example is below: set BESTSERP_API_KEY and run it.

The problem

Retrieval from your own documents does not cover current events, prices, product releases or anything on the public web. Agents need a way to look things up and cite where an answer came from.

A SERP API gives the model the same starting point a person has: the ranked results for a query, with links to open. Returning a compact list (title, link, snippet) keeps the context small and makes citations easy.

Suggested architecture

  1. 1

    Tool definition

    Expose one function, web_search(query), to the model with a JSON schema. Most LLM APIs accept the same shape.

  2. 2

    Compact results

    Call /search (or /news for current events) and return only position, title, link and snippet for the top 5 to 10 results.

  3. 3

    Grounded answer

    Number the results in the prompt and ask the model to cite them as [1], [2]. Keep the links so your UI can show sources.

  4. 4

    Guardrails

    Cap searches per conversation and cache answers for repeated queries; the API also serves identical queries from its 1-hour cache.

Complete example

web_search tool and a grounded prompt (Python, any LLM). Set BESTSERP_API_KEY and run it with Python 3.9+ and requests.
import os
import time

import requests

API = "https://api.bestserp.com"
HEADERS = {"X-API-KEY": os.environ["BESTSERP_API_KEY"]}


def search(endpoint: str, **body) -> dict:
    """POST one search and return the JSON. Polls while the API answers 202 (still running)."""
    r = requests.post(f"{API}/{endpoint}", headers=HEADERS, json={"timeout": 120, **body}, timeout=130)
    while r.status_code == 202:
        time.sleep(2)
        r = requests.get(f"{API}/requests/{r.json()['requestId']}", headers=HEADERS, timeout=30)
    r.raise_for_status()
    return r.json()


# Tool schema in the JSON Schema format most LLM APIs use for function calling.
WEB_SEARCH_TOOL = {
    "name": "web_search",
    "description": "Search Google and return the top results with title, link and snippet.",
    "input_schema": {
        "type": "object",
        "properties": {"query": {"type": "string", "description": "What to search for"}},
        "required": ["query"],
    },
}


def web_search(query: str, country: str = "us", news: bool = False) -> list[dict]:
    """The tool implementation: compact results the model can read and cite."""
    data = search("news" if news else "search", q=query, gl=country, hl="en")
    items = data.get("news" if news else "organic", [])[:8]
    return [{"n": i + 1, "title": r.get("title"), "link": r.get("link"), "snippet": r.get("snippet")} for i, r in enumerate(items)]


def grounded_prompt(question: str) -> str:
    """Retrieval-augmented prompt: numbered sources, answer with citations."""
    sources = web_search(question)
    lines = [f"[{s['n']}] {s['title']} - {s['link']}\n{s['snippet'] or ''}" for s in sources]
    return (
        "Answer the question using only the sources below. Cite them as [1], [2].\n\n"
        + "\n\n".join(lines)
        + f"\n\nQuestion: {question}"
    )


if __name__ == "__main__":
    print(grounded_prompt("What changed in the latest Python release?"))
    # Send this prompt to your model, or register WEB_SEARCH_TOOL and call web_search() when the model asks.

What it costs

1 credit per page of 10 results, $0.50 per 1,000 credits. The first 3,000 credits are free.
ScenarioCredits per monthCost per month
1,000 questions, 1 search each1,000$0.50
10,000 questions, 1 search each10,000$5.00
10,000 agent tasks, 3 searches each30,000$15.00

Frequently asked questions

Which endpoint should an agent use?
/search for general questions and /news when the question is about recent events. Both return title, link and snippet for each result.
How many results should I give the model?
Five to ten compact results are usually enough context. More results cost no extra credits (one page holds 10) but use more tokens.
Does the API read the pages behind the links?
No. It returns the results page. If the model needs the full text of a page, fetch that URL yourself.
Can I use it with LangChain?
Yes. Wrap the same HTTP call in a LangChain tool; the LangChain guide has a complete example.

Start with 3,000 free searches

No credit card. Sign in with Google, create a key, send your first request.