Get Google AI Overviews citations in Python

The whole client is requests.post to https://api.agentgeo.org/v1/fetches with a bearer key, a body of {"query": ..., "surfaces": ["google_ai_overview"], "country": "US", "language": "en"}, a raise_for_status(), and a read of answers[0]["sources"]. No SDK exists in any language and none is missing. Below: that call, then the version a schedule can run — several queries, URLs normalized before counting, presence and citations tallied under separate denominators, rows appended to CSV. For the task-level story of what the rows are for, see how to pull citations from Google AI Overviews; this page is the Python.

One POST, one reference list

google_ai_overview takes two request fields the minimal calls on other surfaces don't: country (Google's gl) and language (Google's hl). Both belong in even the smallest version, because together they decide which results page the overview is read from:

import requests

resp = requests.post(
    "https://api.agentgeo.org/v1/fetches",
    json={
        "query": "best password manager for families",
        "surfaces": ["google_ai_overview"],
        "country": "US",   # Google's gl= - the market
        "language": "en",  # Google's hl= - only this surface reads it
    },
    headers={"Authorization": "Bearer ag_live_your_key_here"},
    timeout=200,  # the API can hold a request up to 180s - stay above that
)
resp.raise_for_status()

for src in resp.json()["answers"][0]["sources"]:
    print(f"{src['position']:>2}. {src['url']}")

The response is one JSON object per run; the parts that matter:

{
  "id": "run_9b4e21d7c3a6",
  "query": "best password manager for families",
  "surfaces": ["google_ai_overview"],
  "status": "completed",
  "recordsDelivered": 1,
  "creditsCharged": 1,
  "answers": [
    {
      "surfaceKey": "google_ai_overview",
      "answerText": "For families the overview leans toward managers with shared vaults ...",
      "sources": [
        {
          "title": "The Best Password Managers for Families",
          "url": "https://example.com/guides/family-password-managers",
          "position": 1
        },
        {
          "title": "Shared Family Vaults, Compared",
          "url": "https://www.example.org/blog/shared-vaults/?srsltid=AfmBOopQx3kV",
          "position": 2
        }
      ],
      "fetchedAt": "2026-07-27T07:58:12Z"
    }
  ]
}

position starts at 1 and records the order Google attached the references in — keep it. And look at the second URL: Google decorates result links with its own srsltid tracking parameter, on top of whatever www. and trailing-slash variants the cited site serves. Counted as-is, one page becomes several rows; the longer script strips all of that before anything reaches a Counter.

The version a schedule runs

One call tells you who Google leaned on this morning. The question that justifies a script is who keeps supplying your category's overviews — and, on this surface specifically, how often there is an overview to supply at all. That second number needs its own counter, kept apart from the first. Python 3.9+, requests the only install:

#!/usr/bin/env python3
"""Google AI Overview citations for a query set - presence and citations kept apart."""
import csv
from collections import Counter
from pathlib import Path
from urllib.parse import urlparse, urlunparse

import requests

API = "https://api.agentgeo.org/v1/fetches"
KEY = "ag_live_your_key_here"
COUNTRY = "US"   # Google's gl= - the market whose SERP gets fetched
LANGUAGE = "en"  # Google's hl= - only this surface reads it
OUT = Path("aio_citations.csv")
FIELDS = ["query", "country", "language", "position", "host", "url", "title", "fetchedAt"]

QUERIES = [
    "best password manager for families",
    "password manager with shared family vault",
    "are free password managers safe",
    "how much does a password manager cost",
]


def normalize(url: str) -> tuple[str, str]:
    """(host, clean_url): lowercase host, drop www., strip the query string."""
    p = urlparse(url)
    host = p.netloc.lower().removeprefix("www.")  # Python 3.9+
    return host, urlunparse((p.scheme, host, p.path.rstrip("/"), "", "", ""))


def fetch(query: str) -> dict:
    resp = requests.post(
        API,
        json={
            "query": query,
            "surfaces": ["google_ai_overview"],
            "country": COUNTRY,
            "language": LANGUAGE,
        },
        headers={"Authorization": f"Bearer {KEY}"},
        timeout=200,  # the API can hold a request up to 180s - stay above it
    )
    resp.raise_for_status()
    return resp.json()


hosts = Counter()  # citations per host - denominator: delivered overviews only
delivered = 0      # denominator 1: queries where an AI Overview existed
absent = []        # queries where Google showed no overview (zero credits)
rows = []

for query in QUERIES:
    run = fetch(query)
    for answer in run["answers"]:
        if answer.get("status") == "failed":
            # No AI Overview on this SERP. A result, not an error -
            # and not the same thing as an overview that cited nobody.
            absent.append(query)
            continue
        delivered += 1
        for src in sorted(answer.get("sources") or [], key=lambda s: s["position"]):
            host, clean = normalize(src["url"])
            hosts[host] += 1
            rows.append({
                "query": query,
                "country": COUNTRY,
                "language": LANGUAGE,
                "position": src["position"],
                "host": host,
                "url": clean,
                "title": src["title"],
                "fetchedAt": answer.get("fetchedAt", ""),
            })

new_file = not OUT.exists()
with OUT.open("a", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=FIELDS)
    if new_file:
        writer.writeheader()
    writer.writerows(rows)

# Two rates, two denominators. Never blend them.
print(f"AI Overview presence: {delivered}/{len(QUERIES)} queries")
if absent:
    print("  no overview for: " + "; ".join(absent))

print(f"\ncitation counts (out of {delivered} delivered overviews)")
for host, n in hosts.most_common(10):
    print(f"  {n:>2}/{delivered}  {host}")

print(f"\nappended {len(rows)} rows to {OUT}")

The two printed rates are the discipline this surface demands. Presence — delivered out of len(QUERIES) — is a property of the query set: Google doesn't put an AI Overview on every results page, and when it doesn't, the record comes back status: "failed" at zero credits. Citation counts use delivered as their denominator, because a host can't be cited by an overview that doesn't exist. The CSV opens in append mode and every row carries fetchedAt plus the locale pair — one run is a snapshot, the same file after a month of weekly runs is a series, and the diff is a filter on the timestamp column.

country is gl, language is hl — and only here

Of the six surfaces, google_ai_overview is the only one that honours language, and one of two — with copilot — that honours country. They pass through as Google's hl and gl: interface language and market. That makes them data-defining rather than cosmetic. An en/US fetch and a de/DE fetch read different results pages, cite different hosts, and one may carry an overview where the other has none.

• Hold the pair constant within a series. Switching either mid-quarter silently changes what the numbers measure. The script writes both into every row, so a mixed file can at least be split after the fact.
• Aggregate per market. Comparing markets against each other is analysis; summing them into one Counter is contamination. Filter on the country and language columns before tallying.
• Send nothing else. web_search is read only by chatgpt — on this surface it sits in the payload implying behaviour the fetch isn't getting. query, surfaces, country, language is the complete request.

Absence is a data point

Google decides per results page whether an AI Overview appears, and for plenty of queries it doesn't. When there is none, the answer record carries status: "failed" — at zero credits, since nothing was delivered. The script routes those queries into absent rather than the citation tally, and that routing is what keeps both of its output numbers honest.

Concretely: 12 queries, 8 overviews, your domain cited in 6. Presence is 8/12; your citation rate is 6/8, not 6/12. The first number tracks Google's appetite for answering these queries inline, the second tracks who wins when it does — they move independently, and they mean different things when they do.

Where the numbers go wrong

• A failed record hides behind HTTP 200. raise_for_status() inspects the HTTP response, and a run whose overview was absent still returns 200 — the failure lives on the answer record. Check answer.get("status") before touching sources, or an absent overview becomes a crash at best and a silent miscount at worst.
• Looping until an overview appears rigs your presence rate. Absence drifts between runs the way citation sets do, so a retry can genuinely find an overview the first fetch didn't — but retry-until-success sets presence to 100% by construction and destroys the metric. Record the absence for this run and let the next scheduled run ask again.
• Unstripped URLs split your winners. Google's own srsltid parameter, campaign tags, www. prefixes and trailing slashes turn one page into three or four Counter keys, and your most-cited page ranks below copies of itself. Lowercase the host, drop www., cut the query string — before counting, not after the report looks odd.
• A changed locale looks like a market shift. Edit country or language and the series continues in the same file, now measuring a different SERP. If the pair must change, start a new series — the per-row columns exist so an accidental mix can be unpicked.
• A short timeout works right up until it doesn't. This surface rides the search results page and usually returns fast, which lets timeout=30 survive testing. The API's contract is still a hold of up to 180 seconds on a slow fetch — timeout=200 costs nothing when responses are quick and saves the run when one isn't.

Summary

requests.post, raise_for_status(), timeout=200, read answers[0]["sources"] — with country and language in the body, because this is the surface that reads them. The script worth keeping adds the two counters this engine forces on you — overviews delivered out of queries asked, citations out of overviews delivered — normalizes URLs before they hit the Counter, and appends timestamped rows so the next run has something to diff against. The same call works from Node.js and cURL, and the Python API page covers the endpoint beyond this one task.