Get Microsoft Copilot citations in cURL

Everything here is curl and jq: POST the query, read .answers[0].sources[], shape it with a filter. What's specific to Copilot sits around the pipe, not inside it — it grounds in Bing so the list is usually populated, country steers which Bing results you get, and a slow scrape fails soft with a redeemable id. The manual-versus-API comparison lives in the task-level guide; this page stays in the shell and ends with a ledger cron can keep growing.

The call, with the one field that matters

Prerequisites: curl, jq and an ag_live_ key from onboarding — or run one fetch keyless in the playground first. The body is query, surfaces and country. That third field is the one the endpoint actually reads on this surface: Copilot honours country, which shifts the Bing results it cites. --max-time 200 sits above the API's own 180-second hold, so curl doesn't hang up while a slow scrape is legitimately still running.

curl -s --max-time 200 https://api.agentgeo.org/v1/fetches \
  -H "Authorization: Bearer ag_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{"query": "best email marketing platform for ecommerce", "surfaces": ["copilot"], "country": "US"}' \
  | jq -r '.answers[0].sources[] | [.position, .url] | @tsv'

Two columns out — position, then URL, in the order Copilot cited them. If nothing prints, the filter isn't broken: Copilot answered without searching Bing and the record still holds the full answerText. One line tells you which case you got:

curl -s --max-time 200 https://api.agentgeo.org/v1/fetches \
  -H "Authorization: Bearer ag_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{"query": "best email marketing platform for ecommerce", "surfaces": ["copilot"], "country": "US"}' \
  | jq -r '.answers[0].sources | if length > 0 then "cited, \(length) sources" else "delivered, uncited" end'

Behind both filters, the full response — one record for the one surface requested:

{
  "id": "run_a58c1e0f7d29",
  "query": "best email marketing platform for ecommerce",
  "surfaces": ["copilot"],
  "status": "completed",
  "recordsDelivered": 1,
  "creditsCharged": 1,
  "answers": [
    {
      "surfaceKey": "copilot",
      "answerText": "For an ecommerce store the platforms that come up most are ...",
      "sources": [
        { "title": "Best Email Marketing for Ecommerce 2026", "url": "https://example.com/email-ecommerce", "position": 1 },
        { "title": "Klaviyo vs Mailchimp for Online Stores", "url": "https://example.org/klaviyo-vs-mailchimp", "position": 2 }
      ],
      "fetchedAt": "2026-07-27T10:47:19Z"
    }
  ]
}

position is the field to preserve — flatten sources[] into a set of links and the ranking is gone. fetchedAt is what makes any two runs comparable. The uncited variant of this record is identical except for "sources": [] — same status, same one credit for the delivered record. Failed records cost nothing.

Several queries, one dated ledger

One run answers today's question. The two questions worth a script — which hosts recur across a topic, and how often Copilot cites at all — need a query set, one normalization convention, and rows that outlive the run. Queries go in a plain file, one per line:

best email marketing platform for ecommerce
klaviyo alternatives for a small store
email marketing with abandoned cart flows
mailchimp vs omnisend for shopify

#!/usr/bin/env bash
# copilot-citations.sh — Copilot citations for every query in queries.txt,
# appended as dated rows to a per-market CSV. Two tallies per run: which
# hosts got cited, and how many queries were cited at all.
set -euo pipefail

KEY="ag_live_your_key_here"
COUNTRY="US"                      # steers Bing — one CSV per market
CSV="copilot-citations-${COUNTRY,,}.csv"
TODAY=$(date -u +%F)

# Comma-safe columns first, free text last — cut and awk stay honest.
[ -f "$CSV" ] || echo 'date,cited,position,n_sources,host,query,url,title' > "$CSV"

delivered=0
cited=0

while IFS= read -r query; do
  [ -n "$query" ] || continue

  # jq builds the body — an apostrophe in a query can't break the JSON.
  body=$(jq -cn --arg q "$query" --arg c "$COUNTRY" \
    '{query: $q, surfaces: ["copilot"], country: $c}')

  resp=$(curl -s --max-time 200 https://api.agentgeo.org/v1/fetches \
    -H "Authorization: Bearer $KEY" \
    -H "Content-Type: application/json" \
    -d "$body") || { echo "curl failed: $query" >&2; continue; }

  if [ "$(jq -r '.status' <<<"$resp")" != "completed" ]; then
    # Zero credits — the record may carry a snapshot_id worth redeeming.
    snap=$(jq -r '.answers[0].providerFields.snapshot_id // ""' <<<"$resp")
    echo "not completed: $query (snapshot_id=$snap)" >&2
    continue
  fi

  delivered=$((delivered + 1))

  if [ "$(jq '.answers[0].sources | length' <<<"$resp")" -eq 0 ]; then
    # Delivered but uncited — write the row anyway. The zeros are a metric.
    jq -rn --arg d "$TODAY" --arg q "$query" \
      '[$d, 0, "", 0, "", $q, "", ""] | @csv' >> "$CSV"
    continue
  fi

  cited=$((cited + 1))
  jq -r --arg d "$TODAY" --arg q "$query" '
    .answers[0].sources as $s
    | $s | sort_by(.position)[]
    | [$d, 1, .position, ($s | length),
       (.url | sub("^https?://(www\\.)?"; "") | split("/")[0] | ascii_downcase),
       $q, (.url | split("?")[0]), .title]
    | @csv' <<<"$resp" >> "$CSV"

done < queries.txt

if [ "$delivered" -gt 0 ]; then
  echo "cited rate this run: $cited/$delivered"
fi

Three deliberate choices. Bodies are built with jq -cn --arg, because the first apostrophe in a real query would otherwise end the single-quoted -d string mid-payload. Delivered-but-uncited runs are written with cited set to 0 rather than skipped — drop them and every rate you compute later reads 100%. And the comma-proof columns — date, flag, position, count, host — sit ahead of query and title, the two fields that can carry commas, so cut -d, and awk -F, on the leading columns never lie.

# Most-cited hosts across the whole ledger
tail -n +2 copilot-citations-us.csv | cut -d, -f5 | tr -d '"' | grep -v '^$' \
  | sort | uniq -c | sort -rn | head -15

# Cited-rate trend by run date — each query counted once per run
tail -n +2 copilot-citations-us.csv | tr -d '"' \
  | awk -F, '$2==0 || $3==1 { total[$1]++; c[$1]+=$2 }
             END { for (d in total) printf "%s  %d/%d cited\n", d, c[d], total[d] }' \
  | sort

# crontab -e — Mondays 06:40 UTC; each run adds a dated layer
40 6 * * 1 cd /home/you/geo && ./copilot-citations.sh >> copilot.log 2>&1

The first command is the host table. The second is the cited rate, and it only exists because the zero rows were written — the awk condition counts each run once, via the uncited row or the position-1 row of a cited one. After a few cron cycles the same file answers the diff questions — hosts that entered, hosts that dropped, where your own pages moved — as filters over two date ranges. Spend stays visible without a dashboard: a delivered record is one credit whether it cited nine pages or none, and creditsCharged on every response keeps the log honest.

One knob, and it moves the data

Most surfaces take no request levers; copilot takes exactly one that matters. country is read here — it steers the Bing results Copilot grounds on, so a US body and a GB body are different datasets. web_search is honoured only by chatgpt; language only by google_ai_overview, where it maps to Google's hl. Sending either to Copilot documents a knob that doesn't exist, and the next person reading the script will assume it works.

• Bing grounding is why the list is usually full. Copilot searches before it writes, so a populated sources[] is the common case and the cited URLs are real pages — denser than Gemini, broadly on par with Perplexity.
• A delivered record can still be uncited. Copilot sometimes answers from the model without searching; that record carries a full answerText, a fetchedAt and one credit, with "sources": []. Branch on the run's status first and sources | length second — they say different things, and the script treats them differently.
• This is consumer Copilot. The surface is copilot.microsoft.com — not GitHub Copilot and not Microsoft 365 Copilot. Different products, different answers; don't benchmark one against another.

Common problems

• Real queries break single-quoted JSON. The first apostrophe — "what's the best email platform" — terminates a single-quoted -d string and the API receives invalid JSON. Build every body with jq -cn --arg, in one-off commands as well as the script.
• A client timeout below the server's hold. The API keeps the request open up to 180 seconds while a slow scrape finishes; --max-time 200 keeps curl in the room. The version that wastes an afternoon is an outer layer — a CI step, a proxy, a wrapper — with its own shorter limit that cuts the connection at second 60.
• A failed run is not a lost run — and on Copilot you'll hit it. Copilot is a slow surface, so a scrape occasionally exceeds the 180-second budget and returns status: "failed" at zero credits with providerFields.snapshot_id on the record. POST again with top-level "snapshot_id" and the same single surface — ["copilot"] — and the finished answer is redeemed instead of re-scraped.
• Two normalization conventions in one file. The script lowercases hosts, strips www. and drops query strings at write time. Change any of that later and old rows stop being comparable with new ones — restate the whole CSV, or the diff you built the ledger for will report a migration as movement.
• Two markets in one CSV. A US run and a GB run cite different hosts because Bing served different pages. The filename carries ${COUNTRY,,} for exactly this reason — keep them apart, or a later diff reads a locale switch as a ranking change.

# $SNAP is providerFields.snapshot_id from the failed record.
curl -s --max-time 200 https://api.agentgeo.org/v1/fetches \
  -H "Authorization: Bearer ag_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d "$(jq -cn --arg s "$SNAP" '{snapshot_id: $s, surfaces: ["copilot"]}')" \
  | jq '.answers[0].sources'

Summary

One POST returns Copilot's answer and its Bing-grounded citations at a stable jq path; the shell does the rest. Keep --max-time above the 180-second hold, build bodies with jq -cn, pass country and name the CSV after it, write the uncited rows, and put the comma-safe columns first so cut and awk survive punctuation. The ledger then answers both questions this surface poses — which hosts keep getting cited, and how often anyone does — with uniq -c and one awk line. If the job outgrows the shell, the Python and Node.js versions call the identical endpoint; the wider set of shell idioms lives on the cURL API page, and the docs list every request field and which surface honours it.