What the export gives you that the interface can't
The Search Console performance report is a good tool wearing a small window. The table view stops at 1,000 rows, the query and page dimensions can't be crossed with each other in the interface, and everything older than sixteen months is gone.
The bulk export removes all three constraints at once. It contains all the performance data available to Search Console for your property, with the exception of anonymised queries — the rare-query rows Google withholds for privacy, which arrive flagged rather than absent, so you can at least see how much you're missing.
You get three tables, partitioned by date.
| Table | One row per | What you use it for |
|---|---|---|
searchdata_site_impression | Query, country, device, search type, per day — aggregated across the whole property | Site-level query trends and brand versus non-brand splits. Smaller, cheaper to scan. |
searchdata_url_impression | URL and query, plus country, device and search-appearance flags, per day | Everything interesting: cannibalisation, page-level decay, the long tail. Also the big one. |
ExportLog | Successful export event | Checking the pipeline actually ran. The first thing to look at when a dashboard goes flat. |
The setup, step by step
You need to be an owner on the Search Console property and have permission to create resources in a Google Cloud project. Budget twenty minutes if both are true and a week if you have to ask someone in IT for the second one.
- In Google Cloud, create a project or pick an existing one. Note the project ID, not the project name — they're different, and using the name is the second most common failure here.
- Enable the BigQuery API and the BigQuery Storage API on that project.
- In IAM, add the service account
search-console-data-export@system.gserviceaccount.comas a principal. - Grant it both BigQuery Job User and BigQuery Data Editor, at the project level. This is the step everyone gets wrong: people grant one role, or grant them on a dataset that doesn't exist yet. Two roles, on the project.
- In Search Console, open Settings, then Bulk data export. Enter the project ID and a dataset name.
- Choose the dataset location. Pick the region you'll actually query from and pick it carefully — you can't move a dataset afterwards without recreating the whole export.
- Save, and wait up to 48 hours for the first export. Check
ExportLogto confirm it ran rather than assuming. - Set a partition expiration on both
searchdata_tables. The minimum is 14 days; most sites want somewhere between 18 and 36 months. Skip this and the tables accumulate forever.
What it costs, in rupees
This is the part the setup guides skip, and it's the only part with a recurring number attached. Two lines drive it: how much data you keep, and how much data your queries read.
Google's free tier covers 1 TiB of querying and 10 GiB of storage per month, which is more generous than it sounds. Most Indian sites we see never leave it. The rupee figures below are our estimates from that arithmetic, not a quote — per-unit prices are published by Google and they change, and a single careless query can cost more than a year of storage.
- Storage is predictable. Queries are not. A
SELECT *across every partition of a largesearchdata_url_impressiontable, run by someone building a dashboard, can burn a meaningful share of a month's free allowance in one afternoon. - Always filter on
data_datefirst. The tables are partitioned by it, so a date filter is what stops BigQuery reading years of data to answer a question about last month. - Materialise the things you look at weekly. A scheduled query that writes a small summary table once a day is cheaper than a Looker Studio dashboard that re-queries the raw table every time someone opens it — which is the single most common way an SEO team generates a surprise bill.
- Use the byte estimate. The BigQuery editor tells you how much a query will scan before you run it. Look at it once and the habit sticks.
| Site size | What the export produces | Realistic monthly cost |
|---|---|---|
| Under ~2,000 clicks/mo, a few hundred URLs | Well inside 10 GiB even after two years. Queries scan megabytes. | ₹0. You will not leave the free tier, and honestly you may not need the export at all. |
| ~20,000–50,000 clicks/mo, a few thousand URLs | Storage passes 10 GiB somewhere in year one or two if you never expire partitions. Query volume stays small if you filter by date. | ₹0–₹300. Set an 18-month partition expiry and it usually stays at zero. |
| 100,000+ clicks/mo, ecommerce or marketplace scale | The URL-impression table fans out fast — every URL × query × country × device combination is a row. Tens of GiB a year. | ₹500–₹5,000. The spread is entirely about query discipline, not about your traffic. |
The traffic level below which this isn't worth doing
We'll say the unhelpful-to-us thing: most sites that ask us to set this up don't need it. The export solves a truncation problem, and you have to be big enough to be truncated.
Three tests, and you need at least two to clear the bar:
- Does your query table hit 1,000 rows in the last three months? Open the performance report, set the range to 90 days, and look at the query list. If it comfortably fits, nothing is being hidden from you and the export changes nothing.
- Do you have more than a few hundred URLs earning impressions? Page-level decay analysis is the strongest reason to do this, and it needs a long tail to be worth analysing. Forty pages you can read one by one.
- Will anyone actually write SQL against it? This is the real filter. An export nobody queries is a dataset with a storage bill. If your team's analysis ceiling is a spreadsheet export, spend the twenty minutes somewhere else.
The three analyses that pay for it
Everything below is impossible or painful in the interface, and straightforward once the data is in a table. These are the three we run first on any new account with an export.
Query-level cannibalisation
Group searchdata_url_impression by query, count distinct URLs receiving impressions, and keep the queries where that count is above two and the click share is split rather than dominated by one page. That list is your keyword cannibalisation backlog, ranked by how much traffic is actually at stake — which is the ranking the interface can't give you, because it won't cross query and page.
Shape: SELECT query, COUNT(DISTINCT url) AS urls, SUM(clicks) FROM searchdata_url_impression WHERE data_date >= ... GROUP BY query HAVING urls > 2 ORDER BY 3 DESC.
Brand versus non-brand, at full precision
Brand queries flatter every report. In the interface you can filter them out with a regex and get an approximation capped at 1,000 rows. In BigQuery you classify every row, including the tail, and you can hold the split constant month over month.
Shape: a CASE WHEN REGEXP_CONTAINS(query, r'(?i)yourbrand|your brand|yourbrnad') flag, aggregated by month. Include the misspellings — they're brand traffic too, and on Indian brand names there are usually four or five worth catching.
Page-level decay across the long tail
Sum clicks per URL for the trailing 28 days and the same 28 days ninety days earlier, then rank by absolute decline. This surfaces the fifty pages each losing a little — which together outweigh the one page everybody noticed losing a lot. It's the only reliable input to a rewrite queue, and it's the analysis that most often changes what a content team does next month.
Pair it with average position over the same windows. A page losing clicks at a stable position lost the SERP, not the ranking, and needs a different fix from a page that simply slipped.
What this does not fix
Worth saying, because BigQuery has a way of becoming the answer to questions it can't answer.
It doesn't add data Search Console never had. Anonymised queries stay anonymised, impressions still count the same way, and average position is still an average with all the distortions that implies. Nor does it reconcile anything with analytics — Search Console and GA4 count different events on opposite sides of the click, and putting one of them in a warehouse doesn't change that.
And it doesn't make anyone read the numbers. We've seen exports running for two years in accounts where nobody has opened the dataset. The setup is the easy twenty minutes; the standing weekly habit of querying it is the part that actually produces a decision.