Nobody actually uses the thirty-question checklist
We've written those checklists. So has everyone else. A founder prints one, asks the first four questions, and by question five the meeting has drifted into a conversation about a traffic graph. The remaining twenty-six get skimmed in the car park afterwards.
The deeper problem isn't attention span. It's that a pitch is a controlled environment. The agency picked the case studies, built the slides, chose the reference client, and has already run this deck past thirty other founders. Every question you ask about their wins gets answered from material that strangers have stress-tested on your behalf.
So don't ask about a win. Ask about a loss. One question: name a client who left you in the last twelve months, and tell me why they left.
It's the only question in the room nobody has prepared for, because no agency builds a slide called "people who fired us".
A churn story tells you what a case study structurally can't
Case studies are selected by the seller. That is the only sampling method in existence with a guaranteed hundred percent success rate. Even when every number in one is true — and most are — the format has four problems baked into it.
None of this makes case studies dishonest. It makes them close to useless for the decision you're actually making, which isn't "can this agency ever succeed". It's "what happens when this goes wrong on my account, at my budget, in month five".
A churn answer does the one thing a case study cannot: it forces the agency to describe the conditions under which their model fails. Every model fails somewhere. An agency that can't tell you where theirs fails either hasn't been running long enough to find out, or has decided not to tell you.
- Survivorship. You see the engagements that worked. You never see the denominator — how many clients had to start for three of them to end up in the deck.
- Attribution. A case study rarely separates the agency's work from a funding round, a product launch, a seasonal spike or a competitor going bust in the same quarter.
- Recency. The numbers were true once. A 2023 result on a site that has since been replatformed describes a team that may not exist any more.
- Consent bias. The clients who agree to appear are by definition the delighted ones, and quite often the ones who got a fee discount for saying yes.
The four answers you'll hear, graded
Ask fifty agencies and you'll get four answers. The reason itself barely matters. What matters is the shape of the answer, because shape is very hard to fake under mild social pressure with three people watching.
| What they say | What it actually means | Grade |
|---|---|---|
| "Honestly, nobody's ever left us." | Either untrue, or true because the contracts run twelve months and nobody has reached the end of one yet. Both readings are bad; the second is worse, because it's structural rather than accidental. | Fail |
| "They weren't the right fit / they didn't really understand SEO." | The client is the defendant. Fine once. If the same story covers two or three departures, you've found a pattern inside the agency, not in the market. | Weak |
| "Budget got cut. New CMO. They got acquired." | Frequently true and conveniently unfalsifiable. Reasonable as one answer among several, suspicious as the only answer available. | Neutral |
| "We missed. Here's the number we committed to, here's what we hit, here's the call we got wrong in month two." | Rare, and near impossible to improvise at that level of detail. Nobody rehearses their own failure with dates in it unless it happened to them. | Pass |
Three follow-ups that check the story is real
The story survives the first telling. It rarely survives the second question, because a real event leaves consequences behind it and a borrowed one doesn't. Ask these conversationally, in order, and stop when you're satisfied.
- "What did you change afterwards?" A genuine loss changes a process. You want the specific change — a new onboarding step, a client size they now decline, a scope item they refuse to sell, a rule that technical fixes ship before content starts. "We learned a lot from it" means the story came from somebody else's engagement.
- "Would they take a call from me?" You will almost certainly never make the call. You're buying the two seconds before the answer. An agency that parted honestly says "probably — let me ask" without flinching. An agency that got fired badly finds a reason instantly.
- "What's the shortest engagement you've ever had, and why did it end there?" The cross-check. The shortest engagement is the honest edge of their model: the client profile, budget band or industry where their approach stops working. Every agency has one. The good ones name it, then tell you whether you're standing in it.
- "How many clients did you have this time last year, and how many are still here?" The cold one, and optional. Retention as a number instead of a story. If nobody in the room can produce it inside ten seconds, nobody in the building tracks it — which is its own answer.
Lock-in makes the question easy to dodge
Contract length changes what a churn answer means, so establish it before you ask.
An agency selling twelve-month terms with ninety days' notice — the arrangement an SEO contract most often defaults to — has manufactured its retention. A client who wants out in month five stays until month twelve, because leaving early costs three months of fees for nothing. The agency then reports near-perfect retention while quietly doing a fraction of the work it did in the first quarter. The churn is real. It's just deferred, and invisible from outside.
Month-to-month has the opposite property. Every month is a re-signing decision. There's no contractual friction to hide behind, so unhappiness surfaces immediately and departures are visible to everyone including the agency. Agencies on those terms have more churn stories and better reasons for each one.
Which produces a small paradox worth holding onto: an agency with a few honest departures on short terms is usually the safer buy than one with none on long terms. And if an agency has no lock-in and genuinely no churn stories, it's either very new or not telling you the truth. "We're eighteen months old, we've had nine clients, none have left yet" is a perfectly respectable answer — as long as somebody says it out loud.
Our answer, since we're the ones telling you to demand one
It would be cheap to publish this and then dodge it, so here's how we handle the same question pointed the other way.
Structure first, because structure is what makes the question answerable at all. We take three clients a month. After the first quarter the engagement is month-to-month with thirty days' notice, and you keep every asset — content, accounts, dashboards, all of it — the day you leave. There's no contractual mechanism for us to hold a client through a bad quarter, which means our retention is a live signal rather than a legal one.
What we won't do on a public page is publish a departing client's name next to their numbers. They didn't sign up to be a cautionary tale, and an agency willing to do that in a blog post is telling you exactly how your data gets treated after you leave. On a call, we'll name the engagement, the month and the reason.
Whenever an engagement ends, the reason lands in one of three boxes — and this is true across the industry, not just here. One: the client side couldn't ship. Approvals stalled, dev tickets sat for eight weeks, the subject-matter expert never made the interview. Two: the business changed shape mid-engagement, which is nobody's fault and happens constantly. Three: we misjudged how competitive the keyword set was at the budget we agreed. The third is ours. It's the one worth interrogating, and it's why we now decline engagements where the arithmetic doesn't work rather than taking the retainer and hoping the market softens.
The commitment underneath is deliberately narrow. We don't promise a ranking position for a keyword, because nobody controls Google's index and anyone who says otherwise is selling you a coin flip. We freeze your trailing-ninety-day count of qualified leads from organic search on day one, and we guarantee movement against that number. Miss it in ninety days and we keep working free until we beat it — how the guarantee actually works has the mechanics.
What to do with the answer once you have it
The question isn't a gotcha and shouldn't be delivered like one. You're not trying to catch anybody out. You're trying to find the failure mode of their model, so you can check whether your business happens to be shaped like it.
Ninety seconds, one question, three follow-ups. In our experience it reorders a shortlist more reliably than any side-by-side proposal comparison — mostly because proposals are written to be compared and this answer isn't. If you want the full vetting sequence afterwards, what to ask an SEO agency has the longer list. Just ask this one first, while you're still able to walk away.
- Specific, unflattering and checkable — proceed, then ask the genuinely useful second half: "is there anything about us that looks like that client?" You've just been handed their failure profile. Use it.
- A composite with no client, month or number — ask once more, plainly: "I mean one client, one month." One more dodge and you have your answer without needing to be rude about it.
- "Nobody's ever left" — go back to the contract length. The term will explain the retention, and if the term is twelve months you now know what's really being sold.
- "They're technically still on contract but stopped replying in month four" — the most honest answer in the set, and one almost nobody gives. If you hear it, take that agency seriously.