What is synthetic control?
How does synthetic control work in a geo experiment?
A geo test changes ads in one region. No other region is a perfect twin of yours.
So the method blends several. One analytics vendor's documentation describes it as a weighted combination of untreated units that closely matches the treated unit's pre-intervention trajectory2. In plain terms: weight the untreated regions until the blend follows your region's sales before the change. After the change, the blend is your estimate of what would have happened without it. The gap between your region and the blend is the lift.
Why does the pre-test fit matter?
The whole result rests on the blend tracking your region before the test. The same vendor's documentation says poor pre-period fit invalidates the post-period estimates, and that fit must be checked before the result is read2.
It also describes placebo tests: the method is re-run for each untreated region as if it had been the test region, and your region's effect is compared with those placebo effects2. If untouched regions show "effects" as large as yours, yours is not convincing.
Our view: ask to see the fit before you see the result. A tidy answer built on a poor fit is still a poor answer.
Matched markets or synthetic control?
A matched-market test pairs a test region with one similar region. Synthetic control blends several. One incrementality-testing vendor, Haus, argues that a single control region carries its own outlier events, and that a blend of several regions mirrors the test region better3. That is a vendor view, and it favours the method the vendor sells. The practical question: how well did the control track you?
Worked example
Easily confused with
Synthetic control vs Geo test
The experiment itself. Synthetic control is one way to build its control.
Synthetic control vs Spillover
When ads reach people in the control region. A flaw in the test, not a method.
Synthetic control vs Minimum detectable effect
The smallest lift a test design can reliably detect. A property of the design, not of the control.
The idea behind all of these is incrementality: the extra sales an activity causes beyond what would have happened anyway.
FAQ
Is synthetic control a type of geo test?
No. The geo test is the experiment. Synthetic control is one way to build its comparison.
Can a synthetic control be wrong?
Yes. A poor match before the test, or ads leaking into the regions used for the blend, can both distort the result.
Do I need to build one myself?
You need to understand it well enough to question it. Free tools exist, such as GeoLift, which is MIT-licensed1. The judgement about fit still has to come from someone who reads the output.
Sources
Let’s find what’s actually working.
Next step. Before testing, check what your current ROAS assumes. Use the Is your ROAS real? calculator.