What is spillover in geo tests?
Why does spillover matter?
A geo test changes ads in some regions and leaves others alone. The untouched regions are the control. They show what would have happened without the change.
If shoppers in the control region see the ads anyway, the control is not clean. The gap between regions can shrink. The ads then look less effective than they are.
The same logic works in reverse. Any mixing between test and control blurs the comparison.
Why use regions at all?
Because splitting by person is not always possible. Google's help page on lift types compares the two approaches. It says geo-based studies tend to need a higher budget than user-based ones and that their group split is noisier1. In return, Google lists cross-channel measurement and online plus offline conversions1. Regions are a trade. You accept noise, and you must also guard against leakage.
How do you reduce it?
Spillover is a design problem, so the questions come before the test. Where do the regions' borders fall? Do people move between them? Can targeting stay inside each region?
Our view: a geo test is only as clean as its map. Read the region design before you read the result.
If the regions did leak, the result is harder to read. An inconclusive result is a legitimate finding, and better than a confident wrong one.
Worked example
Easily confused with
Spillover vs Halo effect
The effect of one channel on other channels. Spillover belongs to test design only.
Spillover vs Cannibalisation
Paid activity taking credit for sales that would have arrived anyway. A measurement problem about credit, not about region borders.
Spillover vs Synthetic control
A way to build the control from a weighted mix of untreated regions. It does not remove spillover by itself.
The wider concept is incrementality: the extra sales an activity causes, beyond what would have happened anyway.
FAQ
Is spillover the same as the halo effect?
No. Halo is one channel affecting another channel. Spillover is ads from one region reaching people in another, which distorts a test.
Can spillover be fixed after the test?
It is a design problem, so the fix is usually a better design for the next test. Treat a leaky result with caution.
Why not split people instead of regions?
Often you should. Google's help page says geo-based studies tend to need a higher budget and give a noisier split1. It also notes that user-based studies rely mostly on opt-in traffic, so they can be affected by measurement gaps1. Regions make sense when you need to measure across channels or include offline sales.
Sources
- Comparing lift types (Google Ads Help)Primary source
Let’s find what’s actually working.
Next step. Before you test, see what your current ROAS assumes. Use the Is your ROAS real? calculator.