Post-purchase attribution surveys: useful, biased, and how to combine them with MER

A "how did you hear about us" (HDYHAU) survey asks the customer one question after checkout: where did you first hear about us? Done well, it adds something your click data cannot see.

What it can do:

• Show channels with no click. Podcasts, word of mouth and TV often leave no click trail. Attribution models assign credit by rules on tracked touchpoints. Surveys can show what those rules miss.
• Give a second view. Last-click attribution credits the final click. A survey asks about the beginning of the story.
• Cost little. One question, no experiment, no holdout.

Survey vendors sell HDYHAU tools for this job. Fairing, for example, sells in-moment attribution surveys that install on Shopify¹. Treat what vendors say about them as sales material.

The case for using several methods side by side comes from elsewhere. A MarTech article by a measurement adviser argues for triangulating MMM, attribution and incrementality tests instead of crowning one source of truth². It does not discuss surveys. We add them as one more signal in that mix.

JumpFly, a digital marketing agency, lists "how did you hear about us?" questions during the purchase cycle among practical options for small and mid-sized advertisers, next to geo, product-based and blackout tests³. Read it as an agency's view.

Surveys record memory and perception. Memory is selective. Know the main distortions.

• Recency. Customers name what they saw last, not what started the journey.
• Memorable beats effective. A creator they liked is easy to recall. A retargeting ad they barely noticed is not.
• "Google" as a catch-all. JumpFly warns that people often use "google" as a verb, so a "Google" answer can be misleading³. Search can collect credit for whatever came before the search box.
• Option effects. The answer list steers the answer. A channel missing from the list gets filed under "other" or something close.
• Who answers. People who complete a survey are not a random sample of buyers. High response rates are quoted by survey vendors, for example Fairing, without a cited source⁴. They are vendor claims.
• First purchase vs repeat. A returning customer answers about a habit, not about acquisition.

None of that makes the survey useless. It makes it a signal with a known tilt. Say so whenever you show the numbers.

Because the survey does not measure incrementality: the extra sales a channel causes over and above what would have happened without it. It measures what people recall. The two can diverge in both directions.

A channel that few people remember can still be doing work. An ad seen in passing, such as retargeting, is easy to forget. Cut it because "nobody mentions us there" and you may find out the hard way. JumpFly reports that some of its clients who cut channels on survey results saw site traffic and business drop³.

The reverse also happens. A channel that gets many mentions may be getting credit for demand that was already there.

Our view: a survey answer is a hypothesis about a channel. It becomes a decision only after a second method agrees. If you cannot afford a test, our piece on when not to run an incrementality test lists cheaper methods and the bias each one carries.

Our convention: MER (marketing efficiency ratio) is revenue divided by all marketing cost, not only media spend. Blended ROAS, in our convention, is revenue divided by media spend alone. MER is useful here because it does not split credit by channel, so two platforms claiming the same order do not inflate it.

A workable routine:

1. Run the survey continuously. One question at checkout, reviewed monthly, with the answer options fixed.
2. Track the share of each answer over time. Look at the movement, not the absolute level.
3. Overlay MER. Plot it for the same weeks, with a note on every budget change.
4. Look for matches. If the share of "podcast" answers rises clearly after a spend increase and MER holds or improves, that is consistent. It is not proof.
5. Look for conflicts. If a channel's mentions fall but MER stays flat after a cut, the cut probably cost little. If MER drops, the survey missed something.
6. Set a baseline. Write down the baseline, meaning the level of sales you expect without the change, before you change anything.
7. Escalate big calls. For a large cut or increase, add a regional pause or a test.

Example (illustrative): a brand cuts a creator programme. The share of "creator or influencer" answers falls, as expected. MER stays level for the following weeks and the "search" answers rise. That pattern is consistent with the cut having cost little, though search answers can hide the creator effect. A regional pause would settle it.

Wording shapes the data. Rules we would follow:

• Ask one question. "How did you first hear about us?" Not three.
• Offer a short list plus "other". Include the channels you run, and add a free-text field for "other".
• Add a "Search" option with a prompt. For example, "Search (Google, Bing, etc.). What did you look for?" The free text shows whether people searched for your brand or for a product.
• Randomise the order of the options, so the order does not steer the answer.
• Do not lead. Avoid options that flatter you.
• Keep it after the order, so you do not add friction before payment.
• Read the free text. It is where new channels show up.