Explaining marketing results when attribution is incomplete

eBay's paid search experiments found attribution overstated returns dramatically. How to give the board an honest account of what is measured, estimated and unknown.

Every CMO has faced the question: what did we get for the money? Attribution reports offer a quick answer. Sometimes that answer is badly wrong.

A series of experiments at eBay is one of the clearest demonstrations of how wrong it can be, and of a better way to answer the board.

Why the question is getting harder

Budgets are under pressure. Gartner’s 2026 CMO Spend Survey, of 401 marketing leaders in North America, the UK and Europe, found marketing budgets were effectively flat at 7.8% of company revenue, up from 7.7% in 2025.

The same survey found 56% of CMOs said they lacked the budget needed to deliver their 2026 strategy. In that climate, every line of spend needs a credible explanation.

The eBay experiments

Economists Thomas Blake, Chris Nosko and Steven Tadelis, working with eBay, ran large field experiments on paid search. Their paper was published in Econometrica in 2015.

Brand keywords. eBay stopped buying ads on searches for its own brand on Yahoo! and MSN. The researchers found that 99.5% of the lost paid clicks were replaced by clicks on eBay’s free search listings.

In other words, people searching for “eBay” found eBay anyway. A later test in Germany, where brand ads were paused in a random half of the country’s 16 states, confirmed the result.

Other keywords. For non-brand searches, ads did influence new and infrequent eBay users. But frequent users, whose purchases were not affected by the ads, accounted for most of the spending.

The attribution gap

The most striking part of the paper is the comparison of methods.

Using standard non-experimental analysis, the authors calculated a return on investment for paid search of over 4,100%, or over 1,400% with some controls. Their experimental estimate was minus 63%, with a wide confidence range that still excluded break-even.

The difference comes from a simple problem. People who were already going to buy are also more likely to search and click. Attribution credits the ad for a sale that would have happened anyway.

These results are specific to eBay, a well-known brand, at that time. They are a warning, not a rule that paid search never works.

What the board actually needs

A board rarely needs a perfect number. It needs an honest view of what is known, what is estimated and what is uncertain.

A useful structure has three layers.

What we can measure directly. Sales, leads and customers from channels where cause and effect are clear, such as a controlled test.

What we estimate. Effects measured through models, with the method and its limits stated.

What we believe but cannot yet measure. Long-term brand effects, word of mouth or influence on buying committees, stated as judgement.

Use more than one method

No single tool answers every question. Three approaches work well together.

Experiments. Switching spend off or on in randomly chosen regions or audiences, as eBay did, gives the cleanest evidence of incremental effect.

Marketing mix models. These statistical models estimate how each channel contributes to sales over time. Google publishes an open-source model called Meridian, which it describes as a flexible statistical framework that anyone can use and modify.

Attribution. Click-based attribution remains useful for day-to-day optimisation within channels. Treat it as a guide to relative performance, not as proof of cause.

When the methods disagree, that is a finding. Report it.

How to present uncertain results

Give ranges, not single numbers, when the evidence is uncertain. Say “between £1.20 and £2.10 for every £1 spent” rather than a precise figure that implies false confidence.

Separate short-term and long-term effects. Many attribution systems capture only the first.

Name the decisions that depend on each number. That shows the board why precision matters for some lines and not for others.

An illustrative example

The following example is hypothetical.

A Danish insurer’s attribution report credits brand search with a large share of online policy sales. The CMO commissions a regional test, pausing brand search ads in a random half of regions for six weeks.

Sales in the paused regions fall only slightly compared with the others. The insurer cuts most brand search spend and moves part of the budget to a test of non-brand search for new customers.

The board report shows the original attributed figure, the experimental estimate and the decision taken. That transparency builds credibility for the next request.

The trade-offs

Experiments cost time and can mean giving up some sales during the test. Models need data and expertise. Neither is free.

But the eBay results suggest the cost of not testing can be far higher. Spending on ads that do not change behaviour is waste, however good the attribution report looks.

The takeaway

Incomplete attribution is not a reason to avoid accountability. It is a reason to be precise about what is known.

The CMO who tells the board “here is what we measured, here is what we estimate, and here is how we will find out more” is usually more trusted than one with a single, confident number.