Build an agency scorecard around decisions and outcomes

Most agency scorecards measure service. A better one also scores business outcomes, decision quality and transparency, on a single page reviewed twice a year.

Most agency scorecards measure the relationship. Was the team responsive? Did they meet deadlines? Were the meetings useful?

Those things matter, but they say little about whether the agency is helping the business grow. A better scorecard measures decisions and outcomes as well as service.

Why typical scorecards fall short

Relationship scores tend to drift upwards over time. People are polite to suppliers they work with every week.

They also reward activity. An agency that produces a lot of work can score well even if the work has little commercial effect.

A useful scorecard balances four areas: outcomes, decision quality, efficiency and the relationship itself.

Area 1: Business outcomes

Start with the results the agency is meant to influence. Pick no more than three.

Examples include new customers acquired, sales from a target segment, qualified leads, brand awareness in a defined audience or share of search.

Agree how each will be measured before the period begins. Note honestly where the agency’s influence is partial, because other teams and market conditions also affect results.

Area 2: Decision quality

This is the area most scorecards miss.

Ask whether the agency improved the client’s decisions. Did it bring evidence that changed a plan? Did it recommend stopping something that was not working? Did it test assumptions before large spend?

Score each with real examples. An agency that only executes instructions is a supplier. One that improves decisions is a partner.

Area 3: Efficiency and transparency

Here the evidence from the UK is useful.

In 2020, ISBA, the body representing UK advertisers, published a programmatic supply chain study carried out by PwC. It found that, on average, 51% of advertiser spend reached publishers.

A further 15% could not be attributed to any participant. The study also found inconsistent contract terms and data definitions.

That makes transparency a fair thing to score. Useful measures include:

• The share of media spend reaching publishers or platforms as working media.
• Whether log-level data and audit rights are provided on request.
• How the agency tracks and reduces waste, such as invalid traffic, non-viewable impressions and excessive frequency.

The types of waste listed in the last point are the same ones Procter & Gamble said it targeted, as reported by CNBC in 2018.

Area 4: The relationship

Keep a short section on service quality, but make it specific.

Score responsiveness, proactivity, quality of the senior team’s involvement and how well the agency handles mistakes. Ask both sides to score each other. A client that briefs badly or pays late also damages the relationship.

A simple scoring format

Use a one-page scorecard, reviewed twice a year. For each area:

1. Agree two or three measures in advance.
2. Score each on a 1 to 5 scale, with a written reason and an example.
3. Weight the areas. A reasonable starting point is outcomes 35%, decision quality 25%, efficiency and transparency 25%, relationship 15%.

The weights are our suggestion, not an industry standard. Adjust them to the agency’s role. A creative agency might carry more weight on decision quality, a media agency more on efficiency.

An illustrative example

The following example is hypothetical.

A Belgian B2B software company reviews its performance agency. The relationship score is high: the team is friendly and fast.

The outcome score is middling. Leads are up, but qualified leads are flat.

The decision-quality score is low. In six months, the agency has not recommended stopping a single campaign.

The transparency score is low too. The agency cannot say what share of programmatic spend reached publishers.

The overall score shows a pleasant supplier that is not improving the business. The review leads to a clear brief: introduce lead-quality measures, share placement data and propose two campaigns to cut.

Run the review well

Share the scorecard with the agency at the start of the period, not at the end. People improve what they know will be measured.

Hold the review as a two-way conversation. Ask the agency what the client could do better, and record the answer.

Link part of the fee to the scorecard only once the measures have been tested for at least one cycle. Untested measures can produce unfair results.

Keep the completed scorecards on file. Over two or three cycles, they show whether the relationship is improving, and they give a new marketing leader an honest record instead of a handover based on impressions.

Common mistakes

Too many measures. Twenty metrics dilute attention. Keep each area to two or three.

Moving targets. Changing measures mid-period makes scores meaningless and damages trust.

Scoring without evidence. A score of 4 with no example is an opinion. Require one example for every score.

Ignoring the client’s role. Poor briefs, late approvals and shifting priorities lower agency performance. Score them too.

Keep it light

A scorecard should take hours, not weeks. If it becomes a large administrative task, people will fill it in mechanically.

The test

After each review, ask: did this scorecard tell us something we did not already know? If not, change the measures.