Build a learning agenda around expensive unknowns

A useful learning agenda connects unanswered questions to decisions with owners and dates. Choose methods that can reduce the uncertainty that matters, then decide in advance how findings will be used.

A team can run many tests and still learn little that changes the business. Button colours, subject lines and small format variations may improve local execution while the largest strategic assumptions remain untouched.

A learning agenda begins with decisions, not with a target number of experiments. It asks what the organisation needs to understand before making its next consequential commitments.

The result should be a short sequence of questions that deserve evidence, with a practical route from each finding to a choice. Give the agenda real capacity in the operating plan so research does not depend entirely on spare time.

Name the decision before the research

Write the pending decision in concrete terms: whether to enter a segment, change an offer, expand a channel or invest in a service capability. Include the decision owner and the date by which evidence would be useful.

Then identify the uncertainty that could change the choice. “Understand our customers better” is too broad. “Determine whether implementation risk is the main reason qualified buyers postpone this offer” gives the team something to investigate.

The Magenta Book connects evaluation questions with their intended use and decision context. For marketing, this helps prevent research from becoming an interesting report with no practical destination.

If no plausible finding would change the decision, be honest about the work’s purpose. It may be communication, documentation or execution planning rather than decision research.

Rank questions by consequence

Estimate how much spending, opportunity or risk depends on the answer. Combine that with uncertainty and the feasibility of learning in time.

A small question may deserve priority if it blocks a near-term decision. A large question may need a staged approach if a definitive answer is expensive or slow.

Do not select only questions that are easy to measure. That can create a busy programme of low-value tests while difficult questions about positioning, customer value or delivery capability remain unexamined.

Keep the first agenda short. Three well-chosen questions with owners and decisions can be more valuable than a long catalogue that competes with delivery work and receives little follow-through.

Match the method to the uncertainty

Customer interviews can reveal language, context and possible explanations. They do not by themselves establish how common a preference is across a population.

Observational analysis can identify patterns and segments worth investigating. It may not establish causation because people receiving one treatment or choosing one product can differ from others.

A controlled experiment can address a causal question when its design, scale and practical conditions are suitable. A staged pilot can also expose delivery problems even when it cannot estimate a precise commercial effect.

Choose the smallest credible method that addresses the next decision. Avoid demanding a perfect study when a bounded early choice needs only enough evidence to proceed, pause or narrow the scope.

State the limits in advance

Identify the sample, period, comparison and measures before the work begins. Record what the design will not be able to answer.

The ONS explanation of uncertainty reinforces that an estimate should be interpreted with its uncertainty. A point estimate is not a complete decision when several materially different outcomes remain plausible.

For a hypothetical pilot, the team might learn whether users can complete setup and which steps confuse them. It should not present that result as a reliable annual revenue forecast if the pilot was never designed to estimate one.

This clarity protects both the research and the decision. A modest study can be valuable when it answers its intended question rather than being stretched into evidence for a larger claim.

Agree the response to possible findings

Describe what strong support, weak support and contradictory evidence would lead the team to do. The options may be expand, revise, investigate further or stop.

Define guardrails where the work could create customer or commercial costs. Decide who can pause the activity and what information they need to make that call.

Do not move the success criterion after seeing the result merely to preserve the preferred conclusion. If a changed business context requires a new criterion, record the change and explain its effect on interpretation.

Keep a decision record

Store the original question, method, result, limitations and final choice together. Include what the team learned about execution, even when the commercial result is inconclusive.

Assign an owner to any follow-up. “More research needed” is an incomplete conclusion unless the organisation knows which uncertainty remains and why resolving it is worth the additional effort.

Review the agenda as decisions change. Remove questions whose answers no longer matter and add new ones only when they earn a place against existing commitments.

The strongest learning programme leaves the business with better choices and fewer unsupported assumptions. Its success is not the number of tests completed, but the number of consequential decisions made with clearer evidence and an honest account of what remains unknown.