The Cost of Being Wrong

Product teams often respond to uncertainty in one of two ways. They either ask for more research by default or celebrate speed and decide with whatever evidence happens to be available.

Both reactions confuse uncertainty with risk.

A reversible change shown to a small cohort can be highly uncertain and still be responsible. A familiar change applied to every customer can be dangerous when its failure is hard to detect or undo.

The useful question is not “How certain are we?” It is “What commitment are we making, who is exposed if we are wrong, and how can we limit the damage?”

A decision is not risky merely because it is uncertain

Uncertainty describes what the team does not know. Risk describes what could happen because of that uncertainty.

The distinction matters. Teams cannot research every unknown, and removing one uncertainty may leave the decision unchanged. Other decisions deserve a much higher standard even when the proposed solution feels conventional.

Assess product-decision risk through four lenses:

• Consequence: what becomes worse if the belief or choice is wrong?
• Exposure: how many people, workflows, commitments, or systems can be affected?
• Reversibility: how quickly and safely can the team detect, stop, and repair the outcome?
• Uncertainty: which material assumptions lack credible evidence?

Risk rises when serious consequences combine with broad exposure, poor reversibility, and important unknowns. No arithmetic is required to see that pattern.

This is also why a low-confidence decision is not automatically irresponsible. Limiting exposure or making the commitment reversible can be more valuable than squeezing another small increase in confidence from research.

Define the outcome, commitment, and deadline

Before deciding how much evidence to gather, state what is actually being decided.

“Should we improve onboarding?” is an area of work. “Should we replace assisted setup for new enterprise accounts next quarter?” is a commitment with operational and customer consequences.

Write three parts:

1. Outcome: whose situation should improve, and what change would count as progress?
2. Commitment: which customers, money, systems, promises, or future options will the choice bind?
3. Decision deadline: when does waiting begin to close a valuable option or create material cost?

The deadline is not a delivery date invented to create urgency. It is the point at which delay changes the decision.

A contract renewal, regulatory date, expiring technical option, or seasonal demand may create a real boundary. An executive’s desire to “move faster” does not define one by itself.

How to Prioritise provides a broader way to compare value, timing, evidence, and portfolio cost when several commitments compete.

Map consequence, exposure, and reversibility

Generic labels such as low, medium, and high risk hide the mechanism. Describe what failure would look like instead.

Consequence

Consider more than conversion or revenue. A wrong decision can create financial loss, exclusion, privacy harm, unsafe behaviour, broken customer work, contractual exposure, or an operational burden shifted to another team.

Ask who carries each consequence. A change that saves time for the buyer may add unpaid work for an administrator or remove a safeguard for an affected person.

Some consequences set a floor under the evidence and oversight required. Legal, security, safety, and ethical boundaries should go to the people with authority and expertise to assess them.

They are not merely more points in a product scoring model.

Exposure

Exposure includes breadth, duration, and concentration.

A change may reach few people yet expose a critical workflow. It may affect many people for only a minute, or a small cohort for years. It may also create a dependency that other teams or customers build upon.

Map the initial audience, possible propagation, and time before the team can observe a meaningful result.

Reversibility

“We can roll it back” is incomplete. A deployment can be technically reversible while customer consequences are not.

Can the team identify who was affected? Restore their state? Repair a promise, lost opportunity, disclosed record, or changed habit? Do customers understand that the experience may change again?

Reversibility has a time, cost, and recovery path. If those are unknown, the decision is less reversible than the release plan suggests.

Compare being wrong with waiting

Research is not free, but speed is not free either. The comparison needs both sides.

The cost of being wrong can include customer harm, rework, support load, lost trust, contractual consequences, and options closed by the decision.

The cost of waiting can include a problem continuing, evidence arriving too late, a market window closing, compounding technical work, or another team being blocked.

Treat “do nothing for now” as an option with consequences. It is not a neutral baseline.

The HM Treasury Green Book is written for public appraisal, not product backlogs.

Its principle of proportionality still travels well: decision effort should reflect scale, cost, complexity, and risk. It also treats business as usual as an option and requires uncertainty to be communicated rather than hidden.

Do not import its financial machinery into every product choice. Use the underlying discipline: compare credible options, name material costs and benefits, and test whether a changed assumption would change the preferred option.

Investigate uncertainty that can change the choice

A long list of unknowns can make any proposal appear unready. Most of those unknowns do not control the next commitment.

For each assumption, ask:

• If it is false, does the preferred option change?
• Could it cause a boundary or guardrail to be breached?
• Is evidence available before the decision deadline?
• Would the evidence distinguish between the live options?

An uncertainty that cannot change the choice may still deserve monitoring. It does not necessarily deserve pre-decision research.

How to Approach Assumption Mapping Systematically shows how to identify the belief that controls the next commitment.

That is more useful than producing an inventory of everything the team does not know.

Watch for false precision. A score of confidence multiplied by impact can organise a conversation, but the decimal does not turn judgement into a measurement.

Keep the reasoning visible: what is believed, why it matters, what supports it, and what the team would do if it proves false.

Match evidence to the commitment

The purpose of evidence is to improve a decision, not to certify that the future is safe.

A small, reversible commitment may need enough evidence to avoid an obvious mistake. A broad or hard-to-reverse commitment needs stronger evidence from more than one relevant source.

Choose methods according to the uncertainty:

• interviews or observation for context, behaviour, constraints, and workarounds;
• prototypes or usability work for comprehension and interaction risk;
• technical spikes for feasibility and failure modes;
• historical or operational data for frequency, distribution, and recurrence;
• limited releases for behaviour in a real setting under controlled exposure;
• specialist review for legal, security, safety, accessibility, or ethical boundaries.

Method volume is not evidence quality. Twenty interviews that ask people to predict purchase may be less useful than a few observations of the decision the product must support.

The GOV.UK Service Manual recommends beginning research plans with the questions a team needs to answer and revisiting them as understanding changes.

That prevents a preferred method from becoming the plan before the decision need is clear.

Give research a stopping rule

Without a stopping rule, research ends when the calendar or patience runs out.

Define in advance what the next evidence must accomplish. It might rule out one option, verify a critical boundary, reveal the dominant failure mode, or make a staged release responsible.

A useful stopping rule names:

• the decision the evidence will inform;
• the assumption or risk being investigated;
• the signal that would support each live option;
• the time or resource boundary;
• what the team will do if the result remains ambiguous.

Ambiguity is a possible result, not a research failure. The correct response may be to reduce the commitment, preserve several options, or accept the remaining uncertainty explicitly.

Research-Driven Hypothesis Testing goes deeper into designing evidence capable of changing a decision.

Reduce the commitment when certainty is unavailable

Teams often debate “launch or do not launch” when the better work is changing the shape of the launch.

Reduce scope to the part supported by evidence. Limit the cohort to people who meet clear eligibility rules. Separate a reversible operational test from a permanent product promise.

Add a manual checkpoint where automation would create unacceptable harm. Preserve the old path until migration evidence matures. Build an explicit stop condition and recovery owner.

Staging is not automatically safe. A small cohort can still be inappropriate if it concentrates vulnerable people or if the harm cannot be repaired.

The aim is not to hide a risky decision inside an experiment. It is to make the commitment proportionate to what the team knows and can responsibly learn.

A hypothetical decision with asymmetric risk

Consider a fictional B2B finance product that proposes automatically matching invoices to payments. The scenario is invented to demonstrate the method.

Correct matches would remove repetitive review work. A false match could hide an unpaid invoice and contaminate an audit trail. The benefit is frequent and recoverable; the failure is rarer but more serious.

A team could wait until the model handles every account pattern. That prolongs the existing manual burden and may still not reveal behaviour in live operations.

It could also enable automatic matching for everyone. That creates broad exposure before the team understands false matches in unusual account structures.

The team chooses a smaller commitment. The system suggests matches only for an eligible cohort, shows the supporting records, and requires confirmation before changing financial state.

It records rejected suggestions, prevents duplicate application, and gives operations a queue for ambiguous cases. Automatic action remains outside the decision.

The review trigger is not “after one month.” It is evidence: the observed failure modes, recovery time, and user ability to recognise a wrong suggestion.

The team has not eliminated uncertainty. It has separated a useful learning step from the higher-consequence commitment.

Record the decision and its review trigger

A short decision record makes the accepted risk inspectable later.

Capture the outcome, options considered, material assumptions, evidence, consequence, exposure, recovery path, owner, decision date, and deadline.

Then state what will reopen the choice. A guardrail breach, new segment, changed regulation, different failure pattern, or irreversible next stage is more useful than a routine review with no decision attached.

Record dissent when it concerns a material risk. Do not rewrite the rationale after seeing the result.

A good outcome does not prove the original decision was sound, just as a bad outcome does not prove it was reckless. Review the quality of the reasoning and what the result teaches.

A practical decision-risk review

Before committing, ask:

• What outcome, commitment, and decision deadline are we discussing?
• What becomes worse if the central belief is wrong?
• Who carries the consequence, including people outside the buying organisation?
• How broad, long, and concentrated is the exposure?
• Can we detect, stop, and repair the effect in product and customer terms?
• What happens if we wait, and when does that cost become material?
• Which uncertainty could actually change the preferred option?
• Does the evidence method match that uncertainty?
• What is the stopping rule if evidence remains mixed?
• Can we reduce scope, exposure, permanence, or automation?
• Which boundary requires specialist authority rather than product judgement alone?
• What signal will reopen the decision?

Good product judgement is not a performance of certainty. It is choosing a commitment that the evidence and recovery path can support.

When the cost of being wrong is high, raise the standard or reduce the exposure. When the cost of waiting is higher, act with limits that make learning possible without pretending the risk has disappeared.

Sources

• The Green Book 2026 — HM Treasury
• Plan user research for your service — GOV.UK Service Manual

Read next

How to Approach Assumption Mapping Systematically helps identify the assumption that controls the next commitment and choose the smallest useful evidence step.