Funnel Optimisation: Build a Decision Model, Not a Leak Chart

A funnel can improve overnight even when no user has a better experience.

An identity rule changes. Returning users are counted differently. A longer conversion window allows more late completions into the report. An eligibility check removes difficult cases from the denominator.

The chart rises, and the product has learned nothing.

A funnel is not the customer journey. It is a query that reduces selected behaviour to an ordered model. Every result depends on choices about who enters, which events count, how long the clock runs, and what the team calls success.

Funnel optimisation becomes useful when those choices are explicit and tied to a product decision. Without that discipline, teams optimise the reporting model, the easiest step, or the most visible number.

Start with the decision the funnel must support

“Improve conversion” is not yet a decision.

Name the choice that could change after the analysis. For example:

• Should the team repair account verification before adding another acquisition channel?
• Does the self-serve path work well enough for a defined small-business segment?
• Should an optional setup step move until after the first completed task?
• Is a fall in completion caused by product friction, a different entrant mix, or a measurement defect?

Then state the user outcome that matters beyond passing the final tracked event.

A submitted application, configured integration, or paid order may be an appropriate completion event.

It may still be a poor outcome if the application is unusable, the integration fails on first sync, or the order creates avoidable returns.

Google’s original HEART framework paper describes a process of moving from product goals to signals and then metrics.

It is a user-experience measurement framework from Google, not a funnel method. The relevant discipline is the order: clarify the goal before choosing the observable signal, and choose the signal before writing the metric.

Use the engagement analytics guide when the open question concerns depth or quality of participation rather than progression through a bounded path.

Write a funnel contract

Before opening an analytics tool, record the semantics of the funnel.

Population

Who is eligible to enter the analysis? Include product state, channel, role, geography, plan, device, and any exclusion that could change interpretation.

“All users” often hides several populations with different jobs and available paths.

Entry

Which observable event establishes that the person genuinely started the task? A page view, account creation, and first attempt are different denominators.

Valid outcomes

List every legitimate end state. Completion, ineligibility, deliberate deferral, assisted completion, and transfer to another channel may all be valid outcomes rather than abandonment.

Sequence

Must events occur in a fixed order? Can people skip optional steps, enter midway, go backwards, or finish through another route?

Clock

When does observation start and stop? Is the question about completion in one session, within a day, within a billing cycle, or eventually?

Identity and unit

Are you counting people, accounts, devices, sessions, applications, or attempts? How are anonymous and authenticated activity joined? What happens when several people act on one account?

Re-entry

Does a new attempt replace, continue, or coexist with an earlier one? A person may abandon an attempt without abandoning the outcome.

Decision and guardrails

Which decision will the report inform? Which downstream outcome, service burden, risk, or affected group prevents a local conversion gain from being declared a success?

Version this contract with the report. If a definition changes, mark the break rather than drawing one continuous trend through two different measures.

The denominator is part of the product claim

The GOV.UK completion-rate guidance defines a transaction using explicit start and end points.

It includes partial and failed transactions in the denominator, recognises multiple valid end pages, treats save-and-return as a matching problem, and handles people who stop after an ineligibility check differently from those who proceed.

That guidance is written for UK government transactional services. It does not define the right funnel for a commercial product.

It does show why a completion rate is inseparable from decisions about eligibility, starts, endings, and delayed returns.

Audit the denominator before discussing a “leak”:

• Did people intend to begin the task, or did the interface count passive exposure?
• Could eligible people bypass the tracked entry point?
• Are bots, tests, staff, duplicate attempts, and retries handled consistently?
• Did a product or channel change alter who now enters?
• Are legitimate endings being classified as failure?
• Are users who need assisted or offline steps disappearing from the model?

A smaller, well-defined population is more useful than an impressive number whose membership cannot be explained.

A funnel configuration is an argument about the path

Analytics software makes consequential modelling choices look like report settings.

For example, the Google Analytics Data API documentation distinguishes an open funnel, where a user may enter at any step, from a closed funnel, where the first step is required.

It also allows direct sequencing and a duration between steps.

These are documented behaviours of one analytics product. The larger point is vendor-independent: changing entry, order, or time changes who qualifies for the result.

Compare the report with real paths before simplifying them.

Map at least:

• the intended path;
• common alternative paths;
• recovery after error;
• return in another session or device;
• assisted and offline continuation;
• intentional exit because the product correctly established no action was needed.

Do not force every route into one diagram. Use separate funnels when populations, meanings, or clocks differ enough that a combined rate would conceal the decision.

Treat events as product definitions, not plumbing

An event such as setup_complete looks precise until two clients fire it at different moments.

Give every decision-critical event an owner and a specification:

• business meaning;
• unit and actor;
• trigger and required properties;
• eligible product states;
• deduplication and retry behaviour;
• identity available at capture;
• client or service that emits it;
• known exclusions and failure modes;
• version and change history.

Validate the instrumentation against the product itself. Walk the paths, inspect raw records, reconcile key totals with another operational source, and test error, retry, save, return, and cross-device cases.

Look for impossible transitions, sudden property loss, duplicate emission, delayed ingestion, and shifts that coincide with analytics releases rather than product changes.

The data-informed decision guide provides a broader decision record for evidence quality, disagreement, and reopening conditions.

Do not call an unfinished observation a failure

A conversion window creates a right edge. People who have not completed by that edge include genuine exits and people whose outcome is not yet observable.

The NIST engineering statistics handbook explains right censoring in reliability testing.

A fixed observation period ends while some units have not yet produced the event being studied.

Product behaviour is not hardware reliability, and the appropriate time-to-event method depends on the data and decision. Involve a qualified analyst when that distinction matters.

The transferable warning is simple: recent entrants have had less opportunity to complete.

Do not compare an immature cohort with a fully observed one as if the exposure time were equal.

For slow paths, report:

• completion within fixed, decision-relevant windows;
• time-to-completion distribution, not only an average;
• the share still eligible to complete when observation ends;
• cohort maturity and late-return behaviour;
• results under more than one defensible window when the decision is sensitive to it.

The approved cohort tracking guide covers eligibility, exposure, maturity, and composition in more depth.

A drop-off is an observation, not a diagnosis

“Users leave at verification” describes a boundary in the data. It does not explain the mechanism.

Several realities may create the same shape:

• a technical failure prevents continuation;
• requirements appear too late;
• the user lacks necessary authority or information;
• the product correctly screens out an ineligible case;
• the person pauses and returns later;
• another channel completes the task;
• the entrant never had the problem the funnel assumes;
• the event after verification fails to fire.

Write competing explanations before proposing a solution. Then collect evidence capable of separating them.

Use event and error records to locate affected states, service data to find assisted recovery, observation to see interaction, and interviews to reconstruct a recent episode.

Segment only on characteristics that could plausibly change the mechanism.

Avoid mining dozens of segments until one looks dramatic. Small groups produce unstable rates, and post-hoc stories are easy to invent. A segment split should lead to a testable explanation or a product decision.

Prioritise a decision opportunity, not the largest red bar

Neither the biggest percentage loss nor the largest absolute loss automatically deserves investment.

For each candidate boundary, estimate a range rather than a theatrical point score:

1. Eligible flow: how many relevant attempts reach the boundary?
2. Recoverable share: which exits plausibly come from a mechanism the product can change?
3. Downstream quality: how many recovered attempts are likely to reach the meaningful outcome?
4. Consequence: what user, operational, strategic, or economic difference would that create?
5. Cost and exposure: what delivery work, new burden, or risk accompanies the change?
6. Evidence strength: which parts are observed, inferred, or unknown?

The result is a decision range, not a forecast. It should reveal which assumption makes the opportunity look attractive.

Sometimes an early step has more volume but little intent. Sometimes a later step contains fewer people but exposes a severe, recoverable failure.

Sometimes the right action is to change acquisition, eligibility, support, or positioning rather than the interface.

Protect the outcome from local conversion gains

Removing friction is not a universal good. A step may establish informed consent, prevent an unsafe configuration, confirm authority, set expectations, or help a user discover that the product is unsuitable.

For every target metric, name counter-signals such as:

• successful use after completion;
• reversal, cancellation, return, or early churn;
• errors and recovery time;
• support and manual intervention;
• fraud, safety, privacy, or compliance exposure;
• accessibility and assisted-path outcomes;
• complaints, regret, or loss of control.

A local rate can rise by admitting more poor-fit attempts, moving effort downstream, or hiding an exit. Judge the full outcome path, not the beauty of one transition.

Experiment only after the measurement survives challenge

A before-and-after change can generate a hypothesis. It does not isolate the effect of the product change from seasonality, entrant mix, campaigns, incidents, or simultaneous releases.

Where randomised experimentation is appropriate, predefine the population, assignment unit, primary metric, guardrails, minimum practical effect, analysis plan, and decision rule.

Check exposure and data integrity before interpreting movement.

Microsoft researchers documented twelve metric-interpretation pitfalls.

The paper draws on observations from their online experiments.

That industry paper concerns experimentation at Microsoft, not every product context. Its useful limit here is that even a well-chosen metric can support a wrong conclusion when movement is interpreted carelessly.

Low-volume, high-stakes, networked, or operational changes may need another design and specialist analysis. “Not enough traffic for an A/B test” does not make an uncontrolled comparison causal.

A fictional funnel review

Consider a fictional B2B billing product. Its setup funnel shows a sharp loss between connecting an accounting system and importing the first invoice.

The team does not immediately simplify the connector. It reviews the funnel contract and raw paths.

Some accounts connect only to validate compatibility before procurement. Some imports complete overnight, outside the report’s session window.

Some customers use an assisted migration handled by implementation staff. One connector also emits duplicate start events after a retry.

The team separates evaluation, self-serve migration, and assisted migration into different paths. It fixes duplicate attempts and gives cohorts enough time to mature.

The remaining self-serve loss is concentrated among accounts whose source permissions exclude invoice details. Support records and recent-episode interviews support the same mechanism.

The open decision is now specific: whether to detect insufficient permissions before connection and offer a recoverable path. Success includes a valid first import, lower recovery burden, and no increase in over-broad permission requests.

No result is supplied because the example is invented. It shows how a red bar becomes a bounded product decision only after population, path, clock, data quality, mechanism, and downstream outcome are resolved.

The funnel decision record

Keep one inspectable record for each funnel decision:

1. Decision and accountable owner.
2. User outcome and counter-signals.
3. Population, entry, unit, identity, and exclusions.
4. Valid endpoints, sequence, re-entry, and clock.
5. Event definitions, versions, and validation evidence.
6. Cohort maturity and known missing paths.
7. Observed boundary and competing explanations.
8. Segment rationale and uncertainty.
9. Evidence gathered to distinguish mechanisms.
10. Intervention, evaluation design, decision rule, and reopening trigger.

Funnel optimisation should make a product decision more defensible.

If the team cannot explain who is in the denominator, what the path represents, why people appear to leave, and which outcome must improve, it is not ready to optimise the chart.

Sources

• Measuring completion rate, GOV.UK Service Manual
• Google Analytics Data API: Funnel reports
• Measuring the User Experience on a Large Scale, Google Research
• Censoring, NIST/SEMATECH e-Handbook of Statistical Methods
• A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online Controlled Experiments, Microsoft Research

Read next

Cohort Tracking: Preserve Eligibility, Exposure, and Time shows how to compare groups without hiding maturity, composition, or changing conditions.