Skip to content
Back to the journal

Workbook

148

Measurement & Growth

10 min read

148 / 148

Product Metric Trees: Make the Causal Assumptions Visible

Construct, test, govern, and prune a product metric tree with explicit equations, counter-metrics, shared drivers, and accountable owners.

Topics Analytics Metrics Growth

Share this workbook

A metric tree is usually presented as a tidy answer: one outcome at the top, team metrics underneath, and arrows connecting the layers.

The arrows are the dangerous part.

Each arrow claims that changing one measure should move another. Some arrows are arithmetic identities. Some are supported associations. Others are beliefs the organisation has never tested.

A useful metric tree exposes those differences. It helps a product leader decide which relationship to investigate, which team can act, and which harm must remain visible.

It is not a reporting hierarchy. It is a governed theory of how product behaviour contributes to an outcome.

Give the tree one decision

Do not begin with every metric the organisation reports. Begin with a recurring decision.

Examples include:

  • where to invest to improve retained customer value;
  • which part of activation is failing;
  • whether a local conversion gain damages later use;
  • which team owns the next diagnosis;
  • whether a product bet moved the mechanism it targeted.

Write the decision above the tree. If a branch cannot change that decision or protect it from harm, it may belong in another view.

Product Metrics Practice covers metric contracts, provenance, and reconstructable reporting. This article addresses the relationships between those contracted measures.

Decide whether one root is honest

A single top metric is attractive because it creates focus. It can also collapse incompatible obligations.

A marketplace may need healthy supply and successful demand. A safety-sensitive service may need customer value and a non-negotiable harm boundary. A public service may have an outcome without a revenue analogue.

Use one root only when it represents the decision without hiding a separate condition that can invalidate the result.

Otherwise use a paired root:

  1. the customer or user outcome;
  2. the economic, operational, or public-value condition that keeps the outcome supportable.

Add guardrails for harms that must not be traded away. A guardrail is not another weighted contribution to the root. Crossing it can stop or reverse a decision.

The HEART paper from Google Research maps goals to signals and then metrics. It is useful because the goal precedes the available data.

Its examples come from large-scale web products inside one company. It does not prove that HEART dimensions or a single top measure fit every product.

Write a metric contract before drawing an edge

Every node needs a stable meaning.

Record:

Name
Decision served
Entity and eligible population
Event or state definition
Numerator and denominator
Time window and cohort rule
Source and transformation
Known exclusions
Owner
Freshness expectation
Review and change history

“Activation rate” is not a contract. It leaves the qualifying behaviour, eligible population, time window, assisted users, duplicate identities, and late events unresolved.

Definitions must reconcile across layers. If the parent counts paying workspaces and the child counts registered people, the edge needs an explicit conversion or it is merely visual proximity.

Use Vanity Metrics to challenge nodes that are easy to move but weakly connected to a decision.

Label the relationship, not just the metric

Every parent-child edge should have one of four labels.

Identity

An identity is true by definition.

retained recurring contribution
= retained paying workspaces x average contribution per workspace

Changing either child changes the parent arithmetically, assuming consistent populations and periods.

Decomposition

A decomposition splits a population or process.

activated workspaces
= eligible new workspaces x activation rate

This is arithmetic, but the activation definition can still be poor.

Supported relationship

Historical analysis or experiments support the edge within a stated context.

For example, workspaces completing a shared workflow may retain more often after adjustment for known differences. That supports investigation, not an unrestricted causal claim.

Hypothesis

The team expects a relationship but lacks adequate evidence. The edge should name the test that can strengthen, narrow, or remove it.

Do not allow arrow style or colour to carry this information alone. Put the label and evidence reference in the metric register.

Construct the tree from the root down

Use a disciplined sequence.

  1. Define the decision and root outcome.
  2. Write the exact arithmetic decomposition where one exists.
  3. Add behavioural drivers only when a plausible mechanism connects them.
  4. Add diagnostic measures that can distinguish competing explanations.
  5. Add guardrails and counter-metrics beside the affected branch.
  6. Name the evidence status and owner for every edge.
  7. Stop when another layer would describe implementation activity rather than decision-relevant behaviour.

Do not force one child to have one parent. Reliability can influence activation, repeated use, support demand, and renewal. Duplicating the node without a shared definition creates conflicting truths.

Keep one metric contract and show the shared dependency in several branches. Name the team that maintains the definition and the teams that use it.

A complete fictional metric tree

The following example is hypothetical. It demonstrates construction and claims no measured product result.

A B2B collaboration product wants to decide where to improve eight-week retained workspace value.

It uses paired roots:

CUSTOMER ROOT
weekly successful shared workflows in retained workspaces

ECONOMIC ROOT
retained recurring contribution
= retained paying workspaces x average contribution per workspace

The retained workspace branch is:

retained paying workspaces next period
= paying workspaces at period start
+ retained new workspaces
- churned workspaces

retained new workspaces
= eligible new workspaces
x activation within 7 days
x week-8 retention among activated workspaces

Activation is contracted as a workspace inviting at least two colleagues and completing one shared workflow within seven days. Assisted migrations are reported separately.

Its behavioural branch is:

activation within 7 days
├─ administrator completes setup
├─ at least two valid invitations are delivered
├─ at least one invitee accepts
└─ the group completes a shared workflow

These are not interchangeable drivers. Setup completion is necessary in the current flow. Invitation acceptance is an observed association. Shared workflow completion is a retention hypothesis awaiting a planned test.

The customer root adds quality:

successful shared workflows
= attempted shared workflows
x completion rate
x result accepted without correction

The tree keeps shared dependencies visible:

  • permission reliability affects setup, workflow completion, and support demand;
  • notification delivery affects invitation acceptance and repeated coordination;
  • data freshness affects whether a completed workflow is trusted.

The branch carries counter-metrics:

Proposed improvementPrimary metricCounter-metric or guardrail
Send more invitation promptsaccepted invitationsnotification complaints and opt-outs
Reduce setup stepsactivationassisted recovery and permission errors
Preselect workflow defaultscompletioncorrections after completion
Encourage broader workspace useshared workflowsadministrator time and support demand

An access-control incident is a release guardrail, not a negative weight that can be offset by more activations.

Test the tree before using it to allocate work

Validate each layer in a different way.

Reconcile identities

Recalculate parent values from their children. Check population, time window, currency, late-arriving data, and rounding.

If the identity does not reconcile, the tree is not ready for causal discussion.

Examine supported relationships

Compare cohorts and segments. Check whether the association survives obvious differences in channel, account size, tenure, or assisted onboarding.

This does not eliminate confounding. It tells the team whether the relationship deserves a stronger test.

Test hypotheses proportionately

Use an experiment when assignment is feasible and ethical. Otherwise use a staged release, interrupted time series, matched comparison, qualitative mechanism study, or another design appropriate to the claim.

The UK government’s Magenta Book treats a theory of change as a causal account to examine, including assumptions, context, and unintended outcomes.

It is public-policy evaluation guidance, not a product analytics standard. Its useful warning is that monitoring an expected movement does not, by itself, establish why it happened.

Run a sensitivity check

If several children are combined into an index, vary the weights and normalisation. Does the branch still point to the same decision?

The OECD and European Commission handbook on composite indicators requires transparent construction and sensitivity analysis because weighting and aggregation can change rankings.

Its subject is country-level composite indicators. A product metric tree is different, but the risk of hiding judgement inside weights is directly relevant.

Assign ownership without pretending one team controls the outcome

Separate four roles:

  • Definition owner: maintains the contract and resolves interpretation.
  • Instrumentation owner: protects collection and transformation quality.
  • Decision owner: chooses action when the metric changes.
  • Contributing teams: can influence one or more drivers.

A team may influence invitation acceptance without controlling acquisition quality or eight-week retention. Hold it accountable for the decision and mechanism inside its boundary.

Do not cascade a company target into identical team targets. That rewards local movement without explaining contribution.

Shared drivers need a review path. When reliability affects several outcomes, product and engineering should agree which customer consequence governs the current investment.

Govern changes as product decisions

A metric tree needs version control.

Record changes to definitions, sources, edges, weights, owners, and guardrails. Preserve the previous version long enough to interpret historical movement.

Review the tree when:

  • strategy or customer unit changes;
  • a new business model alters the economic root;
  • instrumentation breaks or identity logic changes;
  • an edge repeatedly fails to predict its parent;
  • teams optimise a child while the parent stalls;
  • a guardrail reveals an unmodelled cost.

Decision records for product evidence explains how to document the choice and define when new evidence should reopen it.

Prune branches that no longer earn attention

A metric remains in the tree only if it explains a material parent, distinguishes a diagnosis, protects a guardrail, or supports a recurring decision.

Prune it when the relationship has failed, the product no longer uses the behaviour, the data cannot be trusted, or no owner will act on it.

Archive rather than erase. A discarded edge can explain why an older roadmap choice looked sensible at the time.

The final test is not whether a new colleague can recite the tree. It is whether the team can identify which arrow is uncertain, which evidence could change it, and who must decide when it breaks.

Sources

Related books

If you want to go further on this topic, these are two good places to start.

01

communication

The Pyramid Principle

by Barbara Minto

The foundational framework for structured communication, teaching how to present ideas in a clear, logical hierarchy that makes complex information accessible.

Some outbound links are affiliate links and support independent bookstores.