Define a Product Metric Before Putting It on a Dashboard

A dashboard can make an ambiguous number look authoritative. “Active users” is a common example: active in what way, over which period, and with which accounts included?

Before building the chart, write a definition that two people could independently implement and get the same result.

Start with the decision

Ask what the number will help you decide. If the question is whether new users reach a useful outcome, counting sign-ins may be an indirect measure. Identify the behavior that more closely represents the outcome.

For a hypothetical task application, creating an account and completing a first task are different events. Neither should silently stand in for the other.

Define the numerator and population

State the event or condition being counted, the eligible population, and the unit. Are you counting people, accounts, workspaces, or actions? A single person can belong to several workspaces, so these are not interchangeable.

If the metric is a rate, define both numerator and denominator. “Import completion rate” might mean completed imports divided by started imports, or users who completed an import divided by users offered the feature. Those answer different questions.

Fix the time rules

Specify the time window and timezone. Explain how delayed events are handled and whether the period is based on account creation, event occurrence, or processing time.

For a cohort measure, define when a user enters the cohort and how long they have to perform the action. Otherwise, newer cohorts can appear worse simply because they have had less time.

State exclusions

Decide how test accounts, internal users, duplicate events, and failed actions are treated. Keep the exclusions visible and justified. Removing inconvenient data without a rule can make the metric tell the story you hoped for.

Document known instrumentation gaps. A missing event should not be interpreted as a user decision until you know the event collection works.

Validate with examples

Create a few small cases and calculate the expected result by hand: a user who succeeds, one who retries, one who abandons, and one whose event arrives late. Compare the implementation against those cases.

The dashboard becomes useful only after the definition is stable enough to interpret. A short metric contract gives the team a shared meaning for the number and a way to notice when the underlying behavior or instrumentation changes.