Skip to content
Product/measuring-outcomes

Did it work? Go check.

"Measure whether a shipped feature met its goals, and decide what to do if it failed."

Why this exists

Shipping is not the finish line. A feature that moves no metrics is a sunk cost, not a success. Without measurement, teams build confidence from activity like shipping twelve features instead of evidence like improving retention by 8%. Activity is a story. Evidence is product work.

Operational Flow

1

Define the metric before coding: Every spec must state the expected metric, the current baseline from PostHog or Amplitude, and the target. If you cannot name the metric, the feature is not ready.

2

Instrument events in the main PR: Event tracking code must go in the same PR as the feature code. Do not create follow-up tickets or promise to add analytics later.

3

Check outcomes at 30 days: Thirty days after a feature reaches 100% rollout, the product manager must review the metric and write a one-paragraph summary in the Linear issue.

4

Treat negative results as data: If the feature did not move the metric, state it clearly in the retro. Spending six weeks on a build that produces no measurable benefit is critical information for future scoping.

5

Separate leading from lagging indicators: Improving annual retention takes months to show up. Identify a leading indicator, such as weekly active usage or onboarding completion time, to check at 30 days.

6

Avoid vanity metrics: Page views and total signups are not product metrics. If a metric goes up when the system is broken, it is not a health indicator.

What good looks like

  • A Linear issue includes an Outcome section added thirty days post-launch with real numbers and a next step.
  • A retro shows that we moved the activation rate from 41% to 58% using a new empty state, supported by a PostHog chart.
  • A spec is rejected in review because it lacks a metric, and the author updates it with one the next day.

What NOT to do

  • Don't ship a feature and move on without a 30-day check. Monitoring it is not a plan.
  • Don't measure a feature by whether users say they like it. User satisfaction is not the same as behavioral change.
  • Don't change the target metric after shipping because the original did not move. Stick to your chosen metric.
  • Don't let negative results disappear. Documenting them is as useful as celebrating wins.

Teams skip post-launch measurement out of fear, not laziness. If you do not check, you cannot be wrong. The feature remains in a state of quantum success, which feels comfortable. It is also how you end up in a product review six months later unable to explain why retention remains flat despite constant shipping. The habit of checking is uncomfortable at first, but a negative result in thirty days is much more useful than a surprise at the annual review.

The mechanics are simple: name the metric in the spec, track the event in the PR, set a 30-day reminder, and write one paragraph. The paragraph does not have to be good news, but it must be honest. If the feature worked, write why. If it failed, explain what you would change. Both outcomes build product knowledge, making the next spec better and the next prioritization call sharper.