Product Outcome

A product outcome is a measured change in how the product performs after a bet ships. It shows whether more people are actually using what was built, and keep using it.

For example, before the redesign, 12% of weekly active new cardholders visited the rewards tab. After the points progress bar shipped, 31% did. That rise is the product outcome. It sits in the middle of the chain, between the change people felt and the change the business sees.

Why it matters

A better experience only matters if people use it. A team can make a feature easier to understand and still see nobody open it. Without product outcomes, there's no way to tell whether a design win turned into real behavior at scale.

Product outcomes bridge that gap. They're measured by product KPIs such as adoption, feature usage and retention in the product, and they come from real usage data rather than a test session. They explain how a user outcome becomes a business outcome, and they show early when that hand-off breaks.

For each bet, name the product KPI that should move, record the number before and after, and check that it links back to a user outcome and forward to a business outcome.

How we write one

Every product outcome is written as one sentence that names the product KPI, where it started and where it ended up:

A product KPI moved from a starting value to a new value.

Rewards tab visits moved from 12% to 31% of weekly actives.

Writing it this way keeps the focus on real behavior in the product. The group is usually built into the KPI itself, such as "of weekly actives", so it doesn't need its own clause.

How it differs from similar ideas

  • A user outcome is a change in how the experience works for people, such as task success. A product outcome is a change in what people do in the product, such as how many open a feature. The user outcome usually explains the product outcome.

  • A business outcome is a change in revenue, spend, churn or cost. A product outcome is the usage that should lead to it, one step earlier in the chain.

  • A benchmark is a single comparison point, such as a 12% baseline. A product outcome is the change from that baseline after a bet.

  • A UX metric tagged as a product KPI is the measure itself. A product outcome is a recorded change in that measure, tied to a bet.

  • A result judges the whole system against a business goal. A product outcome belongs to a single bet.

A good product outcome

  • Uses a product KPI, such as adoption, feature usage or retention in the product

  • Has a real before and after value

  • Comes from live usage data, not a test session

  • Covers a stated period

  • Comes from one bet

  • Links back to a user outcome and forward to a business outcome

What happens to it

  1. Expected. A bet names the product KPI it should move, such as adoption of a feature.

  2. Baselined. The starting value is recorded from analytics before the change ships, often as a benchmark.

  3. Measured. After launch, the same KPI is pulled again for the same group and period length.

  4. Mapped. The outcome is linked back to the user outcome that explains it and forward to the business outcome it should drive.

  5. Returned. The outcome goes back to Define and Collecting, where it informs new situations and assumptions.


For Ray

This part tells Ray how to find product outcomes in an input, such as an analytics export, a product review deck, a launch retro, a growth team update or a feature dashboard, and how to store them.

How to find one

  1. Mark before and after usage claims. Read the input and mark every statement that compares product usage before and after a change.

  2. Confirm it's a product KPI. Keep it only if the measure describes product behavior, such as adoption, activation, feature usage, return visits or retention in the product. Route experience measures to User Outcome, and money or churn to Business Outcome.

  3. Find the two values. Pull out the starting value and the new value, with the base they're measured against, such as weekly actives.

  4. Find the period. Identify the time window for each value. Make sure the two windows are comparable in length and season.

  5. Find the bet. Identify the shipped change that produced it. If no bet is named, leave it blank and flag it.

  6. Find its neighbors. Look for a user outcome that explains the change and a business outcome it should feed.

  7. Write the sentence. Use the format: [Product KPI] moved from [x] to [y].

  8. Fill the record, score it, link it and store it using the rules below.

Cues to look for

  • Usage language: "adoption", "active users", "opened", "visited", "used the feature", "came back"

  • Rates with a base: "% of weekly actives", "per user per month", "30-day retention"

  • Launch comparisons: "since launch", "post-release", "vs. the month before"

  • Analytics sources: dashboards, event exports, funnel reports, growth updates

What doesn't count

<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 50px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>If the statement...</p></th><th colspan="1" rowspan="1"><p>Store it as</p></th></tr><tr><td colspan="1" rowspan="1"><p>Reports a change in task success, ease of use or comprehension</p></td><td colspan="1" rowspan="1"><p>User Outcome</p></td></tr><tr><td colspan="1" rowspan="1"><p>Reports a change in revenue, spend, churn or cost</p></td><td colspan="1" rowspan="1"><p>Business Outcome</p></td></tr><tr><td colspan="1" rowspan="1"><p>Gives a single usage number with no before value</p></td><td colspan="1" rowspan="1"><p>Benchmark or UX Metric</p></td></tr><tr><td colspan="1" rowspan="1"><p>Reports usage from a test session or prototype</p></td><td colspan="1" rowspan="1"><p>UX Metric in Findings</p></td></tr><tr><td colspan="1" rowspan="1"><p>Describes what users are doing without a measured change</p></td><td colspan="1" rowspan="1"><p>Situation</p></td></tr></tbody></table>

The record

<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 75px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>Field</p></th><th colspan="1" rowspan="1"><p>What to fill in</p></th><th colspan="1" rowspan="1"><p>Values</p></th></tr><tr><td colspan="1" rowspan="1"><p>Statement</p></td><td colspan="1" rowspan="1"><p>The one-sentence outcome</p></td><td colspan="1" rowspan="1"><p>[Product KPI] moved from [x] to [y].</p></td></tr><tr><td colspan="1" rowspan="1"><p>Product KPI</p></td><td colspan="1" rowspan="1"><p>The usage measure that moved</p></td><td colspan="1" rowspan="1"><p>Link to UX Metric tagged Product KPI</p></td></tr><tr><td colspan="1" rowspan="1"><p>Before</p></td><td colspan="1" rowspan="1"><p>The starting value, with its base</p></td><td colspan="1" rowspan="1"><p>Number with unit, or blank</p></td></tr><tr><td colspan="1" rowspan="1"><p>After</p></td><td colspan="1" rowspan="1"><p>The new value, with its base</p></td><td colspan="1" rowspan="1"><p>Number with unit</p></td></tr><tr><td colspan="1" rowspan="1"><p>Group</p></td><td colspan="1" rowspan="1"><p>Whose usage was measured</p></td><td colspan="1" rowspan="1"><p>Link to Audience</p></td></tr><tr><td colspan="1" rowspan="1"><p>Period</p></td><td colspan="1" rowspan="1"><p>The before and after windows</p></td><td colspan="1" rowspan="1"><p>Dates or periods</p></td></tr><tr><td colspan="1" rowspan="1"><p>Data source</p></td><td colspan="1" rowspan="1"><p>Where the numbers came from</p></td><td colspan="1" rowspan="1"><p>Short text, such as product analytics</p></td></tr><tr><td colspan="1" rowspan="1"><p>Bet</p></td><td colspan="1" rowspan="1"><p>The shipped change that produced it</p></td><td colspan="1" rowspan="1"><p>Link to Bet, or blank</p></td></tr><tr><td colspan="1" rowspan="1"><p>User outcome</p></td><td colspan="1" rowspan="1"><p>The experience change that explains it</p></td><td colspan="1" rowspan="1"><p>Link to User Outcome, or blank</p></td></tr><tr><td colspan="1" rowspan="1"><p>Source</p></td><td colspan="1" rowspan="1"><p>The input and exact passage, slide or chart</p></td><td colspan="1" rowspan="1"><p>Link to Input</p></td></tr><tr><td colspan="1" rowspan="1"><p>Confidence</p></td><td colspan="1" rowspan="1"><p>How solid the before and after are</p></td><td colspan="1" rowspan="1"><p>High, medium, low</p></td></tr></tbody></table>

Before and After must use the same KPI definition and base. If the base changed, such as from all users to weekly actives, store both but lower confidence and say why.

Scoring confidence

Confidence measures how well the change is evidenced, not how big it is.

<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 50px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>Level</p></th><th colspan="1" rowspan="1"><p>When to use it</p></th></tr><tr><td colspan="1" rowspan="1"><p>High</p></td><td colspan="1" rowspan="1"><p>Both values from the same analytics source and definition, with comparable periods and a named bet</p></td></tr><tr><td colspan="1" rowspan="1"><p>Medium</p></td><td colspan="1" rowspan="1"><p>Both values given, but the base, period or bet is unclear</p></td></tr><tr><td colspan="1" rowspan="1"><p>Low</p></td><td colspan="1" rowspan="1"><p>One value is estimated, the definition changed between periods, or the claim is secondhand</p></td></tr></tbody></table>

If any part of the record can't be filled from the source, leave it blank and lower confidence by one level.

Linking

  • Check for duplicates first. If an existing outcome covers the same KPI, group and period, add the new source to it instead of creating another.

  • Link to the bet that produced it and the benchmark that supplied the before value.

  • Link back to the user outcome that explains it, such as task success rising before tab visits did.

  • Link forward to the business outcome it should drive, such as card spend per new cardholder.

  • Note breaks in the chain. If the user outcome improved and the product outcome didn't, or the product outcome improved and the business outcome didn't, link them and flag the break.

When to store and when to ask

  • Store automatically when confidence is high.

  • Add to the review digest when confidence is medium or low, when it contradicts its bet, or when the chain to user or business outcomes is broken.

When to flag

  • A shipped bet has no product outcome measured after 60 days

  • A product outcome has no bet linked to it

  • A product outcome rose, but its linked business outcome didn't move

  • The KPI definition or base changed between before and after

  • A product outcome moved the wrong way

Worked example

Input: a product analytics export shared in the April 2027 growth review.

"Rewards tab: weekly visitors among new cardholders, as % of weekly actives. Jan avg 12%. Apr avg 31% (progress bar live since Feb 24). Same event definition."

What Ray finds:

  • "Rewards tab weekly visitors as % of weekly actives" is feature usage, a product KPI.

  • "Jan avg 12%" is the before value and "Apr avg 31%" is the after value.

  • "Progress bar live since Feb 24" ties the change to the bet.

  • "Same event definition" confirms the two values are comparable, which supports high confidence.

Stored record:

<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 50px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>Field</p></th><th colspan="1" rowspan="1"><p>Value</p></th></tr><tr><td colspan="1" rowspan="1"><p>Statement</p></td><td colspan="1" rowspan="1"><p>Rewards tab visits moved from 12% to 31% of weekly actives.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Product KPI</p></td><td colspan="1" rowspan="1"><p>Rewards tab visits, % of weekly actives (Behavioral, Lagging, Product KPI)</p></td></tr><tr><td colspan="1" rowspan="1"><p>Before</p></td><td colspan="1" rowspan="1"><p>12% of weekly actives</p></td></tr><tr><td colspan="1" rowspan="1"><p>After</p></td><td colspan="1" rowspan="1"><p>31% of weekly actives</p></td></tr><tr><td colspan="1" rowspan="1"><p>Group</p></td><td colspan="1" rowspan="1"><p>Customers: new cardholders in their first 90 days</p></td></tr><tr><td colspan="1" rowspan="1"><p>Period</p></td><td colspan="1" rowspan="1"><p>January 2027 average; April 2027 average</p></td></tr><tr><td colspan="1" rowspan="1"><p>Data source</p></td><td colspan="1" rowspan="1"><p>Product analytics</p></td></tr><tr><td colspan="1" rowspan="1"><p>Bet</p></td><td colspan="1" rowspan="1"><p>We bet the progress bar will drive adoption</p></td></tr><tr><td colspan="1" rowspan="1"><p>User outcome</p></td><td colspan="1" rowspan="1"><p>Task success moved from 42% to 78% for new cardholders</p></td></tr><tr><td colspan="1" rowspan="1"><p>Source</p></td><td colspan="1" rowspan="1"><p>Analytics export, April 2027 growth review</p></td></tr><tr><td colspan="1" rowspan="1"><p>Confidence</p></td><td colspan="1" rowspan="1"><p>High: same source and event definition, comparable periods and a named bet</p></td></tr></tbody></table>

Next step: stored automatically and linked forward to the card spend business outcome.

Edge cases

  • Vanity counts. "Rewards tab views doubled" may just reflect more users. Ask for a rate with a base before storing it, and keep confidence at medium until you have one.

  • Short windows. A one-week spike right after launch is novelty, not adoption. Note the period and add a watch item to recheck it.

  • Retention that is really churn. Retention in the product, such as return visits, is a product outcome. Customers closing their card account is churn, a business outcome.

  • Several bets at once. If marketing emails and the progress bar launched together, link the outcome to both bets and flag it so the team can decide how to split credit.

Identify where decision quality breaks down

The Glare Design Assessment helps teams spot weak validation, stakeholder friction, alignment gaps, and assumptions that scale without measurable learning—so you have a clearer starting point for improvement.

About 5 minutes · Team-based · Diagnostic snapshot you can act on

Take the Design Assessment