Experiment proposals
An experiment proposal is the format that bridges emerging understanding and committed design work. It exists because most user needs are not yet stable when teams first encounter them, but building without any validation means committing resources to a problem that may be misunderstood. The experiment proposal frames what the team believes, what they do not yet know, and what kind of investigation would resolve the uncertainty before a direction is committed.
Unlike every other format in this stack, an experiment proposal is almost always written at Emerging or Building understanding. Writing one at Stable understanding is unusual and usually a signal that the team is proposing further refinement rather than initial validation. The format is designed for uncertainty, not for confirmation.
A good experiment proposal does not advocate for a specific outcome. It describes what evidence would confirm the need, what evidence would refute it, and what the team would do differently in each case. Both outcomes are treated as useful.
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 452px;"><colgroup><col style="min-width: 25px;"><col style="width: 427px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p><strong>Section</strong></p></td><td colspan="1" rowspan="1" colwidth="427"><p><strong>Purpose</strong></p></td></tr><tr><td colspan="1" rowspan="1"><p>The Dashboard Scenario</p></td><td colspan="1" rowspan="1" colwidth="427"><p>The context and source output used in examples throughout this page</p></td></tr><tr><td colspan="1" rowspan="1"><p>When to Use Experiment</p></td><td colspan="1" rowspan="1" colwidth="427"><p>Situations where this format fits and where it does not</p></td></tr><tr><td colspan="1" rowspan="1"><p>What an Experiment Proposal Contains</p></td><td colspan="1" rowspan="1" colwidth="427"><p>The structural logic and the purpose of each element</p></td></tr><tr><td colspan="1" rowspan="1"><p>Format Rules</p></td><td colspan="1" rowspan="1" colwidth="427"><p>Structural elements and guidance for each field</p></td></tr><tr><td colspan="1" rowspan="1"><p>Understanding Strength</p></td><td colspan="1" rowspan="1" colwidth="427"><p>How Emerging and Building needs differ in experiment form</p></td></tr><tr><td colspan="1" rowspan="1"><p>Examples</p></td><td colspan="1" rowspan="1" colwidth="427"><p>Two worked examples at different understanding states</p></td></tr><tr><td colspan="1" rowspan="1"><p>Near-Miss</p></td><td colspan="1" rowspan="1" colwidth="427"><p>The most common way experiment proposals go wrong</p></td></tr><tr><td colspan="1" rowspan="1"><p>Agent Operations</p></td><td colspan="1" rowspan="1" colwidth="427"><p>How AI Skills produce experiment outputs</p></td></tr></tbody></table>
The Dashboard Scenario
A fintech product team is reviewing behavior on their mobile dashboard. Users are opening the app frequently and viewing their financial scores, but very few are clicking recommendations or taking any downstream action. Support tickets consistently ask what specific scores mean. Two early interviews describe the experience as interesting but not clear enough to act on.
The team has formed an early view: users may need to understand what their Planning score means before they can evaluate what to do next. This is an Insightful need at the Personal layer. It is Emerging. The evidence is present but has not yet been confirmed across multiple sources and the team does not yet know whether the root cause is comprehension, findability, or motivation.
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 478px;"><colgroup><col style="min-width: 25px;"><col style="width: 453px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p><strong>Field</strong></p></td><td colspan="1" rowspan="1" colwidth="453"><p><strong>Dashboard Scenario Content</strong></p></td></tr><tr><td colspan="1" rowspan="1"><p>User Need</p></td><td colspan="1" rowspan="1" colwidth="453"><p>Insightful</p></td></tr><tr><td colspan="1" rowspan="1"><p>Category</p></td><td colspan="1" rowspan="1" colwidth="453"><p>Personal</p></td></tr><tr><td colspan="1" rowspan="1"><p>Summary</p></td><td colspan="1" rowspan="1" colwidth="453"><p>Users may need to understand what their Planning score means in their own situation before they can evaluate or act on recommendations.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Evidence (Emerging)</p></td><td colspan="1" rowspan="1" colwidth="453"><p>Analytics: repeated short sessions with near-zero recommendation engagement. Support tickets: questions asking what the Planning score means. Two interviews: users describing scores as interesting but unclear.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Evidence (Building)</p></td><td colspan="1" rowspan="1" colwidth="453"><p>All of the above, plus: four additional interviews reinforcing the same pattern. Users consistently dwell on the score without navigating forward. Support ticket volume increasing.</p></td></tr><tr><td colspan="1" rowspan="1"><p>What is not yet known</p></td><td colspan="1" rowspan="1" colwidth="453"><p>Whether the gap is comprehension (users cannot interpret the score), findability (users cannot locate recommendations), or motivation (users understand but choose not to act).</p></td></tr><tr><td colspan="1" rowspan="1"><p>UX Metrics (directional)</p></td><td colspan="1" rowspan="1" colwidth="453"><p>Comprehension, Recommendation Engagement, Completion Rate</p></td></tr></tbody></table>
When to Use Experiment
Experiment format works when the team has enough evidence to believe a need is real but not enough to commit to a design direction. It is the format for reducing uncertainty before committing resources, not for confirming what the team already believes.
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 337px;"><colgroup><col style="min-width: 25px;"><col style="width: 312px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p><strong>Use experiment format when...</strong></p></td><td colspan="1" rowspan="1" colwidth="312"><p><strong>Consider a different format when...</strong></p></td></tr><tr><td colspan="1" rowspan="1"><p>The understanding is Emerging or Building</p></td><td colspan="1" rowspan="1" colwidth="312"><p>The understanding is Stable and the team is ready to commit</p></td></tr><tr><td colspan="1" rowspan="1"><p>The team needs to distinguish between competing root causes</p></td><td colspan="1" rowspan="1" colwidth="312"><p>The root cause is already clear from existing evidence</p></td></tr><tr><td colspan="1" rowspan="1"><p>A stakeholder or research committee needs to approve investigation time</p></td><td colspan="1" rowspan="1" colwidth="312"><p>The team has autonomy to investigate without formal approval</p></td></tr><tr><td colspan="1" rowspan="1"><p>The next design decision depends on which explanation is correct</p></td><td colspan="1" rowspan="1" colwidth="312"><p>The design direction is the same regardless of which explanation is correct</p></td></tr><tr><td colspan="1" rowspan="1"><p>The team wants to establish UX metric baselines before designing</p></td><td colspan="1" rowspan="1" colwidth="312"><p>Baselines are already established and targets are defined</p></td></tr></tbody></table>
Experiment versus Brief
A brief asks for approval of a direction. An experiment proposal asks for approval of an investigation. If the team knows what they want to build and needs stakeholder support to build it, Brief is the right format. If the team does not yet know what to build and needs support to figure that out, Experiment is the right format.
In practice the two formats often appear together: the experiment proposal is attached to or follows a brief that explains why validation is needed before committing to a direction.
What an Experiment Proposal Contains
Each element of an experiment proposal exists to reduce a specific type of uncertainty. Understanding why each field is there helps teams write them more precisely.
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 506px;"><colgroup><col style="min-width: 25px;"><col style="width: 213px;"><col style="width: 268px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p><strong>Element</strong></p></td><td colspan="1" rowspan="1" colwidth="213"><p><strong>What uncertainty it reduces</strong></p></td><td colspan="1" rowspan="1" colwidth="268"><p><strong>What happens if it is missing</strong></p></td></tr><tr><td colspan="1" rowspan="1"><p>What we believe</p></td><td colspan="1" rowspan="1" colwidth="213"><p>Surfaces the assumption being tested. Makes the team's starting position visible so it can be examined.</p></td><td colspan="1" rowspan="1" colwidth="268"><p>The team does not know what they are testing. Findings cannot be interpreted as confirmation or refutation.</p></td></tr><tr><td colspan="1" rowspan="1"><p>What we do not yet know</p></td><td colspan="1" rowspan="1" colwidth="213"><p>Names the specific gap between current evidence and the understanding needed to commit. Forces the team to be honest about the limits of what they have.</p></td><td colspan="1" rowspan="1" colwidth="268"><p>The experiment may answer the wrong question or produce findings that do not resolve the uncertainty.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Competing explanations</p></td><td colspan="1" rowspan="1" colwidth="213"><p>Distinguishes between causes that would produce different design responses. Ensures the experiment is designed to tell them apart.</p></td><td colspan="1" rowspan="1" colwidth="268"><p>The team may validate one explanation without ruling out another, leaving the root cause ambiguous.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Proposed approach</p></td><td colspan="1" rowspan="1" colwidth="213"><p>Specifies the method, participants, and effort required so stakeholders can make an informed resource decision.</p></td><td colspan="1" rowspan="1" colwidth="268"><p>Stakeholders cannot evaluate whether the investment is proportionate to the uncertainty.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Success criteria</p></td><td colspan="1" rowspan="1" colwidth="213"><p>Names what evidence would confirm the need, refute it, or redirect the team toward a different explanation. Both outcomes must be named.</p></td><td colspan="1" rowspan="1" colwidth="268"><p>The team may interpret ambiguous findings as confirmation. The experiment produces results but not resolution.</p></td></tr><tr><td colspan="1" rowspan="1"><p>What happens next</p></td><td colspan="1" rowspan="1" colwidth="213"><p>Connects the experiment outcome to a design or planning decision. Shows that the investigation has a purpose beyond knowledge generation.</p></td><td colspan="1" rowspan="1" colwidth="268"><p>Findings are produced but not acted on. The team cannot explain why the experiment was worth running.</p></td></tr></tbody></table>
Format Rules
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 477px;"><colgroup><col style="min-width: 25px;"><col style="width: 452px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p><strong>Element</strong></p></td><td colspan="1" rowspan="1" colwidth="452"><p><strong>Rule</strong></p></td></tr><tr><td colspan="1" rowspan="1"><p>User need and understanding</p></td><td colspan="1" rowspan="1" colwidth="452"><p>Need type, category, and understanding state at the top. Required.</p></td></tr><tr><td colspan="1" rowspan="1"><p>What we believe</p></td><td colspan="1" rowspan="1" colwidth="452"><p>One to two sentences stating the team's current view of the need. Written as a belief, not a fact. 'We believe users need...' not 'Users need...'</p></td></tr><tr><td colspan="1" rowspan="1"><p>Evidence available</p></td><td colspan="1" rowspan="1" colwidth="452"><p>The signals currently available, organized by type. This is what the team is working from. Honest about gaps.</p></td></tr><tr><td colspan="1" rowspan="1"><p>What we do not yet know</p></td><td colspan="1" rowspan="1" colwidth="452"><p>The specific question the experiment is trying to answer. One to three questions, each resolvable through the proposed approach.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Competing explanations</p></td><td colspan="1" rowspan="1" colwidth="452"><p>Two to three possible causes that would each point toward a different design response. Named specifically. Not exhaustive, but covering the most plausible alternatives.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Proposed approach</p></td><td colspan="1" rowspan="1" colwidth="452"><p>Method, participants, and estimated effort. Specific enough that a stakeholder can evaluate the resource request. Not a research design document.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Success criteria</p></td><td colspan="1" rowspan="1" colwidth="452"><p>Required. Both outcomes named: what evidence would confirm the need and what would refute it or redirect the team. Each outcome includes what the team would do next.</p></td></tr><tr><td colspan="1" rowspan="1"><p>UX metrics this study will generate</p></td><td colspan="1" rowspan="1" colwidth="452"><p>Named specifically. These become the baselines that later design work is measured against.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Estimated effort</p></td><td colspan="1" rowspan="1" colwidth="452"><p>Researcher time and calendar time. Specific enough to evaluate against the uncertainty it is resolving.</p></td></tr></tbody></table>
Understanding Strength in Experiment
Experiment format is almost always used at Emerging or Building understanding. The differences between these two states show up most clearly in the competing explanations field and the proposed approach.
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 535px;"><colgroup><col style="min-width: 25px;"><col style="width: 159px;"><col style="width: 170px;"><col style="width: 181px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p><strong>State</strong></p></td><td colspan="1" rowspan="1" colwidth="159"><p><strong>What we believe wording</strong></p></td><td colspan="1" rowspan="1" colwidth="170"><p><strong>Competing explanations</strong></p></td><td colspan="1" rowspan="1" colwidth="181"><p><strong>Proposed approach</strong></p></td></tr><tr><td colspan="1" rowspan="1"><p>Emerging</p></td><td colspan="1" rowspan="1" colwidth="159"><p>'We believe this may be...' or 'Early signals suggest...' The belief is tentative.</p></td><td colspan="1" rowspan="1" colwidth="170"><p>Three or more explanations are plausible. The team cannot yet rule any out.</p></td><td colspan="1" rowspan="1" colwidth="181"><p>Exploratory: interviews or observation sessions focused on understanding the situation before testing hypotheses.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Building</p></td><td colspan="1" rowspan="1" colwidth="159"><p>'We believe...' The belief is stronger. Evidence from multiple sources points in the same direction.</p></td><td colspan="1" rowspan="1" colwidth="170"><p>Two explanations remain plausible. One is more likely but not yet confirmed.</p></td><td colspan="1" rowspan="1" colwidth="181"><p>More targeted: concept tests, comprehension measurement, or structured usability sessions designed to distinguish between the two remaining explanations.</p></td></tr><tr><td colspan="1" rowspan="1"><p>Stable</p></td><td colspan="1" rowspan="1" colwidth="159"><p>Rarely used for validation experiments. More appropriate for refinement studies or secondary questions after the primary need is confirmed.</p></td><td colspan="1" rowspan="1" colwidth="170"><p>One explanation is confirmed. The experiment may be testing a specific design response rather than the need itself.</p></td><td colspan="1" rowspan="1" colwidth="181"><p>Highly targeted: A/B testing, benchmark measurement, or longitudinal tracking of a specific metric.</p></td></tr></tbody></table>
Examples
The first example shows an experiment proposal at Emerging understanding, where the team has two interviews and an analytics pattern and needs to distinguish between three possible explanations before committing to any design work. The second shows a proposal at Building understanding, where the team has narrowed to two explanations and is proposing a more targeted study to distinguish between them.
Example 1: Emerging need
The team has two interviews and an analytics pattern. They are proposing five to eight interviews to understand whether the gap is comprehension, findability, or motivation before any design work begins. A research committee needs to allocate researcher time.
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 522px;"><colgroup><col style="min-width: 25px;"><col style="width: 132px;"><col style="width: 122px;"><col style="width: 126px;"><col style="width: 117px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p>Format: Experiment Proposal</p></td><td colspan="1" rowspan="1" colwidth="132"><p>Understanding: Emerging</p></td><td colspan="1" rowspan="1" colwidth="122"><p>Purpose: Validate</p></td><td colspan="1" rowspan="1" colwidth="126"><p>Tone: Exploratory</p></td><td colspan="1" rowspan="1" colwidth="117"><p>~258 words</p></td></tr></tbody></table>
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 25px;"><colgroup><col style="min-width: 25px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p>User Need: Insightful (Personal) | Understanding: Emerging</p><p> </p><p>What we believe</p><p>We believe users may need to understand what their Planning score means in their specific situation before they can evaluate what to do next. Early signals suggest the dashboard is surfacing score data that users find interesting but cannot connect to a next step.</p><p> </p><p>Evidence available</p><p>Behavioral: Users open the dashboard and dwell on the Planning score without navigating to recommendations. Recommendation engagement is near zero.</p><p>Attitudinal: Two interviews describing the score as interesting but unclear. Support tickets asking what the Planning score means.</p><p>Measurable: None yet established.</p><p> </p><p>What we do not yet know</p><p>1. Whether users cannot interpret the score (comprehension), cannot find the recommendations (findability), or understand the score but choose not to act (motivation).</p><p>2. Whether the pattern is specific to the Planning score or present across all score types.</p><p>3. Whether the gap is stronger for new users or returning users.</p><p> </p><p>Competing explanations</p><p>Comprehension: Users cannot interpret what the score number means in their own context. The intervention would be about meaning-making before recommendations appear.</p><p>Findability: Users understand the score but cannot locate or recognize the recommendations as relevant to it. The intervention would be about navigation or visual hierarchy.</p><p>Motivation: Users understand the score and can find recommendations but do not feel motivated to act. The intervention would be about value proposition or urgency.</p><p> </p><p>Proposed approach</p><p>Five to eight moderated interviews with active dashboard users who have viewed the Planning score at least three times in the last 30 days and have not clicked a recommendation. Sessions of 45 minutes. Focus: ask users to explain what their Planning score means, then observe how they navigate toward or away from recommendations.</p><p> </p><p>Success criteria</p><p>Confirmed: Users consistently cannot explain what their Planning score means in their own words. Understanding strengthens to Building. Design exploration focuses on comprehension before recommendations.</p><p>Refuted (findability): Users explain the score but cannot locate relevant recommendations. Understanding shifts to a Findable need. Design exploration focuses on navigation.</p><p>Refuted (motivation): Users understand the score and find recommendations but do not feel compelled to act. Understanding shifts toward a Desirable or Valuable need.</p><p> </p><p>UX metrics this study will generate</p><p>Comprehension (interview-assessed baseline)</p><p>Initial observation data toward Recommendation Engagement baseline</p><p> </p><p>Estimated effort: 1 researcher, 2 weeks</p></td></tr></tbody></table>
What to notice:
• 'We believe users may need...' is the right opening for an Emerging proposal. 'May' signals that the team is testing a direction, not confirming one.
• The three competing explanations each have a different design implication. Comprehension, findability, and motivation are not interchangeable. The experiment is designed to distinguish between them, not to confirm the one the team finds most intuitive.
• Success criteria names all three outcomes: confirmed, refuted toward findability, refuted toward motivation. Each outcome tells the team something useful and points toward a different next step.
• UX metrics are labeled as baselines to be generated, not as established measurements. At Emerging understanding, metrics are directional.
• Estimated effort is specific: one researcher, two weeks. A stakeholder approving this request knows exactly what they are committing.
Example 2: Building need
The team has completed six interviews. Findability and motivation have been largely ruled out. Four of six participants could not explain the score in their own words. The team is proposing a comprehension measurement study to establish a baseline before design work begins.
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 522px;"><colgroup><col style="min-width: 25px;"><col style="width: 132px;"><col style="width: 121px;"><col style="width: 128px;"><col style="width: 116px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p>Format: Experiment Proposal</p></td><td colspan="1" rowspan="1" colwidth="132"><p>Understanding: Building</p></td><td colspan="1" rowspan="1" colwidth="121"><p>Purpose: Validate</p></td><td colspan="1" rowspan="1" colwidth="128"><p>Tone: Collaborative</p></td><td colspan="1" rowspan="1" colwidth="116"><p>~212 words</p></td></tr></tbody></table>
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 25px;"><colgroup><col style="min-width: 25px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p>User Need: Insightful (Personal) | Understanding: Building</p><p> </p><p>What we believe</p><p>We believe users cannot interpret their Planning score in their own context, and that this comprehension gap is preventing them from evaluating recommendations. Evidence across interviews and behavioral analytics is pointing consistently in this direction. We have largely ruled out findability and motivation as primary causes.</p><p> </p><p>Evidence available</p><p>Behavioral: Users dwell on the Planning score without navigating to recommendations. Pattern consistent across six interview sessions.</p><p>Attitudinal: Four of six interview participants could not explain the Planning score in their own words. Two participants who understood the score still hesitated on recommendations, but for different reasons.</p><p>Measurable: No comprehension measurement yet established. Recommendation engagement under 2% from analytics.</p><p> </p><p>What we do not yet know</p><p>1. The current comprehension rate at scale, not just across six interviews.</p><p>2. Whether the comprehension gap is specific to the Planning score or present across other score types.</p><p> </p><p>Remaining competing explanation</p><p>Comprehension is the primary gap but users who do understand the score may still face a motivation barrier. The two factors may co-exist. This study will help isolate comprehension as the primary driver before design work begins.</p><p> </p><p>Proposed approach</p><p>Comprehension measurement on the Planning score display using a structured comprehension instrument. Target: 40 to 60 users from the active dashboard population. Measure: ask users to explain in their own words what their Planning score means and what would change it. Score responses on a defined comprehension rubric.</p><p> </p><p>Success criteria</p><p>Confirmed: Comprehension rate at scale is below 60%. Understanding strengthens to Stable. Design sprint on Planning score comprehension is approved and scoped.</p><p>Partially confirmed: Comprehension rate is between 60% and 74% benchmark. Understanding remains Building. Team investigates whether the gap is score-specific or universal before committing to a full sprint.</p><p>Refuted: Comprehension rate is at or above 74% benchmark. Team revisits the motivation explanation and designs a different study.</p><p> </p><p>UX metrics this study will generate</p><p>Comprehension (scale baseline, Planning score)</p><p> </p><p>Estimated effort: 1 researcher, 1 week</p></td></tr></tbody></table>
What to notice:
• 'We believe...' without 'may' reflects the stronger Building state. The team has evidence pointing in a clear direction, not just early signals.
• 'We have largely ruled out findability and motivation as primary causes.' This is a significant statement that the Emerging proposal could not make. Building understanding earns the right to narrow the field.
• The remaining competing explanation acknowledges that comprehension and motivation may co-exist. Honest about what the study will and will not resolve.
• Success criteria has three outcomes rather than two: confirmed, partially confirmed, and refuted. The partially confirmed outcome is important at Building understanding, where the team may find themselves between states.
• The proposed approach is more targeted than the Emerging version: a structured comprehension instrument rather than open-ended interviews. Building understanding allows for more focused methodology.
• Estimated effort is smaller: one week rather than two, because the question is narrower.
Near-Miss
The most common experiment proposal failure is writing success criteria that only describe confirmation. The team names what evidence would confirm the need but not what evidence would refute it or redirect the investigation. This turns the experiment into advocacy for a predetermined conclusion rather than a genuine test of the understanding.
The wrong version
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 522px;"><colgroup><col style="min-width: 25px;"><col style="width: 132px;"><col style="width: 122px;"><col style="width: 126px;"><col style="width: 117px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p>Format: Experiment Proposal</p></td><td colspan="1" rowspan="1" colwidth="132"><p>Understanding: Emerging presented as advocacy</p></td><td colspan="1" rowspan="1" colwidth="122"><p>Purpose: Validate</p></td><td colspan="1" rowspan="1" colwidth="126"><p>Tone: Persuasive, incorrectly applied</p></td><td colspan="1" rowspan="1" colwidth="117"><p>~118 words</p></td></tr></tbody></table>
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 25px;"><colgroup><col style="min-width: 25px;"></colgroup><tbody><tr><td colspan="1" rowspan="1"><p>User Need: Insightful (Personal) | Understanding: Emerging</p><p> </p><p>What we believe</p><p>Users cannot understand their Planning score and this is preventing them from using recommendations effectively.</p><p> </p><p>Proposed approach</p><p>Five to eight interviews with active dashboard users to confirm the comprehension problem and understand its severity.</p><p> </p><p>Success criteria</p><p>Interviews confirm that users struggle to understand the Planning score. We will use these findings to scope a redesign of the score display.</p><p> </p><p>Estimated effort: 1 researcher, 2 weeks</p></td></tr></tbody></table>
What is wrong:
• 'Users cannot understand their Planning score' is stated as a fact in the 'What we believe' field. At Emerging understanding, this is a belief, not a confirmed condition. The proposal starts by treating the conclusion as given.
• 'To confirm the comprehension problem' in the proposed approach reveals the team's intent: they are running interviews to justify a decision already made, not to test whether the decision is correct.
• Success criteria only describes confirmation. There is no outcome for refutation, no description of what the team would do if the interviews do not confirm the comprehension explanation, and no acknowledgment that other explanations exist.
• 'We will use these findings to scope a redesign' assumes the outcome before the study runs. A study with a predetermined conclusion cannot produce a refutation.
• Competing explanations are entirely absent. The team has not considered that the problem might be findability, motivation, or something else entirely.
Why it happens
Teams that have been watching a pattern for several weeks often feel certain about the explanation before they have validated it. Writing an experiment proposal that genuinely holds the conclusion open feels like admitting doubt in a context where confidence is expected. The proposal becomes a justification document rather than an investigation plan.
The problem is that an experiment designed to confirm rather than test will produce confirmation regardless of what the data shows. Ambiguous findings get interpreted as support. Contradictory signals get dismissed as noise. The team spends two weeks and comes back with the same answer they started with, having learned nothing that could change the direction.
The right version
See Example 1. The right proposal names what the team believes, names the competing explanations that would point toward different design responses, and defines success criteria for all plausible outcomes. If the interviews come back pointing toward findability rather than comprehension, the team knows exactly what to do next.
Agent Operations
This section defines how AI Skills produce experiment proposal outputs for User Needs. Where the sections above are written for practitioners, this section is written for Skills.
Trigger conditions
Produce an experiment proposal when the user asks to validate a user need before committing to design work, or when they use phrases like 'write an experiment proposal', 'research proposal', 'validate this need', 'how do we test this', 'proposal for interviews', or 'what would we need to confirm this.'
Also produce an experiment proposal when the user has a Brief that asks for validation investment and wants the experiment proposal to accompany it.
Required inputs
Before producing an experiment proposal, confirm: the User Needs output being formatted, the current understanding strength (almost always Emerging or Building), and what the team does not yet know. If the team cannot articulate what they do not know, the proposal cannot be written. Ask: 'What question would this study answer that would change what you build?'
Also confirm: what competing explanations the team has considered. If the team has only considered one explanation, surface the most common alternatives for the need type before drafting the proposal.
Production rules
• What we believe: written as a belief for Emerging ('We believe this may be...'), written with more confidence for Building ('We believe...'). Never written as a confirmed fact.
• Evidence available: organized by type, honest about what is missing. If no measurable signals exist, say so.
• What we do not yet know: specific, resolvable questions. Each question should be answerable through the proposed approach.
• Competing explanations: at least two for Emerging, at least one for Building. Each explanation names the design implication if it turns out to be correct.
• Proposed approach: specific method, participant criteria, and estimated effort. Not a full research design, but specific enough for a resource decision.
• Success criteria: required. Both confirmation and refutation outcomes named. Each outcome includes what the team would do next.
• UX metrics: named as baselines to be generated, not as established measurements.
What a Skill must not do
• Write success criteria that only describe confirmation
• State the need as confirmed in the 'What we believe' field
• Write a proposed approach that is framed as 'confirming' rather than 'testing'
• Produce an experiment proposal for a Stable need without flagging that validation experiments are unusual at this state
• Omit competing explanations
• Write an experiment proposal that assumes a specific design outcome regardless of findings
Routing
After producing an experiment proposal, if the user wants to share it with a stakeholder who needs to approve the investigation time, suggest attaching it to a Brief. If the experiment is complete and the understanding has strengthened to Stable, suggest Roadmap for the planning entry or Docs for the research handoff. If the findings refuted the original need and pointed toward a different one, route back to the User Needs Playbook to structure the revised understanding.