[Image]
This experience sits at a moment where work either continues or stalls. Users have taken action, hit an error, and need to decide whether they can recover on their own or if they’re blocked. For the business, this moment directly affects productivity, trust, and how often users need outside help to keep moving.
We tested a Snowflake worksheet error state that appears after a failed SQL query. Participants were asked to review the message and share what they thought caused the error, how they felt about it, and what they would do next. The test focused on success, comprehension, sentiment, and intent to surface how clearly the error communicates cause, confidence, and next steps.
Error states like this often reveal gaps that don’t show up during successful flows. Small uncertainty here can turn into hesitation, support tickets, or abandoned work. Understanding how people interpret failure helps teams see where clarity holds and where confidence breaks, especially in high-effort, high-stakes moments where users need the system to guide them forward.
User Needs & Business Goals
This experience sits at a fragile moment where users need clarity to keep moving, while the business needs to reduce friction, support costs, and stalled work. Users want to understand what went wrong and what control they have next, and the product aims to help them resolve issues without breaking momentum or trust.
Audience
This concept was tested with data practitioners and technical users, primarily based in the United States, who work with Snowflake worksheets. Participants reviewed a worksheet screen showing a failed SQL query and an error message. They were asked to interpret what caused the error, how it made them feel, and what they would do next.
User Needs
In this moment, users are trying to recover from an interruption and decide how to move forward without wasting time or confidence.
-
The experience should clearly explain what happened so users can orient themselves quickly (intuitive).
-
The experience should make it easy to understand the error message and its implications (understandable → usable).
-
The experience should feel trustworthy and accurate, especially when pointing to permission or access issues (credible).
-
The experience should help users find the right next step without hunting for answers elsewhere (findable).
-
The experience should help users feel supported rather than blocked when something goes wrong (empowering).
Together, these needs matter because error states are decision points where clarity determines whether work continues or stalls.
Business Goals
From the business side, this experience plays a key role in keeping users productive and confident during moments of failure.
-
Reduce workflow abandonment by helping users recover from failed queries.
-
Lower support and escalation volume related to permission and access errors.
-
Strengthen trust in system feedback by delivering clear, accurate error messaging.
-
Support user self-sufficiency so teams can resolve issues without external help.
-
Maintain long-term engagement by minimizing frustration during high-effort tasks.
When these goals are met, users move forward faster and the business benefits from sustained usage, trust, and lower operational friction.
Choose Metrics to Test Your Failed Query
We tested the Failed Query experience to understand how people make sense of an error and decide what to do next. A focused design stack of UX metrics was selected by mapping core user needs to observable signals. The metrics used in this test were success, comprehension, sentiment, and intent.
Intuitive → Success
In this moment, users are trying to quickly understand why their work stopped. Success captures whether people can correctly identify what caused the error without guessing. It reflects whether the message provides enough clarity for users to orient themselves before moving on.
Usable → Comprehension
After the error appears, users need to make sense of the message and its implications. Comprehension shows how well people understand what the error means and what options are available. It surfaces whether the explanation supports confident interpretation, not just recognition.
Empowering → Intent
Once users believe they understand the situation, they need to choose a next step. Intent captures whether people feel able to act rather than freeze or abandon the task. It signals whether the experience supports momentum after failure.
Credible → Sentiment
Error messages shape trust in the system. Sentiment reflects how the message makes users feel about the product and the situation. It helps reveal whether the experience feels reliable and fair, or frustrating and opaque.
Establish Hunches to Direct Your Testing
We start with hunches to make uncertainty visible before testing. These hunches reflect where teams often feel least confident when something breaks. Turning them into questions helps focus the test on what might slow people down or help them recover.
Example: Snowflake Failed Query
<table xmlns="http://www.w3.org/1999/xhtml" style="min-width: 75px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>Hunch</p></th><th colspan="1" rowspan="1"><p>Question</p></th><th colspan="1" rowspan="1"><p>UX Metric</p></th></tr><tr><td colspan="1" rowspan="1"><p>People may understand that the query failed but not fully grasp why it failed. Permission errors often blur responsibility between the system, the data, and the user, which can weaken confidence.</p></td><td colspan="1" rowspan="1"><p>What do you think caused this error?</p></td><td colspan="1" rowspan="1"><p>Success</p></td></tr><tr><td colspan="1" rowspan="1"><p>The error message may explain the situation without clearly guiding interpretation. Users might read it, but still feel unsure how to make sense of it</p></td><td colspan="1" rowspan="1"><p>How well do you understand what this error message is telling you?</p></td><td colspan="1" rowspan="1"><p>Comprehension</p></td></tr><tr><td colspan="1" rowspan="1"><p>Even when users believe they understand the message, they may hesitate when choosing a next step. That hesitation can signal a lack of confidence, not a lack of options.</p></td><td colspan="1" rowspan="1"><p>What would you most likely do next after seeing this message?</p></td><td colspan="1" rowspan="1"><p>Intent</p></td></tr><tr><td colspan="1" rowspan="1"><p>The tone and framing of the error may affect how users feel about the product. If the message feels abrupt or opaque, trust can soften even when the system is technically correct.</p></td><td colspan="1" rowspan="1"><p>What impression does this error message give you?</p></td><td colspan="1" rowspan="1"><p>Sentiment</p></td></tr></tbody></table>
Together, these hunches aim to evaluate whether the experience supports understanding, confidence, and forward motion when something goes wrong, or whether uncertainty quietly slows users down.
Turn Hunches into Test Questions
Hunches only become useful when they are turned into questions people can actually answer. Pairing each UX metric with a clear question type helps turn uncertainty into observable signals.
**Success (Open-ended interpretation)**
Question type: Open-text response
Example: What do you think caused this error?
**Comprehension (Likert scale)**
Question type: Likert scale
Example: How well do you understand what this error message is telling you?
**Sentiment (Multiple-choice impressions)** Question type: Multiple-choice impressions Example: Which of the following best describes how this error message makes you feel?
**Intent (Multiple-choice next step)**
Question type: Multiple-choice action selection
Example: What would you most likely do next after seeing this message?
Calculate UX Metric Scores from User Feedback
We tested the Failed Query experience to understand how people respond when a SQL query breaks and interrupts their work. In this moment, users are trying to diagnose the problem, regain confidence, and decide how to move forward. The design stack included a mix of behavioral and attitudinal metrics: success, comprehension, sentiment, and intent.
-
Very Good = 90% and above
-
Good = 70%–89%
-
Average = 50%–69%
-
Poor = 30%–49%
-
Very Poor = below 30%
**Success (65% — Average):**
Most users recognized that the error was related to permissions, but fewer could clearly explain the specific cause. Responses often reflected partial understanding rather than confident diagnosis. This suggests people can orient themselves, but not fully confirm what went wrong.
**Comprehension (83% — Good):**
Users generally understood what the message was communicating. The wording and structure helped people interpret the situation without confusion. However, understanding the message did not always translate into certainty about resolution.
**Sentiment (66% — Average):**
Emotional responses were mixed. While the message felt legitimate and system-driven, it also triggered frustration and uncertainty. This indicates the experience maintains credibility, but does not fully reassure users at the point of failure.
**Intent (82% — Good):**
Most participants could identify a clear next step, such as checking permissions or seeking access. Users were willing to continue rather than abandon the task. This shows momentum is preserved, even when confidence is not.
[Image]
Taken together, the scores point to an experience that keeps users moving but leaves them uneasy. Understanding and action hold up better than diagnosis and confidence. The dominant tension is between forward momentum and clarity, especially at a moment where users want the system to help them feel certain, not just informed.
Click here to check out the raw survey data and UX metric scores for Snowflake’s product.
Draw Signals from Your Design Stack
Here’s how signals were surfaced from Snowflake’s Failed Query test results by following five steps:
1. Focus on poorly scoring or imbalanced metrics
[Image]
The overall test score landed at 74% (Good). Comprehension and intent were the strongest signals, while success and sentiment trailed behind. Most people understood the message and could decide what to do next, but fewer felt confident they knew exactly why the error occurred or that the system was helping them resolve it.
Signal: Understanding is present, but diagnostic clarity and confidence break down at the point of failure.
2. Identify patterns across metrics
[Image]
Comprehension and intent reinforce each other. Users can read the message, make sense of it, and choose a next step. Where the experience strains is at interpretation and ownership. Success and sentiment dip because people are unsure whether the problem is theirs, the system’s, or someone else’s. This creates a tension between forward motion and certainty.
3. Determine if user needs are being met
[Image]
-
Intuitive: Partially met — The message is readable, but the cause is not always obvious.
-
Usable: Partially met — Users can move forward, but not always with confidence.
-
Credible: Met — The error feels legitimate and system-generated, not random.
-
Findable: Partially met — Next steps exist, but are not clearly surfaced in the moment.
-
Empowering: Not met — Users often feel blocked rather than supported when the error appears.
4. Compare outcomes to business goals
-
Reduce workflow abandonment: Partially supported — Many users continue, but some hesitate.
-
Lower support volume: At risk — Unclear causes increase the chance of escalation.
-
Strengthen trust in system feedback: Partially supported — Credibility is present, clarity is not.
-
Support user self-sufficiency: At risk — Users may need outside help to resolve access issues.
-
Maintain long-term engagement: Supported — Most users do not disengage outright.
5. Surface signals & establish a direction
Signals derived from the data:
-
Users can interpret the error, but not always diagnose it.
-
Confidence drops when responsibility for the error is unclear.
-
People often move forward despite uncertainty, not because it’s resolved.
-
The experience signals “something went wrong” more clearly than “what to do next.”
Direction based on business context:
[Image]
The evidence points toward an experience that keeps users moving, but not fully informed. Momentum is preserved, but confidence is fragile. Clarifying ownership and resolution paths appears central to reducing hesitation and support dependency.
This is a recoverable experience, not a reassuring one. Users stay engaged, but the lack of diagnostic clarity creates strain at a moment where confidence matters most.
