Measuring User Trust in AI
Why “do you trust this?” is the wrong question to ask when researching AI products.
How do you measure user trust in an AI product?
Measuring user trust in AI products means separating trust attitude, what users say, from trust behavior, what users do. Stated trust rarely predicts reliance: research going back to Lee and See (2004) shows people who say they trust an AI often verify, override, or delegate its output instead. Researchers capture real trust with behavioral probes that ask what a user would do next, revealing calibrated reliance rather than performative confidence.
Picture an unmoderated study of an AI financial advisor. A participant reviews a tax-savings recommendation. They click accept. They tell the screen, with confidence, that they trust the AI's output. The platform logs a successful task and a confident user. The product team gets the readout: users trust the recommendations.
Now picture what the screen recording missed. The participant spent time off-camera, on their phone, running the numbers themselves before clicking accept. The unmoderated platform captured the active desktop window. It did not capture the actual decision-making.
The participant did not trust the AI's output. They verified it. The fact that they verified the output tells us much more than them telling us that they trusted the output.
This is the gap that can break AI product research. Stated trust and trust behavior diverge constantly, and the standard discussion guide is often built almost entirely around stated trust.
Trust attitude versus trust behavior
Two definitions to keep in your back pocket.
Trust attitude
What participants say. "Yes, I trust it." "I feel pretty confident in the recommendation." "I think the AI is reliable." It is what people report when you ask them how they feel.
Trust attitude is performative. People want to seem reasonable, modern, comfortable with technology. They will tell you they trust the AI for the same reasons they will tell you they recycle and exercise: because the alternative makes them sound bad.
Trust behavior
What participants do when something is at stake:
- Accept: take the suggestion without modification.
- Verify: check it against another source.
- Override: change what the AI produced.
- Delegate: route some decisions to the AI, keep others.
It is observable, and it does not lie.
Why most discussion guides miss this
Open up the last AI product discussion guide you wrote. Count how many of your trust-related questions use the words "feel," "confident," or "trust" directly.
If you are honest about the count, it will likely be most of them. The standard formulation gives us questions like "How confident do you feel in this recommendation?", "Do you trust the AI's output here?", "On a scale of 1 to 5, how reliable does this seem?" That formulation comes from twenty years of survey research and Likert scales. It produces clean, quantifiable, satisfying-sounding data.
It also produces data that has almost no predictive power for what users will actually do. Multiple studies in the human factors literature, going back to Lee and See's work in 2004, show that stated trust does not reliably predict reliance behavior under conditions of uncertainty. AI products are conditions of uncertainty. So the gap is real and persistent.
This is not a fixable problem with attitude questions. They are not bad questions; they are questions that measure a different thing than what you need. The fix is to add probes that surface what participants do, not what they think they would do.
Three behavioral probes you can use this week
"Walk me through what you would do next."
Replace "do you trust this recommendation?" with "walk me through what you would do next with this recommendation." The answer surfaces verification behavior, override behavior, or acceptance behavior, depending on what the participant actually does. "I would just go with it" is acceptance. "I would double-check the source" is verification. "I would adjust the number" is override. Each is a behavior you can analyze.
Most participants who say "I trust the AI" will, when asked this follow-up, describe a verification behavior. That is the gap, made visible.
"If the AI suggested something that surprised you, what would you do?"
Replace "would you trust an AI recommendation that seemed off?" with a concrete scenario. "If QuoteAI suggested a price 15 percent higher than you expected, what would you do?" "If the assistant generated a meeting summary that left out something important, what would you do?"
The concreteness matters. Abstract trust questions invite abstract answers. Specific scenarios invite specific behaviors. The participant has to imagine themselves in the moment and tell you what they would actually do, which is much closer to behavior than to attitude.
"Tell me about the last time something like this got it wrong."
This probe trades the hypothetical for the autobiographical. The participant tells you a story about themselves. Stories are far more honest than predictions. People are bad at predicting their own future behavior. They are pretty good at describing their past behavior.
The story they tell will reveal whether they are someone who verifies, someone who overrides, someone who delegates, or someone who has never been burned and so has not yet developed a calibrated relationship with AI tools.
What changes in your analysis
When you switch from attitude probes to behavioral probes, your data changes shape. Instead of a 1-to-5 trust score per participant, you have a behavioral profile. For each session, you can map: did they accept, verify, delegate, or override? Across what kinds of decisions?
The pattern that almost always emerges in my own studies is that the same participant exhibits all four behaviors, depending on what is being recommended. They accept low-stakes suggestions without verification. They verify high-stakes ones. They delegate routine work to the AI. They override the AI in cases where they have strong personal expertise.
This is calibrated reliance. It is the actual goal of AI product design. You want users who reliably distinguish where the AI is helpful from where it isn't, and who behave accordingly. Stated trust gives you no insight into whether your users are doing this. Behavioral observation tells you exactly.
The cost of getting this wrong
If you measure trust as attitude and report a finding like "users trust the AI's recommendations," the product team will use that finding to make decisions. They will deprioritize work on trust calibration features. They will assume the trust problem is solved. They will be confused, six months later, when usage telemetry shows users are heavily underutilizing the AI's suggestions.
If you measure trust as behavior and report a finding like "users verify the AI's high-stakes recommendations and accept the low-stakes ones," the team has actionable design direction. They know what to build for. They know what users are doing, not merely what they say.
The fix is not complicated once you see it. But you have to see it first.
Turn trust behavior into a working discussion guide.
I teach a five-week live course at the UXR Institute, UX Research for AI Products: Methods for a Moving Target. Week 3 is dedicated entirely to the trust-provenance challenge: separating stated trust from reliance, designing behavioral probes, and evaluating provenance over confidence. Capped at 15 students per cohort.
Explore the course →
