Guide · From the book

Explainable AI in UX: closing the trust gap

People’s sense that they understand a conversational AI rises quickly, while their grasp of how it works grows slowly. The space between the two is a trust gap, and trust built on it breaks at the first visible error. Explanations that state the outcome, main reason, confidence, what to verify and what to do next help close it.

Why AI widens the distance

Traditional software follows explicit rules people can learn. Machine learning systems work through statistical patterns, and their behavior is harder to predict. Conversational interfaces then hide that complexity behind a single text box and fluent language. People readily treat computers as social actors (Reeves and Nass 1996), so a confident answer reads as evidence of understanding.

A chart over time using the system. A red curve for perceived understanding rises steeply and levels off near the top. A blue area for grasp of the mechanism rises slowly. The space between them is labelled cognitive distance.

The curves in the figure are illustrative, not measured. The point is their shape: perceived understanding outruns real understanding, and the space between them is cognitive distance.

Why that trust is fragile

Early answers are useful, so people rely on the system for harder tasks. Their trust rests on an incomplete model: they may assume it checks facts or applies logic. Research on automation has long warned that trust which outruns a system’s reliability leads to misuse, and trust below it leads to disuse (Lee and See 2004). When an error becomes visible, miscalibrated trust can collapse all at once.

The explanation ladder

The aim is not to expose every parameter but to give people enough to interpret and act. The book proposes five steps, from disclosing an output to helping the person use it:

  1. Outcome statement. Say clearly what the system produced or decided.
  2. Primary reason. Name the strongest contributing factor in terms the person recognizes.
  3. Confidence qualifier. State certainty, limits and known conditions of uncertainty.
  4. Verification guidance. Suggest what to check before acting.
  5. Next step. Offer a concrete way to refine the request or get an alternative.

Explanations must match how people reason

An accurate technical explanation can still be impossible to follow. “Because you watched this film” works because it ties the output to something the person knows. Studies of end users found that sound and complete explanations improve mental models, while oversimplified ones can leave people confident but wrong (Kulesza et al. 2013).

Design moves that help

  • State what the system can and cannot do, and how well (Amershi et al. 2019).
  • Show reasoning cues, such as how a request was interpreted or why an item was recommended.
  • Invite people to question, refine and compare results instead of accepting the first output.
  • Test whether explanations work: ask people to predict and explain the system’s behavior, and watch for high confidence with low accuracy.

Questions people ask

What is the trust gap in AI?

The difference between how well people feel they understand an AI system and how well they actually understand how it works. Trust built on that gap is fragile and can collapse after a single visible error.

What makes a good AI explanation for users?

It states the outcome, the main reason in familiar terms, how confident the system is, what the person should verify, and what they can do next. It should also be tested with users, not assumed to work.

Is explainable AI only a technical problem?

No. Model interpretability methods help, but an explanation only reduces distance if it connects the system’s behavior to concepts the person already understands. That is a design problem.

Read more in the book

This guide is drawn from From Cognitive Load to Cognitive Distance by Izaias Cavalcanti. The whole book is free to read online and as a PDF.

Cite this guide

Cavalcanti, I. (2026). From Cognitive Load to Cognitive Distance: Reframing Usability and Accessibility in Information Systems. https://book.izaias.xyz/