Mental Models and the Architecture of Understanding

When people encounter a new system, they do not begin from zero. They immediately start forming explanations. Sometimes these explanations are correct, often they are incomplete, and occasionally they are entirely wrong, yet they still guide behavior. Researchers call this internal explanation a mental model (Johnson-Laird 1983; Gentner and Stevens 1983).

Mental models let people predict what will happen when they act. They provide a sense of structure even when the system itself is complex, and without them interaction would feel chaotic. Cognitive distance emerges precisely when the model a user constructs cannot align with the conceptual structure embedded in the system, which makes understanding how mental models form central to understanding the distance problem.

3.1 How People Build Mental Models

Understanding evolves through repeated cycles of action and feedback, summarized in Figure 3.1.

Humans naturally search for causal explanations

When people use a tool, they instinctively try to understand how it works, and they build explanations even when information is incomplete. Someone using a microwave oven may know nothing about electromagnetic radiation, yet they still develop a working account of how the device behaves: pressing certain buttons produces certain outcomes. Digital systems trigger the same process. When users interact with software, they observe outcomes and form hypotheses. If clicking a button produces a result, they associate the two; if an action produces an unexpected response, they revise their explanation. Over time these observations form a personal model of the system. It may not reflect the system’s true internal logic, but it allows the user to operate within it (Kieras and Bovair 1984).

Experience from the physical world shapes digital expectations

Human reasoning about tools developed long before digital technology existed, and people carry expectations from physical interaction into digital environments. Opening a folder on a computer borrows from the physical act of organizing documents; dragging a file to a trash icon echoes the act of discarding something. Early interface design relied on these metaphors deliberately. Douglas Engelbart’s work on augmenting human intellect, Alan Kay’s vision of personal dynamic media, and the Xerox Star’s desktop of documents and folders all tried to make computers understandable by connecting digital actions to familiar concepts (Engelbart 1962; Kay and Goldberg 1977; Smith et al. 1982).

These metaphors helped people form mental models quickly. As systems have grown more complex, however, the connection to physical experience has weakened. Many modern interfaces rely on abstract concepts with no real-world counterpart, and when that happens users struggle to build reliable models.

Learning through exploration and error

Most people do not read manuals before using software. They explore: they click, observe the result, and gradually infer how the system behaves. The process resembles scientific experimentation in miniature. A user forms a hypothesis about what an action might do, performs it, and observes the outcome. If the outcome matches the expectation, the model strengthens; if it contradicts it, confusion appears. When systems behave consistently, exploration is a powerful way to learn. When they behave unpredictably, users cannot refine their understanding, and the model stays unstable. That instability is one of the clearest signs that cognitive distance is present.

3.2 When Mental Models Break Down

Systems that hide their internal structure

Some systems obscure the logic behind their behavior, showing results without revealing how they were produced. Search engines are a classic example. A user enters a query and receives a ranked list, but the system rarely explains how the ranking was made. Users therefore construct simplified explanations. They may believe certain keywords always produce better results, or assume the first result is always the most accurate. These shortcuts help them navigate, even though the true mechanism is far more complex. But if the system changes its behavior, the user’s model can collapse suddenly.

Interfaces that contradict user expectations

A different breakdown occurs when an interface violates intuitive expectations. A user expects the back button to return them to the previous screen. If it instead takes them somewhere else or resets part of their progress, their model becomes unreliable. These small violations accumulate. After several unexpected outcomes, users lose confidence in their understanding and stop reasoning through the interface. Instead, they rely on memorized sequences, following steps they learned earlier without knowing why those steps work. The shift is subtle but important: the system has become operable rather than understandable.

Terminology that reflects system logic rather than human language

Language plays a powerful role in shaping mental models. When labels reflect the vocabulary of developers or organizations rather than users, interpretation becomes difficult. Consider an option labeled “data reconciliation module”. An engineer knows exactly what it means. A user who simply wants to check their financial information has to translate the system’s language into their own, and every such translation adds to the distance. This is why usability guidelines insist on using the user’s language rather than the system’s; Jakob Nielsen’s heuristic of a “match between system and the real world” states the principle directly (Nielsen 1994).

3.3 Mental Models in Complex Systems

The challenge of invisible processes

Modern systems rely on processes that operate far beyond the visible interface. Cloud computing, machine learning models and distributed data processing add layers of complexity that users never see. From the user’s side, the system simply produces outputs while the reasoning behind them stays hidden, so people must guess. They may attribute human-like reasoning to algorithmic processes, or assume the system understands their intentions when it is following statistical patterns. That gap between perceived intelligence and actual mechanism increases cognitive distance dramatically.

Artificial intelligence and unstable models

Generative AI adds another layer (Zhang, Wang, and Yi 2025). These systems produce responses that appear coherent and conversational, and users often read that fluency as evidence of deep understanding, when the underlying mechanism is a statistical model trained to predict likely continuations of text (LeCun, Bengio, and Hinton 2015). However powerful such systems are, their internal logic is rarely transparent to everyday users, and the mental models people build of them are unstable. Users believe they understand the system until it behaves unexpectedly. When it gives a wrong answer or contradicts itself, their explanation collapses and the interaction suddenly feels unreliable.

Trust and cognitive alignment

Mental models are closely tied to trust. When people believe they understand how a system behaves, they feel confident predicting its outcomes, and that confidence supports consistent use. When they cannot build a stable model, trust weakens. They may keep using the system, but cautiously: they double-check results and hesitate before relying on automated decisions (Lee and See 2004). In high-stakes domains such as healthcare and financial services, that uncertainty can have serious consequences. Reducing cognitive distance therefore matters beyond usability. It affects how reliably people can work with increasingly powerful systems.

3.4 The Designer’s Challenge

Designers face a difficult task. They must create interfaces that let people build accurate mental models without exposing the full complexity underneath. Too much technical detail overwhelms; too little explanation leaves people guessing. The goal is not to teach users the entire architecture of a system. It is to provide conceptual structures that let them predict outcomes and understand the logic of their interactions. That balance requires careful decisions: metaphors chosen thoughtfully, terminology drawn from real user language, and interaction patterns that behave consistently. When these elements align, users develop mental models that mirror the system’s structure. When they do not, cognitive distance grows.