1. Introduction
The rapid development of Artificial Intelligence (AI) systems has intensified the scientific debate around their reliability, robustness, and alignment with fundamental principles that enable their responsible integration into sociotechnical environments (de Brito Duarte et al., 2023). In this context, the notion of Trustworthy Artificial Intelligence (TAI) has emerged as a key interdisciplinary concern, bringing together technical, ethical, and cognitive considerations related to how systems behave and how this behaviour is interpreted by users (Hoffman et al., 2023).
Despite broad recognition of its importance, advancing toward truly TAI remains a challenging task. Technical issues, such as the need for more interpretable models, robust risk-management mechanisms, or effective bias-mitigation strategies, coexist with organizational and human-centred difficulties (Brdnik & Šumak, 2024). The trustworthiness of a system critically depends on how end-users understand its capabilities and limitations, the expectations generated by its outputs, and the dynamics that unfold during real-world interactions (Haque, Islam & Mikalef, 2023). These users often interact with AI systems without a detailed understanding of their underlying mechanisms, making them especially vulnerable to misinterpretation, overreliance, or mistrust (Bernardo & Seva, 2023; Habiba et al., 2024; Umm-E-Habiba & Habibullah, 2024).
Within this landscape, advances in Explainable AI (XAI) have become especially relevant, as the explanations accompanying system behaviour can help users understand why a recommendation was produced and evaluate its reliability (Cabour et al., 2021; de Brito Duarte et al., 2023). Despite the increasing research interest in XAI, a significant portion of this research targets technical experts or developers, while the cognitive, emrotional, and contextual factors that shape end-user trust remain insufficiently explored (Haque, Islam & Mikalef, 2023; Myakala, Jonnalagadda & Bura, 2025). As a consequence, practitioners lack actionable guidance for designing XAI systems that effectively support trustworthy interactions for these users.
This article aims to identify the key factors that contribute to gaining and sustaining non-expert end-user trust in AI systems, and to outline design considerations or guidelines for designing XAI systems that support such trust. Through on a review and analysis of recent studies, it seeks to contribute to the unification of existing literature by offering an analytical framework that can guide the development of new AI solutions—or the improvement of existing ones—through a lens of explainability aligned with ethical principles and the real needs of non-expert users.
2. Theoretical framework
This section presents the key concepts required to properly contextualize the subject of study. It defines terms such as trust, TAI, and XAI.
2.1 TAI
Trust is a psychological disposition involving a positive expectation regarding the future behaviour of another agent, even when one is vulnerable to that agent’s actions (Rousseau et al., 1998). In the context of AI systems, trust implies the belief that a system will behave in a predictable, safe, and human-aligned manner (Afroogh et al., 2025). However, such trust should not be based solely on superficial impressions or apparent outcomes, but rather on a deep understanding of how and why an AI system makes decisions (Miller, 2019; Liao et al., 2020).
The AI literature has incorporated a taxonomy that distinguishes between different forms of human trust in automated systems. As Jacovi et al. (2021) explain, intrinsic trust is grounded in the observable reasoning processes of the AI and the types of explanations it provides. In contrast, extrinsic trust arises from external cues, such as the provider’s reputation, the perceived quality of the user interface, or the visible outcomes the system delivers. Another relevant distinction is between justified trust—where trust in the AI corresponds accurately to its actual ability to fulfil a predictable behavioural contract—and unjustified trust, where such alignment is absent and users believe in the system’s reliability without sufficient basis. Finally, it is important to distinguish between cognitive trust, which is based on a rational analysis of evidence and capabilities, and emotional trust, which stems from subjective perceptions formed through interaction, even without technical understanding of the system (Glikson & Woolley, 2020).
The concept of Trustworthy AI (TAI) has been strongly promoted by the European Union, among other international actors, in response to the increasing autonomy and opacity of AI systems. According to the High-Level Expert Group on AI (AI HLEG) of the European Commission, AI is considered trustworthy if it complies with the law, respects fundamental ethical principles, and is safe and reliable throughout its lifecycle (
European Commission, 2019). Within the TAI framework, trust is viewed as a contractual transaction (Jacovi et al., 2021), in which a user (trustor) expects the AI system (trustee) to fulfil a set of explicit or implicit expectations (contract).
The need for TAI arises from its growing influence over critical decisions that significantly impact people’s lives, such as medical diagnoses, judicial processes, or credit evaluations. Without guarantees of transparency and fairness, there is a risk that these systems will perpetuate injustices or errors, undermining public trust and limiting their beneficial adoption.
Users often trust AI based on its observable outcomes, a form of extrinsic trust. However, this trust can be misleading if the internal decision-making process behind those results is not understood. Intrinsic trust, rooted in knowledge of the system’s reasoning and internal functioning, is essential for a comprehensive assessment of its reliability. The absence of intrinsic trust can lead to unjustified trust, where users accept AI decisions without scrutiny, potentially overlooking underlying errors or biases. To prevent this, it is necessary to develop systems that foster both extrinsic trust (through verifiable and consistent outcomes) and intrinsic trust (via explainability, auditability, and human-centred design). Only when both forms of trust converge and are calibrated to the system’s actual capabilities can we speak of justified trust and, consequently, of truly trustworthy AI (Vianello et al., 2023).
2.2 XAI
Explainable Artificial Intelligence (XAI) has emerged in response to the increasing complexity and opacity of AI models—particularly those based on deep learning—which often function as “black boxes” that are difficult to interpret. The need to understand and explain the decisions made by these systems has driven the development of XAI, which aims to provide comprehensible interpretations of the internal workings of AI models, thereby promoting transparency and trust in their outputs (Markus et al., 2021). XAI seeks to answer questions such as “Why was this decision made?”, “How was this result reached?”, and “What factors influenced the outcome?”, enabling human users to understand, trust, and effectively manage AI systems. Thus, XAI can be defined as the set of methods and processes that make the outcomes of an AI system understandable to humans without compromising performance (Longo et al., 2024).
XAI and TAI are intrinsically connected. While TAI focuses on ensuring that AI systems are transparent, robust, and respectful of data privacy, XAI provides the necessary methods and techniques to achieve that transparency and comprehension. In other words, XAI acts as a key enabler for realizing the goals of TAI, allowing users to understand and justifiably trust the decisions made by AI systems (Chamola et al., 2023). TAI depends on XAI as an operational component, but its full realization also requires the integration of interpretability-oriented design and adherence to normative and ethical principles (Wu et al., 2024).
The reliability of an AI system depends on its accuracy, its transparency, and the user’s ability to understand and audit its decisions. XAI enables access to information about how the system processes data, which variables influence its decisions, and how it justifies its predictions (Inam et al., 2021). This strengthens both intrinsic and extrinsic trust, thereby facilitating justified trust—an essential requirement for the adoption of such systems in critical domains such as healthcare, justice, or finance (Cornacchia et al., 2021; Larasati et al., 2023; Miller, 2019).
Several studies have documented the implementation of XAI in sectors such as medicine, where it allows healthcare professionals to interpret diagnostic recommendations; in finance, where it is used to explain credit decisions; and in the autonomous vehicle sector, where it is necessary to justify navigation and behavioural logic (Hernandez et al., 2021; Markus et al., 2021; Samek et al., 2017). These applications demonstrate XAI’s potential to enhance the acceptance of intelligent systems and reduce the risk of misinterpretation or misuse.
Nevertheless, despite its expansion across sectors, an integrative framework that systematizes all the factors involved in the reliability of AI systems from the perspective of end users has yet to be developed. This article seeks to identify and thoroughly analyse the factors that scientific literature has highlighted as determinants of end-user trust in AI systems.
3. Research Questions (RQ)
This study has two main objectives: (1) to investigate the key factors that generate and sustain trust among non-expert end-users in AI systems, and (2) to identify the considerations or guidelines to be taken into account when designing new XAI systems or updating existing ones. These two objectives are addressed through the following research questions:
Motivation: According to Jacovi et al. (2021), trust in AI cannot be understood merely as a technical or algorithmic phenomenon, but rather as a contractual relationship between the user and the system. This relationship depends both on the observable behaviour of the system and the user’s understanding of its underlying logic. Other studies have shown that perceived transparency, the clarity of outputs, and the system’s level of autonomy are variables that affect the calibration of user trust, potentially leading to either justified or unjustified trust (Glikson & Woolley, 2020). Identifying the specific factors that influence trust—ranging from cognitive to emotional dimensions—is essential for designing AI systems that are both technically efficient and socially acceptable.
Motivation: The literature highlights that many explanations are tailored to technical audiences and are neither comprehensible nor useful for users without specialized knowledge (Liao et al., 2020). Furthermore, studies such as Kim et al. (2024) emphasize the lack of standardized criteria for assessing explanation quality from a user-centred perspective, which hinders the comparability of systems and slows progress toward a unified evaluation practice in XAI. Therefore, it is crucial to identify practical guidelines and design principles that support the development of explainable systems that genuinely respond to users’ expectations, capabilities, and contexts.
4. Research methodology
The purpose of this section is to detail the process followed to conduct the systematic literature review. This study adopted a systematic literature review that followed PRISMA guidelines and used a snowballing procedure (i.e. citation and reference searching) due to the emerging, multidisciplinary, and fragmented nature of the research field related to AI system explainability and the factors that influence end-user trust in such systems. This method is particularly suitable when contributions are dispersed across different disciplines or when theoretical frameworks are not yet well established; it ensures the comprehensiveness, depth, and relevance of the bibliographic corpus (Wnuk & Garrepalli, 2018; Wohlin, 2014).
4.1 Snowballing literature review process
To identify the factors that influence end-user trust in AI systems, the methodological guidelines established by Wohlin (2014) were followed, as illustrated in
Figure 1. Wohlin defines the process as an iterative review based on the systematic tracing of citations both backward (looking at papers cited) and forward (looking at papers citing), starting from an initial set of relevant studies (start set).
Figure 1. Snowballing literature review procedure.
Adapted from Wohlin (2014).
4.1.1 Selection of the initial set of studies (start set)
Step 1: Database selection and search query. To initiate the search, the Web of Science database was selected due to its usefulness in multidisciplinary fields such as XAI and Human-AI Interaction (HAII), as it integrates publications from computer science, engineering, psychology, social sciences, and technology ethics, among others. This provides a solid and diverse foundation from which to explore the state of the art. Additionally, the search was limited to articles published after 2010 to ensure the relevance and currency of the reviewed studies, given that significant advances in AI—especially in deep learning and autonomous systems—have occurred over the past decade (Azizov et al., 2024).
A set of keywords was identified to locate articles related to the topic of research, resulting in the following search query: “trust” AND “AI” AND “end user”.
Table 1 presents the rationale for each selected term.
Table 1. Keywords selected for the snowballing literature review.
| Keyword | Justification for inclusion |
|---|
| Trust | Central axis of the study, as it examines trust as a critical factor in the adoption and acceptance of AI systems. |
| AI | Delimits the technological domain of interest, ensuring the search focuses on AI systems rather than other types of technologies. |
| End user | Focuses the search on studies targeting individuals who interact directly with AI—especially those without technical expertise—thus excluding work aimed solely at developers or technical experts. |
Step 2: Selection of the initial set of articles. The literature search was conducted in January 2025. To ensure the currency and relevance of the studies included in this review, the inclusion criteria outlined in
Table 2 were applied. Specifically, the selection was restricted to articles written in English or Spanish (IC1) and published between January 2010 and December 2024 (IC2), resulting in a total of 68 candidate studies. After reviewing the titles and abstracts (EC1), 48 articles were excluded. Upon reading the introduction and conclusion of the remaining 20 candidates, 14 articles were retained. Two reviewers independently performed all screening stages, resolving disagreements through discussion, with a third reviewer acting as arbiter when necessary. No automation tools were used.
Table 2. Inclusion and exclusion criteria used in the snowballing literature review.
| Code | Description |
|---|
| IC1 | Articles published in English or Spanish. |
| IC2 | Articles published between January 2010 and December 2024. |
| EC1 | Articles that do not address end-user trust or do not include relevant considerations for the design or improvement of XAI systems—for example, studies focused on internal AI mechanisms or developer-oriented explanations. |
| EC2 | Duplicate articles, previously rejected, or already included in the literature review. |
Finally, the selected articles were read in full and subjected to an in-depth analysis to determine their inclusion in the literature review. Seven candidates were excluded, resulting in a final start set of seven articles.
4.1.2 Backward and forward snowballing in iterations
Starting from the initial set of seven articles, two backward and forward snowballing iterations were conducted. The backward iteration involved examining the reference lists of each article in the initial set, while the forward iteration identified studies that cited those articles (Wohlin, 2014). A three-phase selection procedure was applied in each iteration.
In the first phase, a preliminary screening was conducted based on the title and abstract of each publication, classifying them into three categories: YES, NO, and MAYBE. In the second phase, the introduction and conclusion sections of the articles classified as YES and MAYBE were reviewed in order to more thoroughly assess their theoretical and methodological relevance. Finally, the full texts of the selected articles were read in detail. Only those that met the established inclusion criteria were incorporated as relevant studies for the subsequent iteration of the process. Risk of bias was assessed using the Joanna Briggs Institute (JBI) critical appraisal checklist for systematic reviews (Aromataris et al., 2015).
4.2 Results of the snowballing literature review iterations
Across the two backward snowballing iterations, a total of 1,169 articles were identified and examined. After applying inclusion criteria IC1 and IC2, and conducting a preliminary screening based on titles and abstracts according to exclusion criterion EC1, 1,108 publications were excluded for failing to meet the established requirements. In the second evaluation phase, the introductions and conclusions of the remaining 61 articles were reviewed, leading to the exclusion of 46 candidates. Finally, a full reading of the remaining 15 candidates resulted in the exclusion of an additional six documents. As a result, nine relevant articles were selected and incorporated into the final corpus through the backward snowballing procedure.
In parallel, forward snowballing retrieved 4,048 articles that cited the initial set. In the first phase, 11 duplicate documents were removed according to exclusion criterion EC2. Subsequently, 3,944 publications were excluded after applying inclusion criteria IC1 and IC2 and conducting an initial screening based on titles and abstracts in line with EC1. In the second phase, the introductions and conclusions of the remaining 93 articles were reviewed, leading to the exclusion of 48 additional papers due to lack of thematic or methodological relevance. A full reading of the 45 remaining papers resulted in the exclusion of another 26. Ultimately, 19 relevant articles were added to the final corpus of this study.
Table 3 presents a summary of the iterations carried out, detailing the number of references and citations reviewed in each round. We stopped after two backward and two forward iterations because the yield of new eligible studies dropped sharply and no novel concepts emerged. Citation paths increasingly converged on previously included sources, indicating thematic saturation and making further iterations unlikely to substantially change the synthesis.
Table 4 shows the final selection of articles included in this snowballing literature review.
| Iteration | FS/BS1 | Articles rejected (FS) | Articles rejected (BS) | Articles accepted (FS) | Articles accepted (BS) |
|---|
| 1 | 42/418 | 39 | 412 | 3 | 6 |
| 2 | 4006/751 | 3994 | 744 | 12 | 7 |
Table 4. List of articles included in the literature review.
| ID | Title | Author(s) |
Year |
|---|
| P01 | Affective Design Analysis of Explainable Artificial Intelligence (XAI): A User-Centric Perspective | Bernardo, E; Seva, R | 2023 |
| P02 | Improving Trustworthiness of AI Solutions: A Qualitative Approach to Support Ethically-Grounded AI Design | Vianello, A; Laine, S; Tuomi, E | 2023 |
| P03 | An Explainable Artificial Intelligence (xAI) Framework for Improving Trust in Automated ATM Tools | Hernandez, CS; Ayo, S; Panagiotakopoulos, D | 2021 |
| P04 | Determinants Affecting Consumer Trust in Communication with AI Chatbots: The Moderating Effect of Privacy Concerns | Li, JJ; Wu, LR; Qi, JY; Zhang, YX; Wu, ZY; Hu, SB | 2023 |
| P05 | Designing for Student Understanding of Learning Analytics Algorithms | Yeh, C; Cowit, N; Howley, I | 2023 |
| P06 | Improving the User Experience and the Trustworthiness of Financial Services | Cornacchia, G; Narducci, F; Ragone, A | 2021 |
| P07 | Increasing User Trust in Optimisation through Feedback and Interaction | Liu, J; Marriott, K; Dwyer, T; Tack, G | 2022 |
| P08 | Explainable Artificial Intelligence (XAI) from a user perspective: A synthesis of prior literature and problematizing avenues for future research | Bahalul Haque, Najmul Islam, Patrick Mikalef | 2023 |
| P09 | Guidelines for Human-AI Interaction | Amershi, S; Weld, D; Vorvoreanu, M; Fourney, A; Nushi, B; Collisson, P; Suh, J; Iqbal, S; Paul N. Bennett, Inkpen, K; Teevan, J; Kikin-Gil, R and Horvitz E | 2019 |
| P10 | Metrics for Explainable AI: Challenges and Prospects | Robert R. Hoffman, Shane T. Mueller, Gary Klein, Jordan Litman | 2019 |
| P11 | Human-Centered Trustworthy Framework: A Human–Computer Interaction Perspective | Sousa, S; Lamas, D; Cravino, J; Martins, P | 2024 |
| P12 | Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI | Jacovi, A; Marasovic, A; Miller, T; Goldberg, Y | 2021 |
| P13 | Questioning the AI: Informing Design Practices for Explainable AI User Experiences | Liao, QV; Gruen, D; Miller, S | 2020 |
| P14 | Human Trust in Artificial Intelligence: Review of Empirical Research | Glikson, E; Woolley, AW | 2020 |
| P15 | Explainable AI: A Diverse Stakeholder Perspective | Umm-E-Habiba and Habibullah, KM | 2024 |
| P16 | Towards Explainability as a Functional Requirement: A Vision to Integrate the Legal, End-User, and ML Engineer Perspectives | Umm-E-Habiba; Bogner, J; Wagner, S | 2024 |
| P17 | Human-centered evaluation of explainable AI applications: a systematic review | Kim J.; Maathuis H.; Sent D. | 2024 |
| P18 | “do you trust me?”: Increasing User-Trust by Integrating Virtual Agents in Explainable AI Interaction Design | Weitz K., Schiller D., Schlagowski R., Huber T., André E. | 2019 |
| P19 | Meaningful Explanation Effect on User’s Trust in an AI Medical System: Designing Explanations for Non-Expert Users | Larasati R., de Liddo A., Motta E. | 2023 |
| P20 | Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank | Sipos L., Schäfer U., Glinka K., Müller-Birn C. | 2023 |
| P21 | Four Principles of Explainable Artificial Intelligence | Phillips, P; Han, C; Fontana, P; Broniatowski, D; Przybocki, M | 2020 |
| P22 | User trust in artificial intelligence: A comprehensive conceptual framework | Yang, R; Wibowo, S | 2022 |
| P23 | Trust in AI: progress, challenges, and future directions | Afroogh, S., Akbari, A., Malone, E. | 2025 |
| P24 | Trust, distrust, and appropriate reliance in (X)AI: a survey of empirical evaluation of user trust | Visser, R; Peters, T; Scharlau, I; Hammer, B | 2023 |
| P25 | Explainable AI for Cultural Minds | Kopecka, H; Such, J | 2020 |
| P26 | Measurable Trust: The Key to Unlocking User Confidence in Black-Box AI | Palazzolo, P; Stahl, B; Webb, H | 2024 |
| P27 | AI Trust: Can Explainable AI Enhance Warranted Trust? | Duarte, R; Correia, F; Arriaga, P; Paiva, A | 2023 |
| P28 | A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspective | Bach, A; Khan, A; Hallock, H; Beltrao, G; Sousa, S | 2022 |
| P29 | Invisible Users: Uncovering End-Users’ Requirements for Explainable AI via Explanation Forms and Goals | Jin, W; Fan, J; Gromala, D; Pasquier, P; Hamarneh, G | 2023 |
| P30 | On the Influence of Cognitive Styles on Users’ Understanding of Explanations | Riefle, L; Hemmer, P; Benz, C; Vössing, M, Pries, J | 2022 |
| P31 | Towards an Explanation Space to Align Humans and Explainable-AI Teamwork | Cabour, G; Morales - Forero, A; Ledoux, E; Bassetto, S | 2021 |
| P32 | Towards Human-centered Explainable AI: User Studies for Model Explanations | Rong; Y; Leeman, T; Nguyen, T; Fiedler, L; Qian, P; Unhelkar, V; Seidel, T; Kasneci, G; Kasneci, E | 2022 |
| P33 | Exploration of Explainable AI for Trust Development on Human-AI Interaction | Bernardo E.L.; Seva R.R. | 2023 |
| P34 | Establishing Appropriate Trust in AI through Transparency and Explainability | Kim, S | 2024 |
| P35 | Current Trends, Challenges and Techniques in XAI Field; A Tertiary Study of XAI Research | Brdnik S.; Šumak B. | 2024 |
5. Analysis of the literature review results
A total of 35 articles were selected from the two snowballing iterations.
Figure 2 illustrates the distribution of articles by year of publication. This distribution reveals a growing trend in scientific output related to the study’s topic, with a peak in 2023, during which 11 publications were recorded. In previous years, a moderate progression is observed: 3 articles in 2019, 4 in both 2020 and 2021, followed by a slight increase to 5 in 2022. After the 2023 peak, publication output slightly declined to 7 in 2024 and dropped to 1 in 2025, due to the closure of this review in March 2025. This trend reflects a rising academic interest in explainability, trust, and human-AI interaction, particularly over the last four years.
Figure 2. Publications over the years.
Of the 35 articles analysed, 19 were published in scientific journals, making this the primary medium for disseminating research on AI and human trust. Conference proceedings account for 46% of the studies (16 out of 35). Further details on the publication venues included in this literature review are provided in the Extended Data (Wright et al., 2026a).
Subsequently, each article was analysed to determine its relevance to the research questions posed. Studies were eligible for each synthesis when reporting (a) trust factors (RQ1) or (b) XAI design/evaluation considerations (RQ2). Data preparation involved normalising terminology and de-duplicating overlapping considerations. Results were tabulated and visualised. The detailed examination showed that all the reviewed texts addressed both questions to varying degrees, whether from a theoretical, empirical, or methodological perspective.
5.1 Factors influencing end-user trust in AI systems (RQ1)
End-user trust in AI systems is a critical component for their responsible adoption, particularly in sensitive or high-impact domains (Afroogh et al., 2025). As previously discussed, Trustworthy AI (TAI) has emerged as a movement strongly promoted by scientific institutions and regulatory bodies as both an ethical and technical priority, aiming to ensure that AI systems are safe, understandable, and aligned with human values (S. Kim, 2024). Within this context, Explainable AI (XAI) has become a key strategy for facilitating the understanding of algorithmic decisions and appropriately calibrating user trust (de Brito Duarte et al., 2023).
Numerous studies have shown that trust does not rely solely on the technical performance of systems, but also on a combination of individual, contextual, ethical, and design-related factors, such as transparency, interpretability, provider reputation, or the type of explanation provided (Bach et al., 2024; Brdnik & Šumak, 2024). Understanding these factors is fundamental to the design of AI systems that foster safer, more comprehensible, and socially acceptable interactions.
Table 5 presents the set of 29 trust-related factors identified in the literature, along with a brief description of how each influences end-user trust in AI systems.
Table 5. Factors influencing end-user trust in AI systems.
| Ref | Factor | Definition |
|---|
| F01 | Personality
| Individual differences, such as baseline trust tendency or character, affect the predisposition to trust AI systems (Liu et al., 2022; Palazzolo et al., 2024). |
| F02 | Age
| Age influences how AI trust is perceived and adjusted, as individuals from different generations tend to show varying degrees of familiarity, expectations, and tolerance for technological risk (Liu et al., 2022). |
| F03 | Gender
| Studies have shown that gender differences can impact both the predisposition to trust and the way trust is calibrated in relation to AI systems (Liu et al., 2022). |
| F04 | Educational Level
| Educational attainment is positively associated with trust in AI (Yang & Wibowo, 2022). |
| F05 | Sociocultural Context
| Cultural background influences how system autonomy, acceptable risk, and the legitimacy of delegating decisions to AI are conceptualized. Users from different cultures have distinct trust thresholds shaped by social norms and collective values (Jin et al., 2023; Liu et al., 2022). |
| F06 | Prior Experience
| A previous positive experience fosters a favourable attitude toward the system, whereas a negative one generates distrust (Cornacchia et al., 2021; Li et al., 2023). |
| F07 | Digital Literacy
| Users with greater technical understanding perceive less risk, facilitating trust relationships with AI systems (Palazzolo et al., 2024; Yang & Wibowo, 2022). |
| F08 | Adaptability
| The ability to adjust to individual needs and respond flexibly to change. Tailoring explanations to user context enhances understanding and trust (Cornacchia et al., 2021; Hoffman et al., 2019). |
| F09 | Anthropomorphism
| Assigning human characteristics to non-human entities. Human-like traits may enhance trust, though excess may lead to the opposite effect (Cornacchia et al., 2021; Li et al., 2023). |
| F10 | Mental Model
| The internal representation a person has of a system or concept. Understanding these models helps design more effective explanations (Haque et al., 2023). |
| F11 | Transparency
| The quality of being clear and accessible in the information provided. Systems that clearly explain their processes and decisions foster informed trust by reducing uncertainty and increasing perceived control (Bernardo & Seva, 2023; S. Kim, 2024; Yang & Wibowo, 2022). |
| F12 | Interpretability
| The quality of being understandable and easy to explain. XAI models should provide clear explanations to help users understand decisions (Amershi et al., 2019; Haque et al., 2023; Liu et al., 2022). |
| F13 | Interactivity
| The capacity for bidirectional communication between user and system. Allowing users to ask questions or request clarification enhances understanding and trust (Bernardo & Seva, 2023; Hernandez et al., 2021). |
| F14 | Technical Performance
| The system’s ability to meet a need or fulfil a purpose. Perceived precision and stability promote trust, especially in critical or high-risk tasks (Afroogh et al., 2025; de Brito Duarte et al., 2023; Visser et al., 2023). |
| F15 | System Robustness
| The system’s ability to operate without failure and resist adverse conditions. Providing consistent, well-founded responses even in complex situations is key to enhancing user trust (Hernandez et al., 2021). |
| F16 | System Accuracy
| Trust can be assessed by evaluating reliability, accuracy, and potential bias in system performance (Palazzolo et al., 2024). |
| F17 | Perceived Benevolence
| Users are more inclined to trust systems they perceive as fair and acting in their best interest (Visser et al., 2023; Yang & Wibowo, 2022). |
| F18 | Regulatory Compliance
| Adherence to current laws and regulations. Ensuring compliance is essential for avoiding legal and ethical risks (Amershi et al., 2019). |
| F19 | Emotions (anxiety, trust, etc.)
| Affective reactions to stimuli or situations. Accounting for user emotions can enhance explanation acceptance and understanding (Bernardo & Seva, 2023). |
| F20 | System Design (usability and aesthetics)
| Clear, coherent, user-friendly design enhances perceived credibility and thus trust (Liu et al., 2022). |
| F21 | System Safety Assurance
| Involves protection from risks or threats. Ensuring data protection and user support boosts trust (Hernandez et al., 2021). |
| F22 | Application Sector
| Trust levels vary by sector: higher standards are required in healthcare, finance, or justice than in entertainment (Afroogh et al., 2025; Visser et al., 2023). |
| F23 | System Complexity
| Greater perceived complexity reduces understanding and trust. Algorithmic opacity is a common barrier to sustained trust (Glikson & Woolley, 2020). |
| F24 | Task Complexity
| When task logic is hard to understand, trust may be limited unless supported by appropriate explanations (Palazzolo et al., 2024; Visser et al., 2023). |
| F25 | Perceived Use Risk
| The greater the perceived risk (economic, ethical, physical), the higher the user’s demand for trust assurances (Liu et al., 2022). |
| F26 | HAII Type
| The degree of autonomy, control type (manual, supervised, or independent), and perceived agency significantly impact trust development (Glikson & Woolley, 2020). |
| F27 | System Provider’s Reputation
| General opinion based on track record. Backing by reputable organizations reinforces system trust (Hoffman et al., 2019). |
| F28 | Social Norms and Moral Values (ethics)
| Principles guiding moral behaviour. Users assess trust based on how well the system aligns with their ethical values, justice perceptions, and human rights expectations (Jin et al., 2023; Vianello et al., 2023). |
| F29 | Feedback
| Information provided about an action to improve performance. Showing intermediate solutions can help build trust, although excessive feedback may induce false confidence (Haque et al., 2023; Vianello et al., 2023). |
5.2 Considerations for designing or evaluating an XAI system (RQ2)
The design and evaluation of XAI systems require a differentiated approach from that of conventional AI systems, particularly if the goal is to foster adoption that is safe, comprehensible, and aligned with human values. The concept of TAI specifically emphasizes these qualities, highlighting the need for systems that are not only technically robust but also transparent, auditable, and understandable to end users (S. Kim, 2024). XAI represents a key operational pathway within the TAI framework by offering explanatory mechanisms that enable users to build accurate mental models, calibrate their trust, and exercise informed control over automated decisions (Bach et al., 2024; de Brito Duarte et al., 2023). Ultimately, XAI systems aim to endow AI with qualities that support meaningful, appropriate, and socially responsible human-technology interactions.
Based on this premise, 345 design considerations were identified in the 35 articles reviewed during the systematic synthesis. After a process to eliminate redundancies, the final repertoire was reduced to 120 considerations, grouped into nine categories representing the main dimensions of AI system design and evaluation.
Table 6 describes these categories and illustrates several sample considerations, while the full list is available in the Extended Data (Wright et al., 2026a).
Table 6. Categories for considerations for designing or evaluating AI systems.
| Category | Definition | Sample considerations |
|---|
| Type and Form of Explanation | What the explanation should display, at what level of granularity, when, and in what format (textual, visual, counterfactual, multimodal, conceptual, etc.). Includes pedagogical and rhetorical principles shaping the explanatory discourse. | C02 – Consider explanation format as a key design component; C28 – Present intermediate results to allow exploration of solution space
|
| User Experience and Interaction | Covers usability, personalization, cognitive load, emotional support, adaptability, and feedback mechanisms throughout the usage cycle. | C03 – Tailor design to perceived system reliability and prior AI experience; C98 – Foster system understanding through mechanisms that improve user–technology interaction
|
| Trust, Transparency, and Human Oversight | Factors that sustain system credibility, auditability, and the possibility for intervention or correction by experts or end users. | C85 – Establish accessible accountability mechanisms; C114 – Acknowledge transparency as a key driver of trust among non-expert users
|
| Robustness, Safety, and Reliability | Measures to ensure stable operation under failures, adversarial attacks, or changing conditions, along with ongoing performance monitoring. | C44 – Build extrinsic trust using robust and verifiable evaluation schemes; C62 – Design with modularity and adaptability to manage divergent perspectives
|
| Data, Traceability, and AI Lifecycle | Addresses data quality, process documentation, decision traceability, and long-term maintainability. | C15 – Implement well-documented learning lifecycles; C73 – Include key content in explanations: condition, treatment, outcomes, processes, and data
|
| Regulation and Compliance | Adherence to legal frameworks (e.g., GDPR) and audit/accountability requirements imposed by national or supranational directives. | C63 – Promote a holistic approach integrating technical, legal, and experiential aspects
|
| Ethics, Values, and Social Responsibility | Covers fairness, bias mitigation, alignment with collective values, cultural inclusion, and corporate social responsibility. | C06 – Explore trust-building factors through multiple ethical lenses (utilitarian, deontological, virtue ethics); C56 – Consider individual and cultural differences in AI perception
|
| Evaluation Methods and Metrics | Addresses experimental protocols, objective and subjective metrics, and iterative evaluation strategies with real users. | C08 – Enable iterative evaluations with real users in real decision contexts; C46 – Assess trust experimentally through causal manipulation
|
| Usage Context and Organizational Factors | Refers to corporate purpose, anticipated benefits, task–system fit, and sectoral or organizational constraints on deployment. | C55 – Ensure tasks assigned to the system match its actual capabilities; C112 – Consider subjective task value (Expectancy-Value Theory)
|
These considerations address various decision-making levels—from explanation formats to regulatory and organizational frameworks—and reflect a more nuanced understanding of the requirements for genuinely human-centred XAI. Key guidance lines from the analysed studies include:
•
Type and Form of Explanation: Explanatory design must go beyond merely presenting technical rationales (C05, C33, C35). It is crucial to employ accessible and varied formats (C02, C69), such as multimodal explanations and conceptual representations (C118), tailored to users’ cognitive styles (C104, C105) and roles (C24). Explanations should be clear, comprehensible, and grounded in real usage contexts (C22, C49), ideally incorporating examples, counterexamples, and hypothetical scenarios (C50). Interactive explanations are also emphasized (C21), enabling users to explore the system (C28, C76).
•
User Experience and Interaction: From a UX perspective, both cognitive and affective processes should be activated (C01), while avoiding information overload (C31) and fostering curiosity as a driver of engagement (C38). Iterative human-centred design mechanisms (C25) that incorporate direct user feedback (C119) are also essential. Factors such as calibrated trust (C42, C52), perceived risk (C83), and developer reputation (C19) significantly shape the user–system relationship.
•
Transparency, Human Oversight, and Trust: Transparency and human oversight form a central axis. Multiple studies emphasize the need for clear accountability mechanisms (C85), explicit trust contracts (C91), and for the system to declare its limitations (C79). Trust should be viewed as dynamic and situated (C40, C72), with evaluation relying on causal manipulation protocols (C46) and validated multidimensional metrics (C87, C88).
•
AI Lifecycle, Regulation, Ethics, and Evaluation: Concerning the technical and ethical lifecycle of AI, traceability (C13), learning documentation (C15), and ethical principles like fairness, sufficiency, and non-discrimination (C27, C30, C56) are critical. These practices should be supported by comprehensive quantitative and qualitative evaluations (C111) and compliance with legal frameworks such as the GDPR (C108, C99, C63).
•
Usage Context and Organizational Aspects: Finally, the contextual and organizational nature of explanatory design is emphasized. Explanations should be adapted to the application domain (C101), the system’s actual capabilities (C55), and the subjective value users place on tasks (C112). Sociotechnical and ethnographic approaches (C109) can help capture these variables more holistically.
Taken together, these guidelines underscore that explainability should not be reduced to a technical system property. Rather, it must be understood as a relational, situated, and context-sensitive quality. Incorporating these considerations from the early stages of design can lead to AI systems that are more comprehensible, adaptive, and aligned with the expectations, capabilities, and rights of non-expert users.
6. Discussion
This section examines how the study’s key findings connect to broader debates on user trust and explainable AI (XAI). We first explore the interplay between factors influencing end-user trust and design considerations for XAI systems, highlighting how these relationships can guide human-centred AI development. Then, we introduce emerging perspectives that extend beyond current literature, emphasizing dynamic, context-sensitive, and interaction-level aspects of trust, as well as strategies for error handling and trust recovery.
6.1 Relationship between factors influencing end user’s trust and considerations for XAI systems design and evaluation
The results obtained from research questions RQ1 and RQ2 allow for a coherent articulation between the 29 factors influencing user trust and the 120 design considerations of XAI systems, understood as levers for modulating that trust. This interrelationship is fundamental for building AI that is genuinely human-centred and aligned with the principles of TAI.
Firstly, many of the factors influencing end-user trust cannot be directly modified, but they can be meaningfully addressed through the design of explanations. For instance, personality (F01), particularly traits like risk aversion or need for control, can be positively influenced through emotionally adaptive explanations (C61), which activate affective processes and enhance the perceived closeness of the system (C01).
Similarly, prior experience (F06) is a key factor in a user’s initial disposition toward technology and can be incorporated into iterative HCD cycles (C25) that adjust the complexity, modality, and depth of explanations based on users’ existing knowledge.
Another group of factors relates to the user’s ability to understand and interact with the system. Digital literacy (F07), for example, should be considered when selecting comprehensible and human-meaningful metrics (C87, C61), as well as when generating multimodal or progressive explanations. The same applies to interpretability (F12), which depends on the use of appropriate explanatory formats (C02), adaptive interface design (C62), and support in constructing understandable mental models (C43, C39).
Transparency and accountability (F11), among the strongest normative and functional pillars of TAI, can be linked to considerations such as explanation of inputs, outputs, and system boundaries(C49), active transparency (C93), or compliance with regulations like the GDPR (C108), which function as key mechanisms to ensure that users understand what the system does, why it does it, and what its limitations are.
Factors such as system design (F20) and perceived ethical values (F28) suggest the need to move beyond a purely technical view of explainability and adopt a more holistic approach that integrates affective, cultural, and social dimensions. Considerations like ethical principles (C27), justified trust (C42), and regulatory compliance (C108) are particularly relevant here, as they link explanatory design to users’ moral and legal frameworks.
Finally, elements such as perceived benevolence (F17) are shaped by the system’s ability to communicate its intentions transparently, appropriately, and empathetically. Consideration (C42) addresses this challenge by emphasizing the importance of designing for justified trust, thereby avoiding both unjustified scepticism and blind overconfidence.
It is proposed that trust-related factors can serve as structuring criteria for the design and evaluation of explainable systems, enabling the development of XAI that informs, communicates, explains, and engages with the real needs and expectations of end users.
6.2 Emerging perspectives on AI trust and explainability
Beyond the factors and design considerations identified in the literature, our analysis highlights a number of underexplored but essential perspectives for the development of trustworthy and explainable AI systems for end users. In what follows, we outline six key domains that deserve further attention.
(i)
The micro-scale of trust in HAII. Much of the existing research adopts a macro-level approach to trust, focusing on system attributes, user characteristics, or broader contextual elements. However, our findings suggest that trust is also shaped through micro-interactions that occur during real-time engagement with the system. Elements such as response latency, the structure and clarity of information, or the tone of communication can significantly affect how trustworthy the system is perceived to be. For instance, overly fast responses may be interpreted as superficial or insufficiently reasoned, while excessively slow ones may be seen as indicators of inefficiency or incapacity. Similarly, subtle aspects like discourse pauses, the granularity of explanations, or the coherence between user input and system output can serve as cues of competence, transparency, or alignment with user expectations. This perspective positions trust as a dynamic and emergent phenomenon that is sensitive to the interactional details of use, thus emphasizing the importance of designing and evaluating XAI systems at the micro scale, which in turn will impact the overall user experience.
(ii)
The dynamic nature of trust. Trust is neither static nor universal. It evolves in response to both personal and contextual changes. Many of the factors identified in the literature, such as users’ educational background, prior experience with AI, or digital literacy, are subject to change over time. Additionally, broader contextual variables such as regulatory frameworks, cultural norms, or public discourse around AI can shift, altering the landscape in which trust is constructed. Even at a day-to-day level, trust may fluctuate depending on the specific task at hand or the user’s emotional state or level of fatigue. These dynamics pose challenges for the design of long-term trustworthy AI systems: it is not enough to ensure reliability under current conditions; systems must also be resilient and adaptive in the face of evolving user profiles and usage contexts.
(iii)
Context-specific salience of trust factors. Another underexplored dimension is the situational salience of trust-related factors. While it is valuable to map the broad range of variables that can influence trust, not all factors carry equal weight in every situation. The same feature (for instance, the tone of interaction) may enhance trust in one context but undermine it in another. An informal tone might foster rapport with non-expert users but may be perceived as unprofessional or inappropriate in expert or mission-critical environments. Therefore, designers must not only consider a wide set of influencing factors but also identify those that are especially significant or even decisive within a given domain, task, or user profile. Understanding and prioritizing these high-impact factors is crucial for fostering trust where it matters most.
(iv)
Handling errors and recovering trust. A notable blind spot in much of the XAI literature is the implicit assumption that AI systems can or should be error-free. Given the probabilistic nature of many AI models and the complexity of the environments in which they operate, errors are inevitable. In this regard, insights from service science, particularly the concept of service recovery, may offer valuable guidance. Service recovery refers to the strategies deployed to regain user trust following a failure (Van Vaerenbergh & Orsingher, 2016). Prior research shows that a well-managed recovery process can enhance user satisfaction to levels higher than before the error occurred (McCollough & Bharadwaj, 1992). For AI systems, such recovery mechanisms might include post-hoc explanations, automated apologies, opportunities for user correction, or escalation to human oversight. Understanding how trust is damaged and how it can be restored is a critical area of inquiry that remains underdeveloped in the current discourse on explainable AI.
(v)
Social and relational dimensions of trust in AI. Another significant gap in the literature is the limited attention to the social and relational dimensions of trust in human-AI interaction. In related fields such as social robotics or human-robot interaction (HRI), it has been shown that trust is shaped not only by technical competence but also by relational cues. Concepts such as self-other recognition, altruistic behaviour, and the ability to predict human intentions are crucial to the formation of trust in these domains. As AI systems increasingly assume collaborative and proactive roles, particularly in settings that demand teamwork or shared decision-making, these relational dimensions become ever more relevant. Treating AI solely as a functional tool risks overlooking how users emotionally and socially engage with these systems, especially in high-stakes or long-term interactions.
(vi)
Human-centred evaluation methods for trustworthy AI. Finally, there is a clear need for evaluation frameworks that are grounded in human factors, with a particular focus on trust. While technical metrics and algorithmic transparency are important, they offer an incomplete picture of how trust is experienced by end users. Cross-disciplinary insights from human-computer interaction and psychophysiology suggest a range of potentially applicable methods, from heuristic evaluations and usability testing to biometric techniques such as heart rate variability, electrodermal activity, brain activity, or eye-tracking. These methods can provide more nuanced and less biased insights into how trust is formed, maintained, or lost. Adapting and validating such approaches for AI interaction contexts could significantly advance our capacity to assess and optimize trust-related aspects of system design.
7. Conclusions and Future Directions
This study has presented a systematic literature review conducted using the snowballing method, with the aim of identifying, synthesizing, and connecting two key dimensions in the field of explainable and trustworthy AI: on the one hand, the factors that influence end-user trust in AI systems (RQ1), and on the other, the relevant considerations for the design and evaluation of XAI systems (RQ2).
The findings have made it possible to construct a comprehensive and structured view of trust as a multifactorial, situated, and dynamic phenomenon, one whose consolidation depends both on individual user characteristics and on the technical, communicative, and ethical design of the system. The identification of 29 trust-related factors and 121 design considerations, extracted from a refined corpus of 35 reviewed articles, provides an empirical foundation for advancing toward design frameworks that are more responsive to the diversity of users and contexts.
The use of the snowballing approach has enabled the coherent mapping of a highly relevant and interconnected body of scientific literature. Moreover, the results presented lay the groundwork for future research aimed at expanding, validating, and operationalizing the identified links between trust, explainability, and user experience. This review contributes to the consolidation of a perspective that regards explanatory design as a central practice for ensuring the ethical, comprehensible, and safe adoption of AI systems.
7.1 Theoretical contributions
The findings of this research provide a significant theoretical contribution to the fields of XAI and TAI by coherently articulating the factors that influence user trust with the design and evaluation considerations necessary for explainable systems. This articulation moves beyond the fragmented approaches that have dominated the literature in recent years, where normative frameworks remain disconnected, by proposing an integrative vision that conceptualizes trust as a multidimensional construct situated at the intersection of individual, contextual, and design-related variables.
In this regard, trust is not conceived as an inherent property of the system, nor as a mere outcome of its technical performance, but rather as a dynamic and relational process that emerges from the interaction between the user, the system, and the context of use. This perspective aligns with recent approaches in Human-Computer Interaction (HCI) and with cognitive and ethical theories that emphasize the importance of factors such as prior experience, risk perception, transparency, or cultural alignment. Integrating these factors into a systematic framework advances the development of a situated theory of trust in AI systems, where explanatory design plays a central role as a social and affective mediator.
The combination of trust factors and design considerations offers a conceptual bridge between the abstract principles of TAI (such as fairness, safety, explainability, or human oversight) and their translation into concrete, specific, and adaptable design practices for XAI systems. This approach helps address a recognized gap in the literature: the challenge of operationalizing ethical values in a practical and assessable way in real-world usage contexts.
This research contributes to consolidating a richer, more critical, and user-oriented vision of XAI, in which design becomes a structural component in the development of AI systems that aim to be socially legitimate, culturally sensitive, and humanly understandable.
7.2 Practical contributions
In terms of practical implications, this study proposes a linkage between trust factors and design considerations that serves as a set of guiding criteria for decision-making during the early design phases, particularly in defining user profiles, conducting task analysis, and modelling the context of use, as well as in the review and evaluation of existing systems.
From an interaction design perspective, the results highlight the importance of generating explanatory experiences that are adaptive, interactive, and emotionally coherent. This requires moving beyond the paradigm of “static explanation” toward activity-centred models that enable users to explore, compare, intervene in, and, if necessary, correct the system’s decisions.
Regarding evaluation, the study proposes tools to operationalize trust as an observable variable, incorporating both objective indicators (such as reliability, consistency, and traceability) and validated subjective measures (such as comprehension, satisfaction, perceived risk, and perceived usefulness).
Moreover, in sectors where AI is implemented in critical tasks or with vulnerable populations, this research offers concrete principles to ensure the protection of rights, equity, and the safety of end users. The integration of ethical principles throughout the design lifecycle, transparent communication of system limitations, and explicit compliance with regulatory frameworks become practically indispensable for reducing the risk of unfair, biased, or misunderstood decisions.
The study underscores that trust is not an emergent property of technical models, but rather a design responsibility. Trust must be designed, mediated, and evaluated based on the quality of interactions, the clarity of communication, the ethical sensitivity of the system, and its ability to align with users’ values, expectations, and vulnerabilities.
7.3 Limitations
This research presents a series of limitations that must be acknowledged, both to properly frame the scope of the conclusions and to guide future lines of inquiry. It is important to note that the domains of XAI and TAI constitute a dynamic, expansive, and still conceptually emerging field, shaped by the convergence of approaches from multiple disciplines, including computer science, psychology, philosophy, requirements engineering, and human-centred design (HCD). This plurality of perspectives results in fragmentation, which complicates the development of unified and comprehensive guidelines, particularly regarding the design of AI systems adaptable to diverse users, contexts, and capabilities. While this study proposes an integrative approach linking trust factors and design considerations, a fully operational theory of explanatory adaptability has yet to be developed. Such a theory would need to systematically address variables such as the cognitive level, sociocultural background, emotional load, and users’ technological experience.
Secondly, the literature review conducted using the snowballing method also entails inherent limitations. Although this approach allows for the in-depth and coherent exploration of interconnected networks of relevant publications, its citation-dependent nature means that the analysed corpus may not fully capture the most recent or emerging literature, especially that published in interdisciplinary conferences or in non-traditionally indexed formats. Moreover, the selection process is influenced by the initial starting points and the inclusion criteria applied to related studies, which may introduce a degree of thematic or methodological bias. The snowballing strategy inherently depends on citation networks, which may omit unpublished studies or research with unreported or negative findings, potentially limiting the completeness of the evidence base. Although measures were taken to ensure the quality and representativeness of the document set analysed, it cannot be claimed that the set of extracted considerations or influencing factors is exhaustive or definitive.
7.4 Future research lines
Building on the limitations identified, several avenues for future research emerge that could help consolidate and expand the findings presented here.
A key priority is the advancement toward the formulation of comprehensive, operational, and adaptable design guidelines that can be applied across various sectors. These guidelines should link individual factors to specific considerations and enable the integration of the five levels of complexity involved in explanatory design: cognitive, affective, ethical, technical, and situational. To achieve this, it will be necessary to delve deeper into how each of the identified trust factors can be addressed with new, more specific considerations beyond those extracted in this review. This development could be enriched through qualitative interviews with experts in human-AI interaction, usability, algorithmic ethics, or technology regulation, allowing for the collection of expert knowledge not published or not yet structured in the academic literature.
Additionally, this work would benefit from being complemented by participatory empirical studies that incorporate the perspectives of end users with varying levels of experience, capabilities, and cultural frameworks. Such studies would allow for the validation of the proposed considerations under real-world conditions, help identify potential barriers to implementation, and support the adaptation of strategies to specific domains. In parallel, a broader approach is recommended, including longitudinal validations and studies that examine how trust evolves over time and in relation to different forms of representation, transparency, and perceived agency. This would contribute to building a more situated, dynamic, and context-sensitive understanding of trust in AI systems, an essential element for a socially robust and legitimate XAI aligned with the values of Trustworthy AI.
Finally, a strategic line of research involves the development of methodological tools derived from this framework, such as decision matrices, self-assessment instruments for design teams, or support systems for selecting explanatory formats based on user profiles. These tools could facilitate the incorporation of trust and explainability principles from the early stages of technological development, reducing the risk of solutions focused solely on technical or functional criteria.
Ethics and consent statement
Ethical approval and consent were not required for this study, as it is a systematic literature review based exclusively on previously published studies.
