Abstract
Explainable Artificial Intelligence (XAI) has traditionally focused on static, model centric explanations, often limited to simplified forms such as feature attributions or short textual outputs. While useful, these explanations fall short of supporting genuine understanding, as they frequently neglect the implicit knowledge that underlies both human reasoning and natural language communication. This thesis addresses the implicit knowledge challenge in XAI, arguing that explanations must be conceived not as isolated outputs but as dynamic, dialogic processes in which human and machine collaboratively construct understanding. The research unfolds along four interrelated strands. First, it establishes a theoretical and formal foundation for modeling implicit knowledge in explanations, drawing on insights from philosophy of explanation, argumentation theory, and computational linguistics. Second, it tackles data scarcity by developing novel resources: e-RTE-3-it, the first Italian natural language inference dataset with explanations; a domain-specific Italian medical corpus and adapted model; and MedExpDial, a pilot dataset of synthetic explanatory dialogues in the medical domain. Third, it advances evaluation by proposing the GEISER framework, which moves beyond surface-level similarity metrics to assess explanations in terms of their ability to surface novel, relevant implicit knowledge, and validates this approach through the GEESE shared task. Finally, it turns to dialogue, exploring the generation of explanatory interaction with large language models and introducing the IUBAS annotation scheme, the first to systematically capture the explainee’s active role in shaping explanatory exchanges through feedback, requests, and critical questions. Together, these contributions provide conceptual, methodological, and empirical advances towards explainable AI systems that are more transparent, interactive, and human-centred. Through this investigation, we aim to contribute to the understanding and development of AI systems that can not only perform complex tasks but also explain their actions and reasoning in a manner that is truly collaborative and comprehensible to their human partners.