Explainable Artificial Intelligence for Enhancing Transparency in Decision Support Systems

Abstract: Artificial Intelligence (AI) is recognized as advanced technology that assist in decision-making processes with high accuracy and precision. However, many AI models are generally appraised as black boxes due to their reliance on complex inference mechanisms.  The intricacies of how and why these AI models reach a decision are often not comprehensible to human users, resulting in concerns about the acceptability of their decisions. Previous studies have shown that the lack of associated explanation in a human-understandable form makes the decisions unacceptable to end-users. Here, the research domain of Explainable AI (XAI) provides a wide range of methods with the common theme of investigating how AI models reach to a decision or explain it. These explanation methods aim to enhance transparency in Decision Support Systems (DSS), particularly crucial in safety-critical domains like Road Safety (RS) and Air Traffic Flow Management (ATFM). Despite ongoing developments, DSSs are still in the evolving phase for safety-critical applications. Improved transparency, facilitated by XAI, emerges as a key enabler for making these systems operationally viable in real-world applications, addressing acceptability and trust issues. Besides, certification authorities are less likely to approve the systems for general use following the current mandate of Right to Explanation from the European Commission and similar directives from organisations across the world. This urge to permeate the prevailing systems with explanations paves the way for research studies on XAI concentric to DSSs.To this end, this thesis work primarily developed explainable models for the application domains of RS and ATFM. Particularly, explainable models are developed for assessing drivers' in-vehicle mental workload and driving behaviour through classification and regression tasks. In addition, a novel method is proposed for generating a hybrid feature set from vehicular and electroencephalography (EEG) signals using mutual information (MI). The use of this feature set is successfully demonstrated to reduce the efforts required for complex computations of EEG feature extraction.  The concept of MI was further utilized in generating human-understandable explanations of mental workload classification. For the domain of ATFM, an explainable model for flight take-off time delay prediction from historical flight data is developed and presented in this thesis. The gained insights through the development and evaluation of the explainable applications for the two domains underscore the need for further research on the advancement of XAI methods.In this doctoral research, the explainable applications for the DSSs are developed with the additive feature attribution (AFA) methods, a class of XAI methods that are popular in current XAI research. Nevertheless, there are several sources from the literature that assert that feature attribution methods often yield inconsistent results that need plausible evaluation. However, the existing body of literature on evaluation techniques is still immature offering numerous suggested approaches without a standardized consensus on their optimal application in various scenarios. To address this issue, comprehensive evaluation criteria are also developed for AFA methods as the literature on XAI suggests. The proposed evaluation process considers the underlying characteristics of the data and utilizes the additive form of Case-based Reasoning, namely AddCBR. The AddCBR is proposed in this thesis and is demonstrated to complement the evaluation process as the baseline to compare the feature attributions produced by the AFA methods. Apart from generating an explanation with feature attribution, this thesis work also proposes the iXGB-interpretable XGBoost. iXGB generates decision rules and counterfactuals to support the output of an XGBoost model thus improving its interpretability. From the functional evaluation, iXGB demonstrates the potential to be used for interpreting arbitrary tree-ensemble methods.In essence, this doctoral thesis initially contributes to the development of ideally evaluated explainable models tailored for two distinct safety-critical domains. The aim is to augment transparency within the corresponding DSSs. Additionally, the thesis introduces novel methods for generating more comprehensible explanations in different forms, surpassing existing approaches. It also showcases a robust evaluation approach for XAI methods.