2021 IEEE SPS Cycle 2 Virtual School on

Networked Federated Learning: Theory, Algorithms and Applications

28.3. - 01.04.2022

70 

Dipl.-Ing. Anna Saranti, TU Graz
Personal Site

Lecture: Tackling the problem of “bad” explanations with the Human-in-the-Loop principle

Abstract: Explainable AI has become a necessity for enhancing transparency, trust, debugging, and support of legal aspects in the use of AI systems. As new neural network architectures emerge, so do Explainable AI methods, tailored to the mathematical aspects thereof and the needs of the users. Although the efforts of researchers to provide even better strategies that cover even more sophisticated aspects of those architectures, new problems arise when the explanations are perceived as “non-intuitive”, “non-reasonable” or “make no sense”. This can have several reasons; among other things the overall performance of the network, the misclassification of the explained sample, or the design of the Explainable AI method that is mathematically sound, but did not take the user’s understanding, requirements and needs into account. The result of that process leaves the human experts irritated by the explanations and the software developers as well as data scientists without any improvement direction. The proposed solution is to bring the domain expert, the human-in-the-loop to question, challenge and act upon the results of the Explainable AI method. By that means, potential faults in each part of the system can be uncovered and the quality, user acceptance and resilience of the overall system are continuously improved.


Presenter: Anna Saranti is a doctoral student at TU Graz, supervised by Prof. Dr. Andreas Holzinger. She obtained a Master degree (with distinction) in Computer Science (TU Graz) with a thesis on “Applying Probabilistic Graphical Models and Deep Reinforcement Learning in a Learning-Aware Application”. She is currently also employed in the FWF Project P-32554 “A reference model of explainable Artificial Intelligence for the Medical Domain" at the Medical University Graz, has been working as a tutor for machine learning courses at both the TU Graz and TU Vienna and also a lector at the St. Pölten University of Applied Sciences. Anna has experience as a software developer and data scientist in several industrial companies.