Evaluating Interpretability in Machine Teaching

Building interpretable machine learning agents is a challenge that needs to be addressed to make the agents trustworthy and align the usage of the technology with human values. In this work, we focus on how to evaluate interpretability in a machine teaching setting, a setting that involves a human domain expert as a teacher in relation to a machine learning agent. By using a prototype in a study, we discuss the interpretability definition and show how interpretability can be evaluated on a functional-, human- and application level. We end the paper by discussing open questions and suggestions on how our results can be transferable to other domains.