Summary of: Adaptive Metamorphic Testing with Contextual Bandits

Metamorphic Testing (MT) is a software testing paradigm that aims at using user-specified properties of a program under test to either check its expected outputs or to generate new test cases [1] , [2] . More precisely, MT tackles the so-called oracle problem which occurs whenever predicting the expected outputs of a system is just too difficult or even impossible. A typical example where MT has been successfully deployed is for testing machine learning models. For instance, in supervised machine learning, we train models for classification problems, but testing these models is hard as only stochastic behaviors of these models can be specified [3] . Indeed, we initially train these models with existing labelled datasets and then we exploit them to classify new data samples. Testing these models means only to reserve some portion of the labelled datasets to control that the correct classification is given for these reserved datasets. However, nothing is really available to test these models on unlabelled data samples.