Measuring Fitness and Precision of Automatically Discovered Process Models: A Principled and Scalable Approach

Automated process discovery techniques allow us to generate a process model from an event log. The quality of automatically discovered process models can be assessed with respect to several criteria, including fitness, which captures the degree to which the process model is able to recognize the traces in the event log, and precision, which captures the extent to which the behavior allowed by the process model is observed in the event log. Many fitness and precision measures have been proposed in the literature. However, recent studies have shown that none of the existing measures fulfil a set of intuitive properties. In addition, existing fitness and precision measures suffer from scalability issues when applied to models discovered from real-life event logs. This article presents a family of fitness and precision measures based on the idea of comparing the k-th order Markovian abstractions of a process model and of an event log. We show that our family of measures fulfils the aforementioned properties for suitable values of k. An empirical evaluation shows that our proposed measures yield intuitive results on a synthetic dataset of model-log pairs, while outperforming existing measures in terms of execution times in real-life context.