Using performance efficiency for testing and optimization of visual attention models

When developing a predictive tool for human performance one needs to have clear metrics to evaluate the model's performance. In the area of Visual Attention Modeling (VAM) one typically compares eye-tracking data collected on a group of human observers to the predictions made by a model. To evaluate the performance of these models one typically uses signal detection (Receiver Operating Characteristic (ROC)) that measures the predictive power of the system by comparing the model's predictions for an image to human eye tracking data. These ROC curves take into account the model's hit and false alarm rates and by averaging over a set of test images provides a final measure of the system's performance. In releasing a commercial visual attention system, we have spent considerable effort in developing metrics that allow for regression testing, that are useful for optimizing our visual attention model that takes into account the Upper-Theoretical Performance Limit for an image or classes of images. We describe how the Upper- Theoretical Performance Limit is calculated and how regression testing and parameter optimization benefit from this approach.