Similarity Evaluation between Query and Retrieved Masses using a Content-Based Image Retrieval (CBIR) CADx System for Characterization of Breast Masses on Ultrasound Images: An Observer Study

The purpose of this study is to evaluate the similarity between the query and retrieved masses by a Content-Based Image Retrieval (CBIR) computer-aided diagnosis (CADx) system for characterization of breast masses on ultrasound (US) images based on radiologists' visual similarity assessment. We are developing a CADx system to assist radiologists in characterizing masses on US images. The CADx system retrieves masses that are similar to a query mass from a reference library based on automatically extracted image features. An observer study was performed to compare the retrieval performance of four similarity measures: Euclidean distance (ED), Cosine (Cos), Linear Discriminant Analysis (LDA), and Bayesian Neural Network (BNN). For ED and Cos, a k-nearest neighbor (k-NN) algorithm was used for retrieval. For LDA and BNN, the features of a query mass were combined first into a malignancy score and then masses with similar scores were retrieved. For a query mass, three most similar masses were retrieved with each method and were presented to the radiologists in random order. Three MQSA radiologists rated the similarity between the query mass and the computer-retrieved masses using a nine-point similarity scale (1=very dissimilar, 9=very similar). The average similarity ratings of all radiologists for LDA, BNN, Cos, and ED were 4.71, 4.95, 5.18 and 5.32. The ED measures retrieved masses of significantly higher similarity (p<0.008) than LDA and BNN. Although the BNN measure had the best classification performance (A z : 0.90±0.03) in the CBIR scheme, ED exhibited higher image retrieval performance than others based on radiologists' assessment.