Computer aided lesion detection, segmentation and characterization on mammography and breast MRI

Breast cancer is one of the most common cancers among females worldwide. Early detection and diagnosis is the most effective way to reduce mortality. Mammography and breast magnetic resonance imaging (MRI) are two commonly used imaging modalities for breast cancer detection and diagnosis. Mammography is the primary screening tool for breast cancer detection. Breast MRI is often used as an adjunct to mammography for diagnostic purposes after suspicious findings are spotted on mammography. However, there are difficulties in the interpretation of both imaging modalities. Mammography is commonly used in population-based screening programs, and the large number of mammograms can be overwhelming to human readers given the shortage of radiologists in Australia. Breast MRI is difficult to interpret since it generates multiple anatomical and dynamic contrast enhanced (DCE) sequences from its 4D data acquisition, and it can be extremely challenging to visually analyze both enhancement kinetics and morphology of lesions at the same time. To assist radiologists in breast image interpretation, breast computer aided detection/diagnosis (CAD) systems have been developed to improve work efficiency and the confidence of decision.Breast lesion detection and segmentation are challenging since breast lesions vary in size, shape and contrast to the background. Previous studies mostly regard lesion detection and segmentation as separate tasks, only focusing on either detection or segmentation. Some others complete the workflow by stacking a segmentation stage onto a detection stage. However, the segmentation is often only performed on the true positive detections manually confirmed by the user, ignoring the false positive detections. It could be more helpful to the radiologists if the CAD systems have the ability to visualize the detection and segmentation results altogether, since shape and margin are important features for human readers to make detection and diagnostic decisions.This thesis presents a series of fully automatic mammographic CAD systems that integrate mass detection and segmentation, and a breast MRI CAD system that combines lesion detection, segmentation and characterization in one framework.The proposed mammographic CAD systems can detect and segment masses simultaneously without user intervention. Both the traditional workflow which involves unsupervised region candidate generation and hand-crafted features, and the deep learning (DL) based workflow are explored. A novel unsupervised region candidate generation technique, multi-scale morphological sifting (MMS), is proposed. The MMS uses morphological filters with oriented linear structuring elements (LSEs) to sieve out mass-like patterns including linear spicules that are normally present in breast masses. This technique can accurately extract masses as region candidates. Class imbalance is an issue that hinders the classification of masses, as the normal region candidates often outnumber the mass candidates. A novel cascaded random forests (CasRFs) and the use of random under-sampling boost (RUSboost) are proposed to tackle this problem. A novel DL-based framework that uses pseudo-color mammograms generated by the MMS and the Mask R-CNN is also proposed. A pseudo-color mammogram is composed with the original grayscale mammogram in one channel and two MMS enhanced images in the adjacent channels, which provides color contrast between masses and background tissue. The mammographic CAD based on the combination of pseudo-color mammograms and the Mask R-CNN achieves the best performance, yielding an average true positive rate (TPR) of 0.90 at 0.9 false positive per image (FPI) for mass detection and an average Dice similarity index (DSI) of 0.88 for mass segmentation. Evaluated on the same datasets as several previous studies, the mammographic CADs outperform the state-of-the-art CADs in both lesion detection and segmentation in a much simpler architecture.The proposed breast MRI CAD can simultaneously detect, segment and characterize lesions as malignant or benign. The MMS is extended into 3D MMS which can segment lesions as region candidates more accurately and efficiently than the brute force searching methods commonly used in previous studies on breast MRI images. Provided with accurately segmented lesion candidates, the system combines hand-crafted analytical features derived from all available image sequences, including the T1- and T2-weighted anatomical sequences and the DCE sequences, to achieve a comprehensive analysis of the morphology and contrast enhancement kinetics of the lesions. Hand-crafted features are adopted since it is still challenging for the DL-based methods to directly handle 4D multi-modal breast MRI data. Evaluated on the same dataset, the breast MRI CAD outperforms the state-of-the-art methods in lesion detection and identifies malignancies additionally, yielding a TPR of 0.90 at 3.19 false positives per patient (FPP) and a median DSI of 0.77 for lesion detection-segmentation, and a TPR of 0.91 at 2.9 FPP for malignancy identification.