Automatic identification of clinically relevant regions from oral tissue histological images for oral squamous cell carcinoma diagnosis

Abstract Identification of various constituent layers such as epithelial, subepithelial, and keratin of oral mucosa and characterization of keratin pearls within keratin region as well, are the important and mandatory tasks for clinicians during the diagnosis of different stages in oral cancer (such as precancerous and cancerous). The architectural variations of epithelial layers and the presence of keratin pearls, which can be observed in microscopic images, are the key visual features in oral cancer diagnosis. The computer aided tool doing the same identification task would certainly provide crucial aid to clinicians for evaluation of histological images during diagnosis. In this paper, a two-stage approach is proposed for computing oral histology images, where 12-layered (7 × 7×3 channel patches) deep convolution neural network (CNN) are used for segmentation of constituent layers in the first stage and in the second stage the keratin pearls are detected from the segmented keratin regions using texture-based feature (Gabor filter) trained random forests. The performance of the proposed computing algorithm is tested in our developed oral cancer microscopic image database. The proposed texture-based random forest classifier has achieved 96.88% detection accuracy for detection of keratin pearls.