TongueNet: Accurate Localization and Segmentation for Tongue Images using Deep Neural Networks

Tongue diagnosis is an important way of monitoring human health status in traditional Chinese medicine. As a key step of achieving automatic tongue diagnosis, the major challenges for robust and accurate segmentation and identification of tongue body in tongue images lay in the large variations of tongue appearance, e.g., tongue texture and tongue coating, caused by different diseases for different patients. To cope with these challenges, we propose a novel end-to-end model for multi-task learning of tongue localization and segmentation, named TongueNet, in which pixel-level prior information is utilized for supervised training of deep convolutional neural network. Firstly, we introduce a feature pyramid network based on the designed context-aware residual blocks for the extraction of multi-scale tongue features. Then, the region of interests (ROIs) of tongue candidates are located in advance from the extracted feature maps. Finally, finer localization and segmentation of tongue body are conducted based on the feature maps of ROIs. Quantitative and qualitative comparisons on real-world datasets show that the proposed TongueNet achieves state-of-the-art performance for the segmentation of tongue body in terms of both robustness and accuracy.