Chest X-ray Lung Chinese Description Generation based on Semantic Labels and Hierarchical LSTM

The automatic generation of chest X-ray report is a hot research topic at present. Considering the lack of research on Chinese report generation, we propose a method suitable for lung description in Chinese reports–a model that combines semantic labels and hierarchical LSTM. The model analyzes the anomaly report, extracts high-frequency keywords as semantic labels, and adds the abnormal binary classification module in the encoder to correct the results of the semantic labels for the templated characteristics of the Chinese report. In the design of the decoder, to address the problem of lack of correlation between semantic Labels, a two-layer LSTM model that fuses semantic tags and image features is proposed. The comparison with the baseline experiment shows that the proposed model can effectively improve the quality of report generation.