MCNet: Multi-level Correction Network for thermal image semantic segmentation of nighttime driving scene

Abstract Current state-of-the-art image segmentation methods are mainly based on visible spectrum images. However, it remains challenging under adverse environmental conditions (e.g., nighttime, foggy, snowy). Thermal images can be captured in darkness and deployed in any weather and time. But the research focus on thermal images semantic segmentation remains underexploited since the lack of thermal images segmentation dataset of nighttime driving scene. On the other hand, existing methods do not adequately capture context between related pixels and edge details in thermal images. In this work, we focus on thermal image semantic segmentation of nighttime driving scene. First, we propose a multi-level correction network (MCNet) with a multi-level attention module (MAM) and a multi-level edge enhancement module (MEEM). Specifically, MAM selectively captures the inter-class and intra-class contextual dependencies by a multi-level correction process. In addition, MEEM continuously combines precise context information and edge prior knowledge in each level to correct the final feature representation. Second, we present a new dataset called SCUT-Seg which contains 2010 thermal images recorded from various road scenes with 10 manually annotated semantic region labels. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art methods on SCUT-Seg and the public MFNet dataset. The source code and dataset are available at https://github.com/haitaobiyao/MCNet .